Skip to content

Show Julia INTERTHREAD slow consumers and declared interthread in the examples - #62

Merged
tsaubergine merged 3 commits into
mainfrom
claude/eloquent-mendel-qtcwli
Nov 20, 2026
Merged

tsaubergine merged 3 commits into
mainfrom
claude/eloquent-mendel-qtcwli

Conversation

@tsaubergine

@tsaubergine tsaubergine commented Nov 20, 2026 •

Copy link
Copy Markdown
Member

Examples for GobySoft/goby3#488, which adds bounded Julia INTERTHREAD subscriptions (GobySoft/goby3#485), and Julia and Python interthread and outer-layer threads declared in interface.yml and drawn by goby interface (GobySoft/goby3#484).

Changes

  • julia/slow_consumer (new). A producer task publishes work at 20 Hz to a slow consumer that takes 0.5 s per item. A monitor task subscribes to the same group on its own task.
    • basic_julia_slow_consumer.pb.cfg chooses the consumer's capacity and overflow, drop_oldest as shipped.
    • Once a second, Main prints Goby.interthread_stats() and publishes InterthreadQueueStats (new, in src/messages/julia_queue_stats.proto) on groups::julia_queue_stats (new).
    • Comes with julia_slow_consumer.launch.
  • julia/basic_multithread.
    • interface.yml declares the interthread traffic: per task, a NATIVE DepthAlert, and capacity: 10 / overflow: drop_oldest on the subscriber task.
    • interface.yml names the task for each interprocess call: SubscriberTask publishes groups::julia_nav, PublisherTask subscribes to it.
    • multithread.jl uses the generated groups.nav and interthread(), and sends a DepthAlert struct instead of a NamedTuple.
  • python/basic_multithread.
    • interface.yml declares the interthread traffic per thread, with a NATIVE DepthAlert, and gives the groups::nav publication thread: DepthWatcher.
    • multithread.py uses the generated group constants. The interthread group is renamed nav → raw_nav, because nav clashed with the interprocess groups::nav and the generator drops clashing short names.
  • Deployments for the Julia multithread, Julia slow consumer and Python multithread examples, added to the deployments list when those targets are built. Each gets a figure, a goby interface validate --strict test and a launch_check test.
    • The Julia multithread figure now shows groups::julia_nav going from SubscriberTask to PublisherTask, instead of looping on the generated C++ class.
  • cmake/GobyClangTool.cmake: goby_export_interface() passes a target's GOBY_INTERTHREAD_MANIFEST to goby_clang_tool -interthread-manifest.
  • Test harness: test/interprocess/zeromq/julia_slow_consumer.test.yml. With drop_oldest, the consumer drops items while the producer and monitor keep their rate. Registered with a 600 s timeout, since Julia compiles on first use.
  • READMEs updated.

Requires Goby 3.8

The three applications need GobySoft/goby3#488. They are built only when GOBY_JULIA_HAS_INTERTHREAD_CONTRACT or GOBY_PYTHON_HAS_INTERTHREAD_CONTRACT is set, and are skipped with a status message otherwise. CI installs the released packages, so it will skip them until 3.8 is packaged; the rest of the build is unaffected.

Testing

Built locally against a goby3 build of GobySoft/goby3#488, with -Dexport_goby_interfaces=ON -Denable_tests=ON -Dbuild_python=ON -Denable_julia_examples=ON:

  • The interface files for the Julia and Python applications include their julia_interthread or python_interthread sections, and their interprocess entries are on the declared tasks and threads.
  • The multithread figures draw each task or thread with its interthread and interprocess edges.
  • The new goby3_example_interfaces_* and goby3_example_launch_* tests pass for all three deployments.
  • goby3_example_validate_julia_slow_consumer and goby3_example_test_julia_slow_consumer pass. In the live run the consumer dropped items while produced and monitor_delivered kept up.
  • The Julia and Python multithread examples ran against gobyd with their interthread and outer-layer thread contracts enforced, with no contract errors or warnings.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TCmHErFgxXnJfoD35VZF2d

…xamples

- julia/backpressure (new): a producer task outpaces a slow consumer,
  whose queue capacity and overflow policy the configuration chooses,
  beside a monitor subscription that keeps up. It reports what each
  subscription delivered and dropped, which a goby test harness checks.
- julia/basic_multithread and python/basic_multithread declare their
  interthread traffic in interface.yml, use the generated group
  constants, and carry a struct or dataclass as a NATIVE message.
- Deployments for the Julia and Python multithread examples and the
  backpressure example, so their tasks and threads are drawn and their
  required subscriptions validated.
- goby_export_interface() passes a Julia or Python target's interthread
  manifest to goby_clang_tool.

These need Goby 3.8 (GOBY_JULIA_HAS_INTERTHREAD_CONTRACT and
GOBY_PYTHON_HAS_INTERTHREAD_CONTRACT); with an older Goby the three
applications are skipped.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01TCmHErFgxXnJfoD35VZF2d
…is made from

thread: on the interprocess entries of the Julia multithread and
backpressure examples and the Python multithread example. The binding
then allows each call from that task or thread only, and the deployment
figures draw it there: the Julia multithread figure shows
groups::julia_nav going from SubscriberTask to PublisherTask, rather
than looping on the generated C++ class.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01TCmHErFgxXnJfoD35VZF2d
The example is named for the situation it sets up, a consumer slower than
its producer, as the other examples are. Its report message becomes
InterthreadQueueStats on groups::julia_queue_stats, after the
Goby.interthread_stats() it reports.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01TCmHErFgxXnJfoD35VZF2d
@tsaubergine tsaubergine changed the title Show Julia INTERTHREAD backpressure and declared interthread in the examples Show Julia INTERTHREAD slow consumers and declared interthread in the examples Nov 20, 2026
@tsaubergine
tsaubergine merged commit b07059f into main Nov 20, 2026
5 checks passed
@tsaubergine
tsaubergine deleted the claude/eloquent-mendel-qtcwli branch November 20, 2026 05:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants