Repository navigation
Show Julia INTERTHREAD slow consumers and declared interthread in the examples - #62
Merged
Merged
Conversation
…xamples - julia/backpressure (new): a producer task outpaces a slow consumer, whose queue capacity and overflow policy the configuration chooses, beside a monitor subscription that keeps up. It reports what each subscription delivered and dropped, which a goby test harness checks. - julia/basic_multithread and python/basic_multithread declare their interthread traffic in interface.yml, use the generated group constants, and carry a struct or dataclass as a NATIVE message. - Deployments for the Julia and Python multithread examples and the backpressure example, so their tasks and threads are drawn and their required subscriptions validated. - goby_export_interface() passes a Julia or Python target's interthread manifest to goby_clang_tool. These need Goby 3.8 (GOBY_JULIA_HAS_INTERTHREAD_CONTRACT and GOBY_PYTHON_HAS_INTERTHREAD_CONTRACT); with an older Goby the three applications are skipped. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01TCmHErFgxXnJfoD35VZF2d
…is made from thread: on the interprocess entries of the Julia multithread and backpressure examples and the Python multithread example. The binding then allows each call from that task or thread only, and the deployment figures draw it there: the Julia multithread figure shows groups::julia_nav going from SubscriberTask to PublisherTask, rather than looping on the generated C++ class. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01TCmHErFgxXnJfoD35VZF2d
The example is named for the situation it sets up, a consumer slower than its producer, as the other examples are. Its report message becomes InterthreadQueueStats on groups::julia_queue_stats, after the Goby.interthread_stats() it reports. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01TCmHErFgxXnJfoD35VZF2d
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Examples for GobySoft/goby3#488, which adds bounded Julia INTERTHREAD subscriptions (GobySoft/goby3#485), and Julia and Python interthread and outer-layer threads declared in
interface.ymland drawn bygoby interface(GobySoft/goby3#484).Changes
julia/slow_consumer(new). A producer task publishes work at 20 Hz to a slow consumer that takes 0.5 s per item. A monitor task subscribes to the same group on its own task.basic_julia_slow_consumer.pb.cfgchooses the consumer'scapacityandoverflow,drop_oldestas shipped.MainprintsGoby.interthread_stats()and publishesInterthreadQueueStats(new, insrc/messages/julia_queue_stats.proto) ongroups::julia_queue_stats(new).julia_slow_consumer.launch.julia/basic_multithread.interface.ymldeclares the interthread traffic: per task, aNATIVEDepthAlert, andcapacity: 10/overflow: drop_oldeston the subscriber task.interface.ymlnames the task for each interprocess call:SubscriberTaskpublishesgroups::julia_nav,PublisherTasksubscribes to it.multithread.jluses the generatedgroups.navandinterthread(), and sends aDepthAlertstruct instead of aNamedTuple.python/basic_multithread.interface.ymldeclares the interthread traffic per thread, with aNATIVEDepthAlert, and gives thegroups::navpublicationthread: DepthWatcher.multithread.pyuses the generated group constants. The interthread group is renamednav→raw_nav, becausenavclashed with the interprocessgroups::navand the generator drops clashing short names.deploymentslist when those targets are built. Each gets a figure, agoby interface validate --stricttest and alaunch_checktest.groups::julia_navgoing fromSubscriberTasktoPublisherTask, instead of looping on the generated C++ class.cmake/GobyClangTool.cmake:goby_export_interface()passes a target'sGOBY_INTERTHREAD_MANIFESTtogoby_clang_tool -interthread-manifest.test/interprocess/zeromq/julia_slow_consumer.test.yml. Withdrop_oldest, the consumer drops items while the producer and monitor keep their rate. Registered with a 600 s timeout, since Julia compiles on first use.Requires Goby 3.8
The three applications need GobySoft/goby3#488. They are built only when
GOBY_JULIA_HAS_INTERTHREAD_CONTRACTorGOBY_PYTHON_HAS_INTERTHREAD_CONTRACTis set, and are skipped with a status message otherwise. CI installs the released packages, so it will skip them until 3.8 is packaged; the rest of the build is unaffected.Testing
Built locally against a goby3 build of GobySoft/goby3#488, with
-Dexport_goby_interfaces=ON -Denable_tests=ON -Dbuild_python=ON -Denable_julia_examples=ON:julia_interthreadorpython_interthreadsections, and their interprocess entries are on the declared tasks and threads.goby3_example_interfaces_*andgoby3_example_launch_*tests pass for all three deployments.goby3_example_validate_julia_slow_consumerandgoby3_example_test_julia_slow_consumerpass. In the live run the consumer dropped items whileproducedandmonitor_deliveredkept up.gobydwith their interthread and outer-layer thread contracts enforced, with no contract errors or warnings.🤖 Generated with Claude Code
https://claude.ai/code/session_01TCmHErFgxXnJfoD35VZF2d