Repository navigation
feat(llm): put LLM capture behind --enable-llm-capture - #350
Merged
Merged
Conversation
Capturing LLM traffic adds work to every socket read and write in the kernel (a capture-map lookup, and a destination check on each new connection's first write) and parses captured traffic in userspace. Clusters with no LLM traffic, or no interest in it, should not pay for that, so it is now opt-in, as Node.js and .NET tracing already are. A one-entry config map, set at load time, gates the kernel side: while it is zero, the only cost is one array lookup per read or write. Userspace skips SNI tagging, API-path detection and the LLM metric families. The TLS ciphertext skip and the HTTP/2 fixes stay unconditional: they are correctness fixes that reduce L7 work for everyone.
There was a problem hiding this comment.
Code Review
This pull request makes the LLM capture feature optional and disabled by default, introducing the --enable-llm-capture flag. It optimizes the eBPF probe paths to bypass LLM capture logic when disabled, minimizing performance overhead, and conditionally registers LLM metrics in userspace. Feedback on the changes suggests adding a defensive nil check when accessing the llm_capture_config eBPF map in tracer.go to prevent a potential runtime panic if the map is missing from the loaded collection.
blue4209211
approved these changes
Oct 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #348, which was merged before this commit reached it.
LLM capture adds work to every socket read and write in the kernel (a capture-map lookup, and a destination check on each new connection's first write) and parses captured traffic in userspace. Clusters with no LLM traffic, or no interest in it, should not pay for that. This makes it opt-in, as Node.js and .NET tracing already are:
--enable-llm-capture/ENABLE_LLM_CAPTURE=true, default off.Behaviour change: LLM metrics now require the flag. The Helm chart needs a value for it in deployments that want them.
Validated on a test cluster (18 nodes, Linux 6.8), one build both ways: