Document generic (non-CCSDS) packet parsing with a custom generator example

[medley56]

Summary

Closes #190.

The library has always supported non-CCSDS packet formats via user-supplied packet bytes
generators, but there was no worked example of doing so. This adds one, plus the small amount of
documentation needed to make the mechanism discoverable.

New example: examples/parsing_non_ccsds_packets.py

It demonstrates, end to end and with no external data files:

• A made-up sensor format with no CCSDS header at all:
  | SYNC 0xDEADBEEF (4 B) | PAYLOAD_LENGTH (1 B) | COUNTER (2 B) | TEMPERATURE (2 B) |
• A custom packet bytes generator, sync_marker_generator, that finds packet boundaries from a
  sync marker and a packet-defined length field — the "use a custom field in the packet to
  determine its length" case named in the issue.
• That the yielded chunk must span every byte the XTCE container describes, sync marker included,
  or parse_bytes will warn about a bit-count mismatch.
• That a definition whose root container is not named CCSDSPacket must pass
  root_container_name to parse_bytes — load_xtce does not accept that argument.
• Driving the same custom generator through create_dataset via packet_bytes_generator, and
  that packets with no PKT_APID field are grouped under key 0.

The XTCE is embedded inline as a string rather than added as a fixture under tests/test_data/.
The existing non-CCSDS fixtures don't fit (test_xtce_4byte.xml has no length field;
udp_packet.xml is already served by the built-in udp_generator), and a self-contained script has
no data-path dependency when CI runs it. This is the same inline-XTCE pattern the Quickstart in
getting_started.md already uses.

Documentation

docs/source/user_guide/generators.md already carried the conceptual prose ("XTCE is not limited
to representing CCSDS packet structures..."), so this change is scoped to the missing example plus a
tightened generator contract. The "Writing Custom Generators" section now states what a generator
must satisfy (accepts a binary source; yields exactly one packet per iteration; each chunk contains
every byte the container describes; may yield a bytes subclass carrying metadata for
packet_filter), adds the length-field generator case, and links the new script. The
"XTCE is not CCSDS-specific" paragraph was promoted to its own heading so it is linkable. The page
is otherwise unchanged. docs/source/examples.md gets a bullet for the new script.

Testing

The example is exercised by the CI run-examples job, which cd examples and runs every top-level
*.py. That is the test coverage for this change — the example is itself the test, and it fails
loudly if the documented behavior regresses. Verified locally from both examples/ and the repo
root, and under python -W error (clean — no bit-count-mismatch warning). Existing examples all
still run, pre-commit run --all-files passes, the docs build with no new warnings, and the full
suite is 481 passed.

One design note: the example deliberately uses a single concrete container, with no abstract
containers or restriction criteria, because the UnrecognizedPacketTypeError message in
definitions.py interpolates packet['PKT_APID'] and would raise KeyError on a non-CCSDS
definition. That is tracked separately as #276 and is out of scope here.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.62%. Comparing base (fb8b015) to head (e19b666).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #288   +/-   ##
=======================================
  Coverage   94.62%   94.62%           
=======================================
  Files          49       49           
  Lines        4203     4203           
=======================================
  Hits         3977     3977           
  Misses        226      226           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

🟡 Changes recommended

Unresolved moderate socket-handling issues remain, along with requested documentation and regression-test improvements.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds a runnable non-CCSDS packet-parsing example with custom generator documentation.

Changes:

• Adds sync-marker and length-based packet parsing example.
• Documents generator contracts and custom length handling.
• Links the example and updates the changelog.

File summaries

File	Summary	Final comments
examples/parsing_non_ccsds_packets.py	Custom non-CCSDS generator example	Moderate (3 votes): socket handling mismatches the advertised API. Nit (1): add stronger regression assertions.
docs/source/user_guide/generators.md	Generator contracts and usage guidance	Moderate (2 votes): socket handling is inconsistent with the documented source types. Nit (1): the PKT_APID grouping statement is too broad.
docs/source/examples.md	Links the new example	None.
CHANGELOG.md	Adds the unreleased feature entry	None.

Review details

Suppressed comments (3)

docs/source/user_guide/generators.md:118

• If the input ends after a complete sync marker but before its length byte, this indexing raises IndexError instead of taking the documented truncated-stream path. Add a header-length check before reading payload_length, as the runnable implementation does.

        payload_length = buffer[start + len(sync_marker)]

docs/source/user_guide/generators.md:144

• create_dataset does not inspect the parsed PKT_APID; it uses an apid property on the yielded bytes and falls back to 0 only when that property is absent (space_packet_parser/xarr.py:212-216). A custom bytes subclass can therefore route packets with no PKT_APID field to a nonzero dataset, so this statement is too broad.

returns its Datasets keyed by APID, so packets that have no `PKT_APID` field are all grouped under
key `0`.

examples/parsing_non_ccsds_packets.py:165

• This new behavior is only smoke-tested: the loop prints parsed packets and later indexes datasets[0], while CI runs plain python, so a shortened-but-nonempty result or wrong packet values can still pass and warnings are not errors. Add assertions for the packet count/field values and dataset length (or a focused generator test) so the documented boundary and grouping behavior is actually regression-tested.

    for packet_bytes in sync_marker_generator(stream):
        packet = packet_definition.parse_bytes(packet_bytes, root_container_name=ROOT_CONTAINER_NAME)

• Files reviewed: 4/4 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[medley56]

Replying to the three suppressed findings from the Copilot review (no separate threads were created for these):

• docs/source/user_guide/generators.md:118 — Fixed in e19b666: added the missing start + header_length > len(buffer) bounds check before reading payload_length, matching the runnable example's existing guard.
• docs/source/user_guide/generators.md:144 — Fixed in e19b666: reworded to describe the actual behavior — create_dataset keys by the apid property on the yielded bytes object (falling back to 0 when absent), not by the parsed PKT_APID field. Same fix applied to the equivalent comment in examples/parsing_non_ccsds_packets.py for consistency.
• examples/parsing_non_ccsds_packets.py:165 — Fixed in e19b666: added assertions on parsed packet count/COUNTER/TEMPERATURE values and on the resulting dataset length, so the example fails loudly instead of only printing if the boundary or grouping behavior regresses.

🤖 AI-assisted comment, reviewed and approved by @medley56 before posting.