Skip to content

feat: migrate to pyxsdata with PEP 562 lazy loading and PolyXML XML acceleration - #192

Open
nth-bailey wants to merge 1 commit into
MMTIS:binary_relation_serializerfrom
nth-bailey:feat/polyxml-ingestion
Open

nth-bailey wants to merge 1 commit into
MMTIS:binary_relation_serializerfrom
nth-bailey:feat/polyxml-ingestion

Conversation

@nth-bailey

Copy link
Copy Markdown

Summary

As discussed in #191, this PR migrates Badger's XML data-binding generation and deserialization stack:

  1. xsdata -> pyxsdata>=1.6.1 with --lazy-load: Generates strongly-typed Python @dataclass models utilizing PEP 562 module attribute lazy loading.
  2. Instantaneous Startup: Replaces the eager domain/netex/__init__.py re-export (which previously loaded ~11,000 classes into memory upon any domain.netex import) with on-demand PEP 562 resolution.
  3. PolyXML Acceleration: Integrates polyxml (native Rust core powered by quick-xml + lexical-core) into MyXmlSerializer.unmarshall, providing native C-speed XML unmarshalling directly into the generated dataclasses, with seamless fallback to pyxsdata.formats.dataclass.parsers.XmlParser.

Benchmarks

1. Package Startup / Import Latency

Operation Before (xsdata eager) After (pyxsdata + PEP 562 lazy) Speedup
import domain.netex 5,418.68 ms 5.56 ms ~974x faster

Eliminates the ~5.4-second freeze whenever any CLI tool, transformer, or module imports domain.netex.


2. XML Deserialization Throughput (5,000 ScheduledStopPoint Objects)

Engine Total Time (5k items) Throughput Relative Performance
XmlParser (pure Python / lxml) 344.1 ms 14,531 ops/sec 1.0x (baseline)
PolyXML (Native Rust) 71.4 ms 70,071 ops/sec 4.8x faster

Verification & Test Results

All 28 unit tests pass cleanly:

Ran 28 tests in 10.892s

OK

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant