Skip to content

dat: implement C deserialization codegen, improve coverage - #3632

Merged
ribbanya merged 3 commits into
doldecomp:masterfrom
ribbanya:pr/dat-native
Oct 5, 2026
Merged

ribbanya merged 3 commits into
doldecomp:masterfrom
ribbanya:pr/dat-native

Conversation

@ribbanya

@ribbanya ribbanya commented Oct 5, 2026 •

Copy link
Copy Markdown
Collaborator

A C library, tools/dat-cli/native, that reads the archives into the game's own types on any platform, from a schema melee-dat native codegen writes from the DWARF. It's a port of the walk that converts as it goes, so melee-dat native expect and the library's trace must agree: they do for all 7048 archives on 64-bit and 32-bit little-endian and 32-bit big-endian (under qemu), and every converted object reads back against its data. The dat preset builds it with the host's compiler and runs its tests in ctest.

  • the schema is a directory of C, generated when configuring: a dat_type_* descriptor for each type, with its members, in a file for the header that declares it (types/<header>.c); roots by module (roots/Pl.c); type indices as an enum (DAT_TYPE_*)
  • descriptors are designated initializers that refer to types by name, with expressions next to their members and annotations as written in comments (dat/schema.h)
  • the walk lays queued objects out as they were reached, so typedef tags apply (the CPU attack lists' DAT_TERMINATED was ignored)
  • CPU attack tables typed (ftCo_AttackList); stage param types moved into their headers; stage param fields typed as their data
  • 47 dat_symbols.txt aliases that read stage data as material animations removed
  • generated C casts only between different types; untyped data typed from the fields that point to it
  • samples report --incomplete, why units aren't complete
  • a native dev shell with the DWARF/dat tools, qemu and a ppc32 cross compiler; the default shell is what CI needs
  • clang-tidy's checks stay on src/: tools/ and the generated schema opt out

🤖 Generated with Claude Code

…on queued objects

A C library, tools/dat-cli/native, that reads the archives into the game's own types on any platform, from tables `melee-dat native codegen` writes from the DWARF. It's a port of the walk that converts as it goes, so `melee-dat native expect` and the library's trace must agree: they do for all 7048 archives on 64-bit and 32-bit little-endian and 32-bit big-endian (under qemu), and every converted object reads back against its data. The dat preset builds it with the host's compiler and runs its tests in ctest.

- the walk lays queued objects out as they were reached, so typedef tags
  apply (the CPU attack lists' DAT_TERMINATED was ignored)
- CPU attack tables typed (ftCo_AttackList); stage param types moved into
  their headers; stage param fields typed as their data
- 47 dat_symbols.txt aliases that read stage data as material animations
  removed
- generated C casts only between different types; untyped data typed from
  the fields that point to it
- `samples report --incomplete`, why units aren't complete
- a `native` dev shell with the DWARF/dat tools, qemu and a ppc32 cross
  compiler; the default shell is what CI needs
- clang-tidy's checks stay on src/: tools/ and the generated tables opt out

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@ribbanya ribbanya added tooling ai-assisted Utilizes a LLM to do the heavy lifting portability Improves non-matching builds labels Oct 5, 2026
ribbanya and others added 2 commits October 5, 2026 12:57
…ype indices

`melee-dat native codegen` writes a directory instead of one tables.c:
each type is a named `dat_type_*` descriptor with its members, in
types/<header>.c for the header that declares it, along with what refers
to it (pointers, arrays); roots are in roots/<module>.c; DAT_TYPE_* and
DAT_NAME_* are enums. Descriptors use designated initializers, refer to
types by name and keep expressions next to their members, with the
annotation as written in a comment where it isn't plain.

- dat/schema.h: the schema's types and the macros descriptors are written
  in; DAT_NONE is 0, so every field defaults to zero
- members, expressions, scripts, binds and roots are pointers, not ranges
  of shared tables; macros expand to a depth when evaluated
- the schema is generated when configuring, again when its inputs change
- codegen split into generator.rs (the rows) and emit.rs (the C)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@ribbanya ribbanya changed the title dat: native archive interface, typed CPU attack tables, typedef tags on queued objects dat: implement C deserialization codegen, improve coverage Oct 5, 2026
@ribbanya
ribbanya marked this pull request as ready for review October 5, 2026 17:12
@ribbanya
ribbanya merged commit d52f449 into doldecomp:master Oct 5, 2026
10 checks passed
@ribbanya
ribbanya deleted the pr/dat-native branch October 5, 2026 17:13
@ribbanya
ribbanya restored the pr/dat-native branch October 5, 2026 17:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-assisted Utilizes a LLM to do the heavy lifting portability Improves non-matching builds tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant