Skip to content

dat: sample each case once, show every piece of each archive, type elided data - #3628

Merged
ribbanya merged 1 commit into
doldecomp:masterfrom
ribbanya:pr/dat-elided
Oct 5, 2026
Merged

ribbanya merged 1 commit into
doldecomp:masterfrom
ribbanya:pr/dat-elided

Conversation

@ribbanya

@ribbanya ribbanya commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator

Samples are now chosen across every archive by a samples pick step: for each type and union variant, the fewest instances that together show every case its fields take anywhere in the game (pointers null, relocated, unrelocated or -1; float and integer classes; bitfields; nonzero padding), preferring instances the walk found clean. Each archive's choice is rewritten only when it changes. Sample-mode C goes from 12,130 samples (245k lines) to 347 (13k lines), and every unit is in objdiff, samples or not. MELEE_DAT_SAMPLES_ALL still samples everything.

Elided data is declared as the type the walk reached it as, spelled as the pointers to it spell it (Mtx, HSD_DisplayList[n], u16[n]), instead of UNK_T. MtxPtr, Mtx44Ptr and ROMtxPtr are redeclared with DAT_TYPE in dat_macros.h for the DWARF build, which uses aurora's SDK headers.

Every piece of each archive outside the samples is now its own symbol in the target, and the base infers each piece the walk explains on its own, instead of only public symbols and data the samples point to, each with everything it reaches. So what's inferred no longer depends on what's sampled: units without samples show all their data, and 106 units are complete in both modes. Bytes a pointer says are bytes (texels, strings, keyframes) and command scripts up to their end command count as explained.

A union member condition that can't be evaluated now stops the search instead of falling through to a later member, so ftData_Item uses DAT_IF(1) as its catch-all.

…der per unit

Samples are now chosen across every archive by a `samples pick` step: for
each type and union variant, the fewest instances that together show every
case its fields take anywhere in the game (pointers null, relocated,
unrelocated or -1; float and integer classes; bitfields; nonzero padding),
preferring instances the walk found clean. Each archive's choice is
rewritten only when it changes. Sample-mode C goes from 12,130 samples
(245k lines) to 347 (13k lines), and every unit is in objdiff, samples or
not. MELEE_DAT_SAMPLES_ALL still samples everything.

Elided data is declared as the type the walk reached it as, spelled as the
pointers to it spell it (`Mtx`, `HSD_DisplayList[n]`, `u16[n]`), instead
of `UNK_T`. `MtxPtr`, `Mtx44Ptr` and `ROMtxPtr` are redeclared with
`DAT_TYPE` in dat_macros.h for the DWARF build, which uses aurora's SDK
headers.

Every piece of each archive outside the samples is now its own symbol in
the target, and the base infers each piece the walk explains on its own,
instead of only public symbols and data the samples point to, each with
everything it reaches. So what's inferred no longer depends on what's
sampled: units without samples show all their data, and 106 units are
complete in both modes. Bytes a pointer says are bytes (texels, strings,
keyframes) and command scripts up to their end command count as explained.

Each unit's C is now `src/<unit>.h`, declaring the archive's public
symbols, its samples and what they point to, and `src/<unit>.c` only for a
unit with samples, instead of a header and source per root in a folder.
A unit without samples links its base object from its inferred data alone.

Clippy is in the dev shell, and its warnings in dat-cli are fixed.

A union member condition that can't be evaluated now stops the search
instead of falling through to a later member, so `ftData_Item` uses
`DAT_IF(true)` as its catch-all.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@ribbanya
ribbanya marked this pull request as ready for review October 5, 2026 01:53
@ribbanya
ribbanya merged commit a1184c9 into doldecomp:master Oct 5, 2026
10 checks passed
@ribbanya
ribbanya deleted the pr/dat-elided branch October 5, 2026 01:54
@ribbanya ribbanya added tooling ai-assisted Utilizes a LLM to do the heavy lifting portability Improves non-matching builds labels Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-assisted Utilizes a LLM to do the heavy lifting portability Improves non-matching builds tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant