Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/skills/adding-a-command/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: adding-a-command
description: Use when adding a session command to xedit-session so the CLI, the serve daemon and the MCP server all offer it. Covers the request and response structs with their JSON Schema, registering the command (name rules, mutates and dry_run, the edit gate), the CLI subcommand, keeping stdout clean, the tests that pick the command up automatically, the coverage ledger entry, the upstream map and the docs to update.
description: Add an xedit-session command and expose its typed request, response, schema, edit gate, and CLI across CLI, serve, and MCP.
---

# Adding a command
Expand Down
2 changes: 1 addition & 1 deletion .claude/skills/auditing-conflicts/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: auditing-conflicts
description: Use when a load order's record conflicts have to be audited with the native xedit CLI of this repository - which override of a FormID wins and what xEdit's conflict statuses mean, narrowing the list to the records a plugin wins or loses, comparing a record's overrides side by side, the mod groups that hide records from the comparison, and the reference index that shows what a conflicting record is used by. The native replacement for the xEdit-llm daemon's comparison routes (comparisons.records, records.conflict_status).
description: Audit Bethesda plugin override winners, conflict statuses, comparisons, mod groups, and reference usage with the native xedit CLI.
---

# Auditing conflicts
Expand Down
2 changes: 1 addition & 1 deletion .claude/skills/building-patches/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: building-patches
description: Use when a conflict found in a load order has to be resolved by building a patch plugin with the native xedit CLI of this repository - xEdit's "Create Merged Patch" for the leveled lists, containers, factions, form lists and keywords several plugins change, or a purpose-built override patch made with files new, records copy and elements set. The native replacement for the xEdit-llm daemon's patch routes (patches.merge, patches.delta).
description: Resolve Bethesda plugin conflicts with a native merged patch or a targeted override patch using the xedit CLI.
---

# Building patches
Expand Down
347 changes: 39 additions & 308 deletions .claude/skills/checking-parity/SKILL.md

Large diffs are not rendered by default.

3 changes: 3 additions & 0 deletions .claude/skills/checking-parity/references/background.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Checking parity

The official xEdit release build of the tag in `upstream-map.toml` is the oracle. The project has no Delphi compiler, so only released binaries can be the oracle. The port is correct when it produces the same output as the oracle on the same input.
18 changes: 18 additions & 0 deletions .claude/skills/checking-parity/references/checks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
## Checks

| Check | Oracle | Port |
| --- | --- | --- |
| Element dump | `xDump` text output | `xedit dump` |
| Round-trip save | input file bytes | load then save, compare bytes |
| Conflict status | `ConflictAllForMainRecord`/`ConflictThisForMainRecord` of every record by a `-script:` of the GUI (`oracle/conflicts.pas`) | `xedit conflicts` (`conflicts.list`) |
| Cleaning | plugin saved by `-quickautoclean` | `xedit clean` |
| Filter | `ApplyFilter` from the GUI's script mode (`oracle/filter.pas`), its message-log counts | `xedit filter apply` (`parity filter`) |
| Merged patch | the patch "Create Merged Patch" writes in the GUI's script mode (`oracle/merged.pas`) | `xedit patch merged` (`parity merged`) |
| Error checks | the message log of `-CheckForErrors` (the GUI) and of `xDump -check` | `xedit check` (`files.check`) and `xedit dump --check` |
| Saved bytes | plugin written by a `-script:` of the GUI (`oracle/save.pas`, `oracle/edit.pas`) | `xedit save`, `xedit batch` |
| Scripts | compile outcome of every script, and the files, log lines and saved plugins of the form-free corpus and the answered interactive cases (a case's `answers`), by a `-script:` run of the GUI (`parity script`) | `xedit script check`, `xedit script run`, `xedit script resume` |
| Archives | `BSArch.exe pack -mt:no`, `unpack` and `-dump` output | `bsarch`, `xedit archive pack` (`parity bsarch`) |
| Resources | the container list, resource lookups and loader lines of a `-script:` run of the GUI (`oracle/resources.pas`) or of a tool mode | the same command line of `xedit` (`parity resources`) |
| NIF and material dumps and saves | `Sniff.exe` JSON converter and universal tweaker; the GUI's script adapter for text dumps | `cargo xtask parity nif` (in process) |
| Sniff operations | `Sniff.exe` automation mode, outputs and log | `cargo xtask parity sniff` (in process) |
| LOD generation | the GUI's `-lodgen` mode: output files, export files and log | `xedit lodgen` (`parity lodgen`) |
12 changes: 12 additions & 0 deletions .claude/skills/checking-parity/references/close-out-summary.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
## Close-out results (2026-10-18)

The parity backlog's close-out (issue #41, pull request #77) ran every mode at the final tip, one step per mode with the oracle cache shared, and `sniff` one step per game (the crash-carrying step runner of rule 16). The numbers per mode are the README Status table and the report of each run is written to `target/parity/<mode>.json`, as the sections below describe. The full logs of the close-out are under the batch's scratch folder, not committed.

What the close-out learned about the oracle itself:

- The released `Sniff.exe` corrupts its heap under threads (`0xc0000374`) and hangs on two Fallout 4 Creation Club texture archives (`cctosfo4001-neosky`, `cctosfo4002_neonflats`, `find-textures-copy`) even on one thread. The harness retries a crash twice and a hang once on one thread, then records the archive as `oracle-failed` and carries on; it never ends the run.
- The `Find textures` report prints `XBOX` from an uninitialised local record on every line of a BA2 DDS archive; the port prints nothing. The saved bytes are equal, so `sniff` counts these as log differences, not failures.
- `clean` of Fallout 76's `NW.esm` loses the oracle log's counts on every run (`oracle-log-lost`); the saved bytes are compared and equal.
- `check-dump` of `Starfield.esm` needs more than the 23.7 GiB per-process cap in a batch run (a rerun alone with `--max-memory 28` also met the machine's limit), and `tool-modes`'s `setesm` on `BlueprintShips-SFBGS050.esm` exceeds the 20-minute oracle limit and still exceeds 60 minutes alone (the other four modes on that file and the other eight `setesm` files are equal).
- The script run phase has four cases whose oracle timed out at 60 or 120 minutes under load; they are rerun owed with `--jobs 1 --oracle-timeout 180`.
- BSArchPro's autopacking mode starts no work when its window is on a desktop nothing displays: `bsarchpro` runs it on the user's desktop, shown. The case list named asset sources as `<work>\<name>` where the fixtures are under `<work>\src`, and the XBox refusal is shown on the `TFormTaskProgress` form (`Packing...`), not a `#32770`: all six cases are equal now (five archives byte for byte, one refusal).
30 changes: 30 additions & 0 deletions .claude/skills/checking-parity/references/inputs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
## Inputs

- **Corpus:** vanilla master files from the local game installs, found through `XEDIT_FO4_DATA` and `XEDIT_SSE_DATA` (each is the game's `Data` directory). Never commit game files. Small synthetic plugins live in `tests/fixtures` and are committed.
- **Oracle output:** produced by the release binaries (xDump, xEdit tool modes, BSArch, Sniff) and cached outside the repository, keyed by release tag, game and input file hash.

### Archives

`cargo xtask parity bsarch [--game <game>]... [--archive <name>]... [--max-size <MB>] [--jobs <n>] [--cross] [--synthetic] [--only <text>] [--textures] [--keep]` runs `BSArch.exe` (in `XEDIT_ORACLE_DIR`) and the port's `bsarch` (built in release) on the archives of the game folders and compares: `list` (the `-dump` text, byte for byte), `unpack` (the files written; an archive the oracle cannot unpack must stop the port with the same message), and `pack` (the folder the oracle unpacked, packed by both into the format of the source with each compression it has, or into every format with `--cross`; every archive written is compared byte for byte, with the oracle at `-mt:no`, the port at all threads and at `-mt:no`, and the text of the run too).

Archives up to `--max-size` MB (40) are taken unless some are named with `--archive`. `--synthetic` adds generated folders for what the game archives lack (two sources merged, filters, flags, `-split`, empty files, names that sort several ways, shared data), and the `edge` and `edge-wide` folders of issue #32 (TES4 hash collisions in one folder, BA2 names with one CRC-32, names that sort differently by ordinal and by the locale, names outside ASCII in and out of the ANSI code page), each packed in the six formats and the oracle's archive then listed and unpacked by both (with `-mt:no`, so an unpack that fails stops on the same file).

The texture archives (`DX10`) go through the same checks (`unpack` writes the DDS files; `pack` packs them with `-fo4dds` or `-sf1dds`); `--textures` takes only those, `--synthetic` adds generated DDS folders (formats, mipmap counts, a cube map, a 24 bit file, identical textures, data that ends early, and files both sides must refuse), and `--only <text>` keeps the synthetic packs whose label contains the text.

The work goes to `XEDIT_PARITY_SCRATCH/<tag>/bsarch` and is removed for an equal case (`--keep` keeps it); a different case keeps its folders and prints the first difference.

The report is `target/parity/bsarch.json`. `BSArch.exe` was seen to hang at its start (no CPU time for a quarter of an hour); a run over 30 minutes is stopped and run once more.

The oracle packs on one thread and compresses at level 12, so a few hundred MB take minutes: the default selection of the 11 games takes about half an hour with `--jobs 4`.

### BSArchPro packing (XBox target)

`cargo xtask parity bsarchpro [--case <name or name part>]... [--keep] [--refresh-oracle] [--oracle-only] [--list]` runs the packing workflow of the GUI `BSArchPro.exe` (parity #38), the 4.1.5q route to the XBox 360 archive target and to the `xtexconv.exe` bitmap conversion: a `.bsarch` list (plain JSON; `LoadList` reads it as well as the zip `SaveList` writes) with the archive type, the target, the archive name and the assets, the fixtures below `src` (generated DDS files, a PC and an XBox one, with the port's own `dds::set_up_header`), the GUI started in its autopacking mode (`BSArchPro.exe <list>`, hidden, its settings folder kept in the work folder through the child's `HOME`/`USERPROFILE`/`HOMEDRIVE`/`HOMEPATH`) and the port's `xedit --edit archive pack --target` over the same sources.

A case with `xtexconv` also puts the `xtexconv-stub` binary (this package's second binary: it rewrites the DDS header as an XBox one, the form the real tool produces) next to the program as `xtexconv.exe`, and the port gets `--program-path` pointed at that folder, so the spawn, the arguments, the temporary file and the read-back are exercised on both sides.

Compared: the archive bytes, or for a case that refuses (the XBox target over a PC texture with no tool) the dialog with no archive written and the port's message against the oracle executable's own UTF-16 strings (`oracle_utf16_strings`; a task dialog's body is drawn by DirectUI and holds no readable text).

Finding (parity #38): the 4.1.5q `BSArchPro.exe` autopacking mode shows its progress form and starts no work when it is driven without a user (no CPU time, empty memos, no file written; with and without `SW_HIDE`, with the threading on and off), so every case is reported `oracle-failed` after the 120 s watch and the run does not fail on it; the mode is kept for a run at a screen.

The port's route is checked meanwhile by the tests of `crates/xedit-session/tests/archive.rs` (`the_xbox_target_refuses_the_other_platforms_textures`, `dds2xbox_gives_upstream_messages`).
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
## Investigating a difference

1. Reduce to the smallest input that differs: one file, then one record, then one element.
2. Read the Pascal code that produces the oracle value. The oracle is right by definition, including its quirks. When the Pascal code does not explain the output, disassemble the oracle binary (`capstone` and `pefile` are installed for Python): the compiled overload resolution and RTL helpers are what actually ran.
3. Fix the port. Add the reduced case as a fixture test.
4. Never change expected output to match the port.
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
## Quirks found and reproduced

- **JSON reads.** `FromJSON` reads values with the `JsonDataObjects` properties: `S[Name]` of an array or an object fails with `Cannot cast Array into String` (or `Object`), `O[Name]` and `A[Name]` add an empty object or array when the key is missing, `null` is a nil object (no struct, and `Cannot cast Object into Array` for an array), a repeated key keeps its first value, and a number reads as `Integer`, `Long`, `ULong` or `Float` as the library parses it.

A Fallout 4 `BSConnectPoint::Children` dump has an array `Name` where `NiExtraData` has its value `Name` (the array replaced the value when it was written), so xEdit cannot build that NIF from its own dump, and neither does the port.
- **Sine and cosine.** Delphi's `Sin` and `Cos` on Win64 are the RTL's own (an argument reduction with a three-part π/2 and two polynomial kernels), not the C runtime's and not correctly rounded: an ulp apart from either at times, which flips the sign or the last bit of a matrix element near zero after `AxisAngleToM33` (90 degree rotations, `Sin(Pi / 4)`). `nif_math::sin` and `nif_math::cos` are ported from the machine code of `Sniff.exe` (`System.Sin` at `0x40BDC0`, `System.Cos` at `0x40BCD0`, the reduction at `0x40B750`, the kernels at `0x40BB40` and `0x40BC10`) and give the bits that the GUI's script adapter returns for 20097 arguments. `ArcTan`, `ArcTan2`, `ArcCos` and `ArcSin` are the RTL's own too (`System.Math.ArcTan2` at `0x4717F0`, the polynomial at `0x40C390`); the Sniff tangent processor found it (the `Bitangent` bytes of Oblivion meshes), and they are ported in `nif_math` since phase 5 step 4. `StrToFloat` is the RTL's own as well: the digits accumulate into a `Double` (`value := value * 10 + digit`), the fraction digits count against the exponent, and the scaling is the same paired-table `Power10` that `FloatToDecimal` uses — `xedit_core::delphi::str_to_float` is that algorithm, fitted byte for byte against the oracle (`probes/numerics-parse.pas`, issue #39): the text of the largest double gives the double one ulp below `MaxDouble`, the 21 digits of Pi give `0x1.921fb54442d17p+1` where Rust's correctly rounded parse gives `...d18`, and the `Inf`/`Infinity`/`NaN` words parse (what `TextToFloat` reads before `Val` runs).

A malformed text and a value the scale sends to zero or infinity raise `EConvertError '<text>' is not a valid floating point value`.
- **Delphi number semantics in Sniff.** `SameValue` of two doubles uses a resolution of `1E-12` (the RTL's `DoubleResolution`, `Max(Min(|A|, |B|) * 1E-12, 1E-12)`); `xedit_core::delphi::same_value` is that function since parity #39 (the record code's `1E-15` was wrong for every site) and `nif_math` re-exports it.

Single expressions on Win64 are worked out in double precision and rounded when stored. `Round` of a NaN gives `-9223372036854775808` (the integer indefinite, as the floating point exceptions are masked), which the byte float setter then refuses with `'-9223372036854775808' is not a valid integer value`; `xedit_core::delphi::round` and `delphi::trunc` give the indefinite for a NaN or an out-of-range value since parity #39, where the script `Round`/`Trunc` keep the low 32 bits of the `Int64` (`Round(NaN)` is 0, `Round(1e19)` is 0) and `Ceil`/`Floor` are `Trunc` plus `Inc`/`Dec` on `Frac` (`Ceil(1e10+0.5)` is 1410065409): a Fallout 4 mesh with a NaN bitangent fails that way in `Apply transform`. `FloatToStrF` scales the value by the RTL's `Power10`, which multiplies by powers of ten held as pairs of doubles (high 26 bits and the rest) and by reciprocal tables for negative powers: `xedit_core::delphi::power10` is ported from `System.Power10` of `Sniff.exe` (0x40CDF0) since phase 5 step 4, which turned `0.000014 * 0.75` into `0.000011` as the oracle writes it (single double multiplications gave `0.000010`).

A probe through the script adapter is the way to read such a function: print each double as its exponent and two 26-bit halves of its mantissa, which the script computes exactly.
- **Name hashes.** The data format units find an element by comparing `dfCalcHash` of the names only (`HashName` of the ANSI string): each character in lower case is xored into the sum, which is then rotated left by five bits. `parity nif --text` compares it on the 1596 element names of the definitions.

The ASCII fold matches the oracle byte for byte (parity #39 probed every byte and name); the non-ASCII part is owed: the oracle's `dfCalcHash(Chr(160))`..`dfCalcHash(Chr(255))` is 65533 for every one of them and `Chr(128)`..`Chr(159)` is 2016 (the `?` of a character the code page cannot map) except at the five undefined CP1252 bytes (129, 141, 143, 144, 157), which are 65533 too, while the port's ANSI conversion plus ASCII fold gives the `rotl5` of the byte; the mechanism behind that constant was not solved (probes/numerics.pas records the table and docs/PLAN.md carries it as owed).

- **Float rounding.** `xDump.exe` 4.1.5q binds `IntPower(10, ADigit)` in `RoundToEx` to the `Single` overload, so a value is divided by and multiplied with the single-precision `10^-digits` in double precision, and 64-bit `FloatToDecimal` scales the result by a power of ten in double precision into an 18 digit mantissa before rounding to the decimals. `delphi.rs` reproduces both (`int_power_single`, `round_to_ex`, `float_to_decimal`). `make_float_probe.py <out.esp>` writes a plugin of 1800 GMST floats to probe the oracle (`xDump.exe -SSE -q -D:<dir> <out.esp>`), and `angle_probe.py <plugin> <oracle dump>` pairs the oracle's REFR rotation lines with the raw radians; both agree with the port.

Upstream's `TwoPi` is `2 * Single(3.1415927)` (`TWO_PI` in `constructors.rs`).

The rounding helpers were found by disassembling `xDump.exe` with `capstone`: `RoundToEx` loads `10.0` as a single before calling `IntPower`, and `Round` is a plain `cvtsd2si`.

The printing surface at full precision is probed by `probes/numerics.pas` (issue #39): `FloatToStrF(value, ffExponent, precision, digits)` has exactly `precision` significant digits from `FloatToDecimal` and pads the exponent to `digits` places only when `digits` is at most 4, so `FloatToStrF(1e22, ffExponent, 17, 0)` is `1.0000000000000000E+22` and `0.1` prints `1.0000000000000000E-1` where Rust's correctly rounded `{:e}` printed `1.0000000000000001E-001`.

Every operation of `FloatToDecimal` is an SSE operation in the `MXCSR` mode a script's `SetRoundMode` set (`xTESEdit.exe` 4.1.5q: `Power10` at `0xE4C0` changes only the precision mask, `Round` is `cvtsd2si` at `0xCB10`), so the digits depend on the mode (`0.3` is `2.9999999999999994E-1` under `rmDown`); the port replays each addition, product, quotient and rounding in the script's mode (`delphi::Arith`, the nearest result corrected by the sign of its exact error) and `sys-round.pas` compares the digits in all four modes. `TextToFloat` is the other way round: it sets the default control word (`SetMXCSR($1F80)`) around the parse, so a literal or a `StrToFloat` never follows the mode.
- **Length-prefixed strings read past their data.** `TwbLenStringDef.ToStringInternal` limits the text to the length of the whole data, prefix included, so a stored length larger than the data reads up to the prefix's bytes past the end of the element, out of the bytes that follow it in the file. A Fallout New Vegas save's dialogue `Response Text` showed it. `len_string.rs` reads those bytes from the block of the data container (`bytes_after` in `element.rs`).
- **Save tables keep elements, not values.** `InitializeSaveWorldspaceArray` stores the elements of the `Visited Worldspace` chapter and reads their values when a worldspace index is shown. Reading the values while the save loads resolved the names of the worldspace records before their strings were loaded, and the empty names stayed cached: a Fallout 4 save showed `<SanctuaryHillsWorld>` without its name.
Loading
Loading