Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ newlines parses as readily as `*` and `~`.
| Path | What it is |
| --- | --- |
| `web/EDIWorkbench.html` | **The app.** Self-contained: markup, styles and engine in one file. |
| `web/tests/` | Eight correctness suites, plus generators for the committed sample files. |
| `web/tests/` | Nine correctness suites, plus generators for the committed sample files. |
| `web/bench/` | Benchmark over synthetic interchanges up to 150 MB. |
| `web/PERFORMANCE.md` | Measured limits and the ordered backlog for improving them. |
| `docs/pacdr-validation.md` | What the post-adjudicated checks do, what they deliberately don't, and why. |
Expand All @@ -146,7 +146,7 @@ it byte for byte, which is what stops the port drifting from the original.
## Running the tests

```sh
node web/tests/all.mjs # browser app: 8 suites
node web/tests/all.mjs # browser app: 9 suites
python -m unittest # Python prototype
```

Expand Down
14 changes: 11 additions & 3 deletions docs/835-validation.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,9 +153,17 @@ code change, audited the same way: the rule's identifier-bearing loops are
opens no loop at all under this feature — see "the governing rule" above —
so `scrubLoop` never reaches one of these NM1s, and `PERSON_LOOPS` itself
never names an 835 loop id to begin with. An 835 handed through Limited Data
Set comes back byte-identical to what went in, which is worth stating loudly
in the app's own UI copy: silence here would hand a user a file they believe
is scrubbed.
Set comes back byte-identical to what went in, and silence about that would
hand a user a file they believe is scrubbed.

So the app says it twice, at the point of use rather than only in the guide:
a caution inside the Limited Data Set card (`#deidRemit`, shown by
`syncRunButton` whenever `has835` is true of the loaded file), and a message
naming the reason after a run that changes nothing. The button is deliberately
left **enabled** for an 835 — disabling it would read as "not yet", where the
truth is "never, by design". `render.mjs` section [5] pins all of it, including
that the caution clears again when an 837 is loaded over an 835, and that the
output really is byte-identical rather than merely reported as such.

## Sources, and what could not be used

Expand Down
14 changes: 13 additions & 1 deletion web/EDIWorkbench.html
Original file line number Diff line number Diff line change
Expand Up @@ -1831,6 +1831,9 @@ <h1 class="wordmark">EDI<span class="sep">*</span>WORKBENCH</h1>
<p class="hint">Dates are shifted per patient rather than removed, and city, state and ZIP are kept &mdash; that is what a <b>HIPAA Limited Data Set</b> permits, and it is what keeps the file usable as test data. Providers are left real: an NPI is not patient information.</p>
<p class="hint caution"><b>The result is still PHI.</b> A Limited Data Set is not de-identified data under HIPAA &mdash; it stays protected, and sharing it still needs a data use agreement. Only Safe Harbor or an expert determination produces data that leaves HIPAA behind, and neither is what this does.</p>
<p class="hint">The mapping is random for every run and is never saved, so this is one-way &mdash; the output cannot be turned back into the original.</p>
<!-- Shown only for an 835. The rule is a no-op on one, and a silent
no-op hands back a file the user believes is scrubbed. -->
<p class="hint caution" id="deidRemit" hidden><b>This file is a remittance advice, and nothing here will be stripped from it.</b> An 835's patient identifiers sit directly inside loop <code>2100</code> (<code>NM1*QC</code>, <code>NM1*IL</code>), outside the loops this rule targets, so the file comes back byte-identical &mdash; unchanged, not de-identified. Run this on the 837 the remittance answers instead.</p>
<button class="btn deid" id="deidBtn" type="button" disabled>
<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
<path d="M12 3 4 6v6c0 4.5 3.2 8.4 8 9 4.8-.6 8-4.5 8-9V6l-8-3Z" />
Expand Down Expand Up @@ -5938,6 +5941,10 @@ <h3>Using it with real data</h3>
// De-identification takes no configuration, so a loaded file is the only
// thing it needs. The rule switches above deliberately don't gate it.
$("deidBtn").disabled = !state.doc;
// The button stays enabled for an 835 -- running it is harmless, and
// disabling it would say "not yet" where the truth is "never, by design".
// Say so instead, before the click rather than after it.
$("deidRemit").hidden = !state.doc || !has835(state.doc.segments);
}

["shiftOn", "swapOn"].forEach((id) => $(id).addEventListener("change", syncCards));
Expand Down Expand Up @@ -6036,7 +6043,12 @@ <h3>Using it with real data</h3>
// way back from here, and the file on screen is now the only copy of this
// particular mapping.
if (!result.changes.length) {
toast("No patient loops found — nothing to strip");
// An 835 is the one file that reaches zero changes by design rather than
// by accident, and "nothing to strip" reads as "already clean" on it. Name
// the reason, so the identical download isn't mistaken for a scrubbed one.
toast(has835(state.doc.segments)
? "Remittance advice — nothing stripped, the file is unchanged"
: "No patient loops found — nothing to strip");
}
}

Expand Down
10 changes: 6 additions & 4 deletions web/PERFORMANCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,9 @@ move the ceiling far enough that this stays unnecessary.

## Test harnesses

All committed. `node web/tests/all.mjs` runs the four correctness suites
(engine/Python parity, outline and qualifier decoding, virtual-list windowing math,
static self-containment lint); `node web/bench/bench.mjs` runs the benchmark that
produced the numbers above.
All committed. `node web/tests/all.mjs` runs the nine correctness suites
(engine/Python parity, outline and qualifier decoding, structure-tree building, the
claim index and CSV export, control counts and claim balance, de-identification,
virtual-list windowing math, the load path against a throwaway DOM, and the static
self-containment lint); `node web/bench/bench.mjs` runs the benchmark that produced
the numbers above.
24 changes: 23 additions & 1 deletion web/tests/render.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ globalThis.getComputedStyle = globalThis.window.getComputedStyle;
globalThis.matchMedia = globalThis.window.matchMedia;

const app = new Function(
`${script}\nreturn { loadFile, SAMPLE_837P, SAMPLE_PACDR, SAMPLE_835, state };`
`${script}\nreturn { loadFile, deidentify, SAMPLE_837P, SAMPLE_PACDR, SAMPLE_835, state };`
)();

const paintedRows = () => (byId.get("docWindow").innerHTML.match(/class="row/g) || []).length;
Expand Down Expand Up @@ -151,6 +151,28 @@ check("the workspace is hidden", byId.get("workspace").hidden, true);
check("an error is shown", byId.get("parseError").innerHTML.length > 0, true);
check("stale readouts are cleared", byId.get("readouts").innerHTML, "");

console.log("\n[5] the Limited Data Set warns on an 835, and stops warning off one");
// The rule is a no-op on an 835 -- its identifiers sit in loop 2100, which
// scrubLoop never reaches. A silent no-op hands back a file the user believes
// is scrubbed, so the caution is the safeguard and this is what pins it.
// Loading an 837 over an 835 has to clear it again: a caution left standing on
// a file the rule *does* scrub is the same failure pointed the other way.
app.loadFile("sample_835.edi", app.SAMPLE_835);
check("the 835 caution is shown for a remittance", byId.get("deidRemit").hidden, false);
check("the button stays enabled anyway", byId.get("deidBtn").disabled, false);

app.deidentify();
check("running it on an 835 changes nothing", app.state.result.changes.length, 0);
check("the output is byte-identical to the input", app.state.result.output, app.SAMPLE_835);
check("the message names the reason", byId.get("toast").textContent,
"Remittance advice — nothing stripped, the file is unchanged");

app.loadFile("sample_837p.edi", app.SAMPLE_837P);
check("the caution clears for an 837", byId.get("deidRemit").hidden, true);

app.loadFile("broken.edi", "this is not an interchange");
check("the caution clears when nothing is loaded", byId.get("deidRemit").hidden, true);

console.log(`\n${pass} passed, ${fail} failed`);
if (fail) process.exit(1);
console.log("clean");