feat(eml): read RFC 5322 email messages

[itsaaronngan]

Closes half of #128: .eml only. .msg is #160, from @vaibhavdabas16 — independent frontends, either order.

Shape

Hand-rolled MIME on std + encoding_rs, no new dependencies. Follows csv.rs: parse(bytes) -> Result<Document, ConvertError>, produces the document model, does not touch the Markdown writer.

• RFC 5322 headers, continuation lines unfolded
• multipart traversal, depth-capped, prefers text/plain
• quoted-printable and base64 transfer-decoding
• RFC 2047 encoded-words, both B and Q
• charset via encoding_rs, mirroring csv.rs::decode
• Subject as H1, From/To/Cc/Date as a metadata paragraph
• extension-only detection, no detect.rs entry, per the CSV precedent for signature-less text formats

Registered in the node, wasm and python enums and in node/cli.js FORMATS.

A body that is text/html with no text/plain alternative returns Unsupported rather than emitting markup as prose. That case wants #52 or #53 landing first; I did not want to guess at HTML here.

Validation

142 real .eml files, Outlook/Exchange via Mimecast: 141 convert, median 4.0 ms.

The one failure is a calendar invite whose text/plain part is 2 bytes and whose content is text/calendar. It returns Unsupported rather than emitting the 2 bytes. Correct as far as I can tell, but it is a judgement call — happy to extract it instead if you would rather.

Running the whole corpus, rather than spot-checking, is what made this worth trusting. It surfaced an RFC 2047 bug I would not have invented a test for: decode_one_word looked for the terminator ?= from the start of the word, but a payload's own =XX escapes can put a literal ?= inside it, so the search landed early and parsing bailed, leaving raw encoded words in the subject. Fixed by locating the terminator after both ? separators, with a regression test. Post-fix the corpus has zero undecoded escapes and zero U+FFFD.

That corpus is private, so the fixtures are synthetic: seven handmade files under tests/fixtures/eml/, plus the fixtures_detect_from_content entry. Writing them turned up four more bugs I had missed against real mail, the worst being that a whitespace-only line (one space, not empty) did not break a paragraph, so it rendered as a hard break with nothing after it. 3267 occurrences across the corpus, now zero.

Fuzzed as well, since the parser walks attacker-shaped bytes: 3,088,842 executions over the two seeds, no crashes, no hangs, no leaks.

Checks, on this branch against v0.2.4:

• cargo test --locked — 309 passed, 1 ignored (13 new unit tests)
• cargo clippy --workspace --all-targets --all-features -- -D warnings
• cargo clippy -p anydoc-wasm --target wasm32-unknown-unknown -- -D warnings
• cargo fmt --all --check
• cargo +nightly fuzz run eml — 3,088,842 executions, clean

Two notes for review

detect.rs. I left .eml on extension-only detection, reading the CSV precedent in the README ("CSV has no such marker, so the extension or an explicit format names it instead") as the rule for signature-less text formats. #160 chose a content-sniff entry for .msg, which is a different case — OLE has a real signature. If you would rather .eml sniffed too, say so and I will add it; I did not want to invent a heuristic that detect.rs warns against.

Shared helper. If both this and #160 land, each frontend carries its own envelope-and-body rendering. @vaibhavdabas16 offered to factor that out as a follow-up once the second one is in, and that is the right shape — he has the clearer view of both. I am happy to take it instead if he would rather not.

---

Summary by cubic

Adds RFC 5322/MIME .eml conversion, closing the .eml half of #128. Previously unsupported, messages now convert to Markdown with the Subject as an H1, sender metadata, and a text/plain body preferred over text/html.

Behavior

• Supports folded headers, RFC 2047 encoded words, quoted-printable and base64 bodies, charsets, and nested multipart messages without new dependencies.
• Excludes attachments, recovers missing boundaries as flat text, and returns Unsupported for HTML-only messages.
• Uses extension or explicit-format selection because EML has no content signature, and exposes the format in the Rust, Node, Python, WASM, and CLI APIs.
• Handles padded closing boundaries without consuming the epilogue and preserves whitespace before malformed encoded-word candidates.

Validation

• Adds fixtures, snapshots, unit tests, and fuzz coverage; 141 of 142 real Outlook/Exchange messages convert with a 4 ms median, and 3M+ fuzz executions completed cleanly.

^{Written for commit af00979. Summary will update on new commits.}

[cubic-dev-ai]

1 issue found and verified against the latest diff

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/formats/eml.rs">

<violation number="1" location="src/formats/eml.rs:344">
P2: Malformed base64 with misplaced or incomplete padding is accepted as valid and can produce corrupted body text. Validate padding and the final quartet before returning `Some`; return `None` for malformed payloads so the existing recovery path runs.</violation>
</file>

<sub>Tip: instead of fixing issues one by one fix them all with cubic

Re-trigger cubic</sub>

[cubic-dev-ai]

P2: Malformed base64 with misplaced or incomplete padding is accepted as valid and can produce corrupted body text. Validate padding and the final quartet before returning Some; return None for malformed payloads so the existing recovery path runs.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/formats/eml.rs, line 344:

<comment>Malformed base64 with misplaced or incomplete padding is accepted as valid and can produce corrupted body text. Validate padding and the final quartet before returning `Some`; return `None` for malformed payloads so the existing recovery path runs.</comment>

<file context>
@@ -0,0 +1,541 @@
+    let mut acc: u32 = 0;
+    let mut bits = 0u32;
+    for &b in input {
+        if b.is_ascii_whitespace() || b == b'=' {
+            continue;
+        }
</file context>

[itsaaronngan]

Thanks — six of the seven are real and are fixed in c47d71d, each with a regression test that I checked fails against the previous commit rather than passing vacuously.

The two that mattered were genuine correctness bugs, and both are cases my 142-message corpus simply never contained:

• Attachment displacing the body (P1). Right, and worse than described: because the text/plain pass runs before the nested-multipart pass, a multipart/mixed of [multipart/alternative, text/plain attachment] returned the attachment and never reached the real message. Now excluded via RFC 2183 disposition.
• Boundary prefix (P2). Also right, and this one lost content silently — a body line beginning --abcExtra was read as a delimiter and everything after it was dropped with no error. Now matched per RFC 2046 5.1.1: the boundary exactly, optionally closed with --, then only whitespace.
• Adjacent encoded-words (P2). Correct per RFC 2047 6.2. Whitespace between a word and ordinary text is still preserved.

The three P3 doc points are fixed too — EML is now named alongside CSV in the Format/from_bytes/to_markdown_bytes doc comments, the CLI's stdin help, and the README's detection paragraph and parser diagram.

One I do not think is valid: the base64 padding finding (P2, eml.rs:344). The claim is that misplaced or incomplete padding is accepted and "can produce corrupted body text". Decoding aGVs=bG8gd29ybGQ= as a quoted body gives hello world, not corruption — skipping = wherever it appears is lenient, but it is lenient in the direction Postel's law wants for mail, and I could not construct an input where it produces wrong text rather than simply tolerating a malformed one. Returning None there would send valid-enough real messages down the recovery path instead. Happy to be shown a counter-example if one exists.

Re-fuzzed after the changes: 1,898,113 executions, no crashes, no hangs, no leaks. Full suite is 312 passing, clippy clean on the workspace and the wasm target, fmt clean.

[cubic-dev-ai]

All reported issues were addressed across 4 files (changes from recent commits).

<sub>Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic</sub>

[itsaaronngan]

Both valid, both mine, both fixed in af00979.

• Padded closing delimiter. Right — I widened is_delimiter to accept the trailing whitespace RFC 2046 5.1.1 permits but left the close check testing the raw line, so --B-- opened a part instead of closing the multipart and the epilogue was parsed. is_delimiter now reports which form it matched, so there is one decision rather than two that can disagree.
• Whitespace before a malformed token. Also right. The gap is now resolved after the candidate decodes, so it is only suppressed between two genuinely decoded words.

Worth noting for anyone reading the history: my first regression test for the padded-close case passed against the unfixed code. The fixture put the text/plain part before the close, so the correct body was found regardless and the test never reached the bug. I only caught that by reverting the fix and checking the test actually went red. Both new tests are verified that way now, as are the three from the previous round.

Re-fuzzed after these changes: 1,652,818 executions, no crashes, no hangs, no leaks. Suite is 314 passing, clippy clean on the workspace and the wasm target, fmt clean.

The base64 padding item is still open from my side — I could not construct an input where lenient = handling yields wrong text rather than merely tolerating a malformed payload, and returning None there would push valid-enough real messages down the recovery path. Genuinely happy to be shown a counter-example.