fix(explainer): harden parse_explanation against CoT echo and marker injection

[AUTHENSOR]

Summary

Explainer.parse_explanation extracted the feature label with re.search(r"\[EXPLANATION\]:\s*(.*)", text, re.DOTALL) — first-match with greedy-to-EOT capture. This has two distinct failure modes, one a parsing defect and one a security issue.

Failure mode 1 — CoT echo (parsing defect)

When the explainer reasons in chain-of-thought and writes [EXPLANATION]: while thinking — e.g. considering then rejecting a hypothesis — the greedy .* with re.DOTALL swallows the entire response from the first marker to end-of-text:

Step 1. I considered [EXPLANATION]: fragments of words. but that does not fit.
Step 2. Actually comparative adjectives.
[EXPLANATION]: The token "er" at the end of a comparative adjective describing size.

The extracted label becomes the whole reasoning chain, not the final verdict. This degrades label quality even with no adversary present — any CoT-flavored explainer run is affected.

Before: 'fragments of words. but that does not fit. Step 2. Actually comparative adjectives. [EXPLANATION]: The token "er"...'
After: 'The token "er" at the end of a comparative adjective describing size.'

Failure mode 2 — marker injection (security)

First-match binding lets an early [EXPLANATION]: token win over the explainer's actual final verdict. The highlighted examples in the explainer's prompt (top-activating text from the subject model, see _highlight / _join_activations) are injected verbatim. A subject model whose top-activating text contains [EXPLANATION]: <benign concept> can cause the explainer to echo it, and the echoed marker — appearing earlier in the response — wins first-match.

The resulting label flows verbatim through ExplainerResult.explanation into DetectionScorer / IntruderScorer / OpenAISimulator and into operator-facing auto-interp tooling (Neuronpedia ingests delphi explanations). There is no human gate on the label anywhere in the pipeline.

Before (injected-early-marker wins): 'benign educational content>> in its output ... [EXPLANATION]: Chemistry educational content.'
After: 'Chemistry educational content.'

The fix

Last-match binding with non-greedy capture (no re.DOTALL):

matches = list(re.finditer(r"\[EXPLANATION\]:\s*(.+)", text))
if matches:
    return matches[-1].group(1).strip()
return "Explanation could not be parsed."

Without re.DOTALL, . does not match newlines, so each [EXPLANATION]: marker binds to only its own line. Taking the last match selects the explainer's final verdict. This matches the system prompt's instruction that "the last line of your response must be the formatted explanation."

Defeats both failure modes; preserves clean single-marker parsing.

Honest residual

If an injected marker is the literal last line of the response, last-match still binds to it. The complete defense — a response-aware scrub that strips any [EXPLANATION]: token appearing in the highlighted-examples input before parsing (analogous to the pattern used in eval-judge hardening) — is a follow-up. This PR closes the common cases (CoT echo + early-line injection) with a minimal, low-risk change. I'm happy to add the scrub in this PR if reviewers prefer the complete defense.

Verification

• 7 new regression tests in tests/test_explainers/test_parse_explanation.py:
  • Clean single-line + inline parsing preserved (no regression)
  • Missing-marker fallback preserved
  • CoT echo no longer swallows the chain
  • Last-match wins when multiple markers present
  • Injected early marker loses to the real verdict
  • Whitespace stripping preserved
• All 7 tests pass.
• ruff check passes (line-length 88, E/F/I rules — repo default).
• Tests load delphi/explainers/explainer.py in isolation (stubbing the heavy optional deps in delphi.clients) so they run without a full install — useful since the full suite needs vllm/torch. The parse_explanation method itself only depends on re and logger.

$ pytest tests/test_explainers/test_parse_explanation.py -v
7 passed in 0.01s
$ ruff check delphi/explainers/explainer.py tests/test_explainers/
All checks passed!

Context

First-match / greedy extraction of an LLM-produced label is a known pitfall in LLM-judge pipelines. The defensive pattern (last-match binding, or anchored extraction) is used by inspect_ai, openai/evals, and METR/CoT-faithfulness-and-monitorability for their judge scorers. The contrast is notable: openai/automated-interpretability (Bills et al. 2023) avoids trusting the label string at all by running a second simulator model that scores how well the explanation reproduces real activations (ev_correlation_score) — a stronger defense, but a larger architectural change than this PR attempts.

I'm doing a wider audit of LLM-judge extraction under an authorized OSS red-teaming scope; happy to keep this PR tightly scoped or adjust the approach. The fix is intentionally minimal and behavior-preserving for clean inputs.

Checklist

• CoT-echo parsing defect fixed (final verdict extracted, not whole chain)
• Early marker injection defeated (real verdict wins)
• Clean single-marker parsing unchanged (no regression)
• Missing-marker fallback unchanged
• 7 regression tests added, all pass
• ruff check passes
• No new dependencies
• Residual (last-line injection) documented honestly

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

John Kearney seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}