A quality gate for prose. It fails 46.9% of machine-written documents and 6.3% of human-written ones.
pip install bluepencil
bluepencil check draft.mdLive demo, nothing to install: huggingface.co/spaces/wolfvswhale/bluepencil
Exit code 1 when something fires at error severity, so it drops into CI, a pre-commit hook, or an agent loop without wrapping.
Most style linters ship a list of banned phrases and a threshold someone picked because it felt about right.
You install it, it fires on your good writing, you uninstall it.
Every threshold here was set by measuring the distribution of that statistic across 2,602 human-written documents spanning three registers: informal forum answers, formal scientific abstracts, and multi-paragraph encyclopedia articles. Each cadence gate is placed at the percentile that makes it fire on 5% of human writing. That rate is a property of the tool, not a hope, and bluepencil gates will print it back to you.
The companion repo prose-eval is the harness that produced those distributions.
$ bluepencil check draft.md
draft.md
3:1 error throat-clearing
"In today's fast-paced world"
-> Delete it. Start with the first real sentence.
3:44 error negative-parallelism
"is not just a nice-to-have, it's"
-> State the correct thing. Leave the wrong thing out.
4:24 error editorializing
"It's important to note"
-> Just write the note.
- warn flat-cadence
Sentence lengths vary less than human writing typically does.
Measured 0.198, below the calibrated threshold of 0.237.
-> Rewrite one sentence in every three-sentence run to a
different length.
1 file checked, 0 clean, 3 errors, 1 warning
Two kinds of finding, and they need different responses.
Pattern findings point at a span with a line and column. The fix is unambiguous and takes seconds.
Cadence findings describe the document. There is no line to fix; the draft needs a pass. These are the ones that catch prose which has had every flagged word swapped out and still reads as machine-written, which is the failure mode a phrase blocklist cannot see.
Fire rates across 1,500 human and 1,500 machine documents, held out from calibration. "Lift" is how much more often a gate fires on machine text than human text.
| gate | human | machine | lift |
|---|---|---|---|
| editorializing | 0.1% | 18.5% | 139x |
| assistant-residue | 0.1% | 7.9% | 119x |
| hollow-conclusion | 1.1% | 21.8% | 20x |
| flat-cadence | 3.4% | 23.1% | 6.8x |
| triplet-habit | 4.7% | 17.4% | 3.7x |
| transition-crutch | 5.0% | 6.9% | 1.4x |
| repeated-openers | 5.1% | 5.9% | 1.2x |
| any finding | 31.3% | 70.1% | 2.2x |
| fails at error severity | 6.3% | 46.9% | 7.4x |
Nine of the eighteen gates fire more often on human writing than on machine writing here. All of them:
| gate | human | machine | lift |
|---|---|---|---|
| copula-avoidance | 5.8% | 0.8% | 0.1x |
| em-dash-habit | 5.5% | 0.0% | 0.0x |
| negative-parallelism | 2.4% | 0.7% | 0.3x |
| participle-tail | 1.5% | 0.1% | 0.0x |
| uniform-paragraphs | 1.5% | 0.0% | 0.0x |
| vague-attribution | 0.8% | 0.1% | 0.1x |
| inflated-significance | 0.7% | 0.2% | 0.3x |
| promotional | 0.3% | 0.2% | 0.8x |
| engagement-bait | 0.1% | 0.0% | 0.0x |
The obvious reading is that those rules are wrong.
The likelier reading is that the corpus is old.
The corpus used here, HC3, was collected around December 2022. Em dashes are the signature of current models: a pre-registered study of 69,632 medRxiv preprints (arXiv:2606.29540) puts em-dash prevalence in Discussion sections at 4.23% before ChatGPT, 8.0% in 2024, and 20.3% in 2025. HC3 predates almost all of that rise, so it cannot contain the behaviour the gate looks for. Newer paired corpora built on current models do exist, including MIRAGE, MAGA-Bench, and AITDNA, and recalibrating against one of them is the obvious next step. Until then these gates stay on and stay labelled unvalidated instead of disproven.
uniform-paragraphs is a different problem: the machine half of the corpus is single-paragraph, so the statistic is unmeasurable there and its 0.0x is an artifact of the data, not a result.
One gate shipped disabled. no-short-sentences could not be calibrated: more than 5% of human documents contain no short sentence at all, so no threshold below the target false-positive rate can ever fire. Calibration detects that and disables it instead of shipping a gate that silently never triggers.
Pre-commit. Add to .pre-commit-config.yaml:
repos:
- repo: https://github.com/wolfvswhale/bluepencil
rev: v0.1.0
hooks:
- id: bluepencilGitHub Actions. --format github emits annotations that render inline on the pull request diff:
- run: pip install bluepencil
- run: bluepencil check . --format github --fail-on neverDrop --fail-on never when you want it to block merges.
Claude Code and other agent tools. Copy skill/SKILL.md into your skills directory. It tells the agent when to run bluepencil, how to read the two kinds of finding, and, importantly, when to overrule one.
Anything else. --format json gives you the findings with line numbers, measurements, thresholds, and suggested fixes.
bluepencil.toml, discovered from the file upward:
[bluepencil]
fail_on = "error" # error | warn | never
min_words = 120 # documents shorter than this skip cadence gates
[bluepencil.gates]
em-dash-habit = "off" # off | warn | error
promotional = "error"
[bluepencil.budgets]
transition-crutch = 6.0 # occurrences per 1000 words tolerated
[bluepencil.thresholds]
flat-cadence = 0.30 # override a calibrated valueRoughly one human document in twenty gets a cadence finding a good editor would overrule. That is the 5% target working as designed. Turn the gate off instead of mangling the sentence.
The shipped thresholds describe general human prose. If your house style is genuinely different, point the calibration at your own corpus:
pip install "bluepencil[calibrate]"
python scripts/calibrate.py --target-fpr 0.05
python scripts/measure.py --limit 1500The first writes bluepencil/thresholds.json. The second regenerates the tables above.
Whether the text is true, whether the argument holds, or who wrote it. It measures the surface. A document can pass every gate and still be empty, and a good writer will sometimes trip three gates on purpose.
It is a gate, not a classifier. A document failing means it contains constructions the standard rejects, which is true of plenty of human writing, as the 6.3% says.
Rules derived from Wikipedia's Signs of AI writing, maintained by WikiProject AI Cleanup, plus a house editing standard. Calibration corpora: HC3, AI-and-Human-Generated-Text, and wikitext-103.
MIT licensed. Built by J. Alderman Lyell (@wolfvswhale).