Skip to content

Fold recursive-writer's tested rules into editor and writing-planner - #825

Merged
eandreeva-twr merged 4 commits into
mainfrom
chore/fold-recursive-writer
Nov 11, 2026
Merged

eandreeva-twr merged 4 commits into
mainfrom
chore/fold-recursive-writer

Conversation

@eandreeva-twr

@eandreeva-twr eandreeva-twr commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

recursive-writer (the manager plus seven rw-* modules, about 24k words and 2k lines of Python) is a second writing pipeline next to doc-author → writing-planner → editor. Evaluations run on 2026-11-11 tested what it adds. This PR moves the parts that showed a gain into the existing skills, so editor can be tested with them. The rw-* skills stay untouched for now.

What the evaluations found

  1. Review, known problems. Four arms reviewed six articles: no skill, current editor, editor plus rw's evidence rule, and the rw check library. All four caught about the same known problems: 10–11 of 12 planted defects, and 3–6 of 19 problems from past review comments. The rw check library produced more findings without more hits.
  2. Review, old articles. Three arms reviewed three articles not edited since January 2026. Their 344 findings merged into 107 issues, labeled blind. Share of findings accepted: no skill 48%, rw checks 45%, editor 39%. rw found 31 of 32 accepted issues across its two runs, the others 28. The two issues only rw found were wrong product terms, from its term check.
  3. Writing from scratch. Three methods wrote flow-custom-elements.mdx from one task description and one verified fact pack, scored blind against the lead writer's review of the real rw draft. The current workflow, in both runs, explained SDK code detail that belongs to the iOS article and described what the UI already shows. Adding rw's reader-question tags and heading tree fixed the first in both runs and the second in three of four judge calls, and cut length by about 10%. rw's full planning scored about the same as the borrowed rules, at twice the time.

What this PR changes

  1. Three planning rules in writing-planner. Step 5 now tags reader questions (act, see, edge, owner), builds the heading tree with a path line per section, and limits limitations and troubleshooting to what a reader would hit. references/plan-templates.md carries the path lines and a Left out list. Source: rw-questions, rw-plan, and rw-plan's structure brief.
  2. The term check and the product glossary in editor. A platform-term rule in Area 11, and references/product-terms.md (Adapty terms and domain rules, loaded on every review). Tested: the new editor caught 8 of 8 planted wrong terms in each of 3 runs, the old one 4–6, with no correct term flagged by either. Source: rw's 07-terms, product-terms.md, domain-rules.md.
  3. Fact-research rules in writing-planner Step 2 (weak evidence). Sources per fact, tracing a control end to end, release state from code. Tested on custom elements: one of two runs got 6 of 12 key facts right against 3–4 for the old step; the other run matched the old one. Both methods missed every fact about what users on older app versions see after a change. Source: rw-research.
  4. A multi-role review in editor (references/multi-review.md). Two reviewers per role (mobile developer; Adapty developer who checks every behavior claim against the transformer, frontend, backend and SDK code; editor; product manager), then one merge into a final review.
    • After writing a new article, or a section or more, editor runs it by default, applies the fixes itself, and presents only the final text.
    • In review mode, editor offers it for a new article, a full-article review, or a diff that touches more than one section, and asks first.
    • Tested on the original custom-elements draft: the final review found 11 of 12 key facts with or without a hint about where the problem was. Without the hint, reviewers found the version facts more often, and the merge step recovered the one issue no single reviewer caught in full. Source: the lead writer's own review prompt.
  5. The rule-conflict decisions below, in editor, writing-planner and scripts/lint-prose.mjs.
  6. Rules for handling the lead writer's review comments (see "Handling review comments" below): editor/references/review-comments.md, a comments mode in editor, the lead writer's standing rules in editor's Area 12 and product-terms.md, and one line in review-assistant that loads the new reference.

Handling review comments

What went wrong before

An analysis of 280 review threads from the lead writer, on the 34 most recent PRs they reviewed for other writers:

  • Most comments were in Russian and short. 70% Russian, 19% mixed, and a quarter under 40 characters. Short forms carry broad meaning: "same here", "why?", "I think not".
  • The usual failure was a right edit in one place only. A rule fixed in the quoted line came back in later PRs. Negative framing came back in 10 PRs, and limitations placed first came back in 6. The same defect also survived in a shared snippet, a sibling platform page, or a heading.
  • Parts of long comments were skipped silently. Authors almost never replied to comments, so nothing recorded what was skipped.
  • "Move it" was handled as "delete it". A fact the reviewer asked to move to troubleshooting was deleted instead.
  • Factual objections were fixed without checking the code. Most such fixes took the reviewer's wording or dropped the claim.

What the new rules do

editor/references/review-comments.md covers:

  1. Reading a comment: the language, what the short forms mean, suggestion blocks, questions, outdated threads.
  2. Finding every instance: search by the comment's own words across the docs and the shared snippets, open every snippet the article renders, and include headings, link text and frontmatter.
  3. Fixing it the way the comment asks:
    • "move" and "only in X" keep the fact;
    • "shorten" isn't "remove";
    • a factual objection gets a source;
    • unrelated files are matched against the task.
  4. An outcome for every comment and every part of one, with a reply draft. Replies are never posted.
  5. Turning repeats into rules: a stated or repeated point is proposed as a rule change, so editor catches it next time.

The new reference also maps the lead writer's 13 standing rules to where editor now carries each one. Seven of them are new in editor's Area 12:

  • no risk framing;
  • limitations last, in one place;
  • warnings kept rare;
  • niche cases left out;
  • content order (important first, text before images, v4 before v3);
  • integration articles don't sell Adapty features;
  • no method names in conceptual text.

Area 8 also gains a line against new untested snippets, and product-terms.md now says Apple Ads and Adapty SDK.

Conflicting rules: decisions

Where editor and recursive-writer gave opposite rules

Word rules

Rules 1–3 come from recursive-writer's personal lint file (lib/tools/extra-rules.mjs), not from its shared check library.

1. "lets you"

  • Conflict: editor used "lets you" as the fix for sentences that give an abstract entity an action ("The page lets you manage your settings"). recursive-writer bans "lets you" on every line.
  • Decision: recursive-writer's rule. Don't use "lets you" or "allows you to".
  • Why: it keeps the abstract entity as the actor; making the reader the subject is direct.
  • Change: editor Area 5 now gives "On this page, you can manage your settings" as the fix and says to make the reader the subject. The sample rewrite in output-templates.md changed the same way. lint-prose.mjs has a new banned-lets-you rule that flags both phrases on changed lines only (59 existing articles use them).

2. Possessives

  • Conflict: editor allows a possessive when it's true of the representation ("the flow's row"). recursive-writer flags possessives and asks to restructure: name the UI place, make the owner the subject, or use "of".
  • Decision: editor's rule.
  • Why: "the flow's row" is clear; banning the form costs readability for no gain.
  • Change: none.

3. "provides" and "these X:"

  • Conflict: editor's model sentences use "This feature provides…" and "Configure these settings:". recursive-writer asks for "offers" and "the following X:".
  • Decision: editor's rule.
  • Why: a single writer's preference, with no reader problem behind it.
  • Change: none.

Structure rules

4. Locating a control by screen position

  • Conflict: editor allows a position when it helps ("Click Save in the top-right corner"). recursive-writer names only the region or panel, never a screen position.
  • Decision: editor's rule.
  • Why: giving a location is standard in technical writing, and leaving it out confuses both people and agents. Almost all users work with the dashboard on desktop, so the layout is stable.
  • Change: none.

5. When steps get numbered

  • Conflict: editor keeps 2–3 actions inline and numbers only 4+ actions, a step that needs separate verification, or an unreadable sentence. recursive-writer makes every sequence of actions a numbered procedure.
  • Decision: neither rule as written: it's a judgment call. Sometimes a two-step numbered list looks odd, sometimes it's the clearest form.
  • Change: editor Area 7 now says when to number (4+ actions, a step that needs its own verification, a sentence that gets hard to read, neighbouring procedures in the same section that are numbered), and that neither form is a defect by itself. The "One concern per unit" rule, which said every prose procedure becomes numbered steps, now points to the same judgment.

6. What an intro opens with

  • Conflict: editor's intro covers what the feature is, why the reader needs it, and when it applies, in that order. recursive-writer opens with who the article is for (app, platform, version, case), then what they get, and uses a definition only where the reader arrives asking "what is this?".
  • Decision: editor's rule.
  • Why: the audience-first opener isn't product-oriented.
  • Change: none.

7. Mixing heading shapes

  • Conflict: editor requires sibling headings to be all verb phrases or all noun phrases. recursive-writer lets one level mix kinds, as long as a reference section never takes a task heading.
  • Decision: both combined.
  • Why: consistency is the right default, but a "Limitations" or "Configuration options" section next to task sections shouldn't be forced into a verb phrase.
  • Change: editor's article-structure.md and writing-planner now say: one shape by default; a reference section among task sections keeps its noun heading; a reference section never takes a task heading, and a task section never takes a noun heading.

9. Passive voice

  • Conflict: editor allows passive voice when the actor is unknown or unimportant ("The request is processed automatically"). recursive-writer requires every sentence to name who acts.
  • Decision: editor's rule.
  • Why: naming an actor the reader doesn't care about adds words without information. The exception stays narrow.
  • Change: none.

10. Sections: entry points or continuations

  • Conflict: editor treats sections as entry points: each one restates the context it needs, and writing-planner requires every H2 to be self-contained. recursive-writer treats a section as continuing from where the previous one left the reader, and its final pass cuts repeated setup.
  • Decision: editor's rule.
  • Why: both people and agents land from search, or read just one heading, and may have no context at that point.
  • Change: none. writing-planner's new path lines set the order of sections only; the self-contained rule stays.

Process rules

11. How fixes get approved

  • Conflict: editor lists all findings, asks whether to go through them, then asks about 4 at a time and edits nothing until the end. recursive-writer keeps one proposal pending at a time, starts in the same reply without asking, and takes approval in plain text.
  • Decision: no change: a matter of taste.
  • Change: none. Separately, after writing a section or more, editor now runs the multi-role review and applies its fixes without asking.

12. Order of findings

  • Conflict: editor groups findings by severity (Critical, then Important), numbered across groups. recursive-writer lists them in article order.
  • Decision: editor's rule.
  • Why: the most important problems come first.
  • Change: none.

13. Style-only suggestions

  • Conflict: editor had a Suggestions tier ("Passive → active where it flows better", "Minor wording improvements"). recursive-writer forbids proposing a change to published text for style preference alone.
  • Decision: recursive-writer's rule.
  • Why: in the old-articles evaluation, editor had the lowest share of accepted findings; style-only proposals are noise.
  • Change: the Suggestions tier is removed from editor's output format. Every finding must name what the reader gets wrong, misses, or can't do. In a new draft, wording is polished during writing instead of reported.

14. When the user corrects a proposal

  • Conflict: editor applies the user's note to the fix. recursive-writer re-proposes the corrected fix as 3 or more structurally different options.
  • Decision: editor's rule.
  • Why: the user already said what they want.
  • Change: none.

Flow & Paywall Builder names

8 / B1. The builder's full name

  • Conflict: editor on main said "Flow Builder". recursive-writer's glossary says Flow & Paywall Builder (capital B, ampersand), in full on first mention and wherever it doesn't hurt readability.
  • Decision: recursive-writer's rule.
  • Change: editor Area 6 now says "Flow & Paywall Builder", not "Flow Builder" or "Flow & Paywall builder".

B2. Later mentions

  • Conflict: editor had no rule. recursive-writer allows "the builder" or "the flow builder".
  • Decision: "the builder" only.
  • Change: in editor/references/product-terms.md.

B3. The deprecated builder

  • Conflict: only recursive-writer has a rule: keep "old Paywall Builder", not "legacy builder"; its paywalls are "paywalls from the old Paywall Builder" or "legacy builder paywall".
  • Decision: recursive-writer's rule.
  • Change: in editor/references/product-terms.md.

B4. Text about both builders

  • Conflict: only recursive-writer has a rule: in text that covers both builder generations, "our builder".
  • Decision: recursive-writer's rule.
  • Change: in editor/references/product-terms.md.

B5. "Flow" for a sequence of steps

  • Conflict: only recursive-writer has a rule: never "flow" for a sequence of steps, since flows are the entity.
  • Decision: recursive-writer's rule.
  • Change: in editor/references/product-terms.md.

Not kept

  • The rw check library as a whole: it didn't beat editor or plain Claude in either review evaluation.
  • The evidence rule: no measurable difference.
  • Repeated wording rounds, three outlines plus a judge, the enforcement hooks, and the mode letters: more time, no better result.

Next steps (not in this PR)

  1. Trim editor's review rules. editor had the lowest share of accepted findings, with no more accepted issues than no skill. Use the minor and rejected findings from evaluation 2 to find the rules that only add noise, and cut them.
  2. Retire recursive-writer and the rw-* modules, in a separate PR. Before deleting, decide on the parts no evaluation tested:
    • Hand-edit tracking (sync.py, snapshots): keep the scripts only if rewording of hand-fixed text is a real problem.
    • rw-comments' PR handling (outdated threads, suggestion blocks, dependent comments): move into review-assistant only if it trips on those cases.
    • Correction-to-rule loop: worth having in doc-author, independent of which rule set it edits.
  3. Version consequences at writing time. The real rw draft's errors came from wrong facts, and the facts both research methods missed are consequences, not lookups: what users on each app version see after each kind of change (a new native view, a changed Type or ID, a reused view, a test before publishing). A candidate rule: for a feature split between dashboard config and app code, list what users on an old and a new app version see after each kind of change. It comes from the same review it would be scored on, so test it on a different feature first.

@eandreeva-twr
eandreeva-twr marked this pull request as ready for review November 11, 2026 15:20
@eandreeva-twr
eandreeva-twr merged commit de9359e into main Nov 11, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant