Skip to content

Generate the Tao Te Ching from the translation, and keep it current - #16

Merged
shalomormsby merged 1 commit into
mainfrom
sources/tao-te-ching-generated
Oct 16, 2026
Merged

shalomormsby merged 1 commit into
mainfrom
sources/tao-te-ching-generated

Conversation

@shalomormsby

Copy link
Copy Markdown
Contributor

Do not merge until opencosmos#222 is deployed. The new headings change every chunk id, and stored citations like #twelve resolve only through #222's mapping in lib/corpus-href.ts. Order: taoteching#30 → opencosmos#222 (deployed) → this.

Summary

  • The stopgap is replaced by a generated document (ADR 0020 §7), written by scripts/knowledge/generate-tao-te-ching.ts from taoteching's data/atlas-export.json. It reads a sibling checkout first, and GitHub otherwise.
    • Each chapter is ## Chapter N: Heshang Gong's title, the English, then the Chinese.
    • The translator's notes go under ### Notes on chapter N, in the 65 chapters that have them. No pinyin.
    • The CC0 frontmatter is rewritten. The non-action tag is dropped (the lock is non-doing), and so is updated_at, so the output depends on the export alone.
    • A YAML comment at the top says the file is generated and how to regenerate it.
  • Two shape decisions, both forced by the embedder, which chunks at H2, H3 and H4:
    • The notes heading names its chapter, because a bare ### Notes would collide 81 times into hashed ids that no citation maps back. Headings inside the notes become bold lines.
    • Every chapter ends with *End of chapter N.*. The embedder opens each chunk with the previous section's last paragraph, so without the marker, chapter 12's chunk would open with a paragraph of chapter 11's notes.
  • .github/workflows/tao-te-ching.yml keeps it current, with no hand step.
    • Triggers: taoteching's text-updated dispatch, a daily run as the fallback, and workflow_dispatch (with ref and dry_run).
    • What it does: it regenerates from taoteching's current main, never the signal's sha, so a late signal can't roll the text back. It stops quietly if nothing changed, and otherwise commits straight to main (your call).
    • Then it runs Knowledge Sync itself. A push made with GITHUB_TOKEN triggers no other workflow, so without this step the corpus would never re-embed. workflow_dispatch is the one trigger that token may fire.
  • groom.py no longer knows this file. Its processor wrote "Translated by Gia-Fu Feng and Jane English" into it, so groom --force would have put that credit back.
  • CONTRIBUTING says this file is the one exception to "Suggest an edit opens the file here".

Counts (verified by count, not exit status)

main (stopgap) this branch
Tao Te Ching chunks (embed:dry ids) 83 148 = 81 chapter-N + 65 notes-on-chapter-N + intro + curator-s-note
Word-heading chunks (#one … #eighty-one) 81 0; the real embed reconciles them away
Constellation section nodes (H2s) 82 82 (81 chapters + Curator's Note)
Feng–English chunk ids 0 0
npm run health — identical to main
  • Chunk sizes. Every chapter chunk fits Cosmo's 2,000-character chunk text; the largest is chapter 39 at 1,285.
  • Long notes are cut off in retrieval. 11 notes sections run over 8,000 characters, and a retrieved chunk shows Cosmo only the first 2,000. The chat's "the reader is on chapter N" context injects the full section, so reading in context is unaffected.

Test plan

  • npm run ttc:generate writes the file. --check then reports it current, so the generator is deterministic.
  • --ref export/reader-fields, fetched from GitHub, is byte-identical to the local sibling's output.
  • On today's main, the generator refuses taoteching's export, because #30 hasn't merged (no "verse"). That's intended.
  • npm run health, the same as main. It caught three relative links in the notes, which led to a fix in taoteching#30's export.
  • npm run quotes:lint is OK.
  • The workflow YAML parses. It has not run: a workflow only runs once it's on the default branch.
  • End to end, after merge:
    1. workflow_dispatch with ref=main should end in "already current".
    2. Then ref=<a taoteching test branch> with dry_run=true shows the regenerated diff.
    3. Then the next real taoteching change (or #30's own merge signal) shows the commit, Knowledge Sync, and the embed's counts.

What's needed from Shalom

  • KNOWLEDGE_DISPATCH_TOKEN in taoteching. It's a fine-grained token: resource owner opencosmos-ai, only opencosmos-ai/knowledge, Contents read and write. Until it exists, the daily run catches changes within a day.
  • The cut-off long notes. Splitting them would need another heading scheme, so I've left them for now. Say if you want them split.

🤓 + 🤖 Created by Shalom with the assistance of Claude Code

Replaces the stopgap with the document opencosmos ADR 0020 §7 describes,
written by scripts/knowledge/generate-tao-te-ching.ts from taoteching's
data/atlas-export.json: one '## Chapter N' per chapter with the English and
the Chinese, the translator's notes under '### Notes on chapter N', no
pinyin. The output depends on the export alone, so an unchanged translation
regenerates byte-identical.

.github/workflows/tao-te-ching.yml regenerates it on taoteching's
text-updated signal and daily, commits it to main (Shalom's call), and runs
Knowledge Sync itself, because a push made with GITHUB_TOKEN triggers no
other workflow.

groom.py no longer knows this file: its processor wrote Feng and English's
credit into it.

Must merge only after opencosmos#222 is deployed: the new headings change
every chunk id, and the old #twelve citations resolve only through its
corpus-href mapping.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@shalomormsby
shalomormsby merged commit b6df8eb into main Oct 16, 2026
1 check passed
@shalomormsby
shalomormsby deleted the sources/tao-te-ching-generated branch October 16, 2026 05:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant