Repository navigation
Conversation
ikashapov
force-pushed
the
issue-153-shaped-clusters
branch
6 times, most recently
from
September 12, 2026 09:48
603216b to
ef48207
Compare
ikashapov
force-pushed
the
issue-153-shaped-clusters
branch
2 times, most recently
from
September 18, 2026 02:49
137dd33 to
e3c277d
Compare
Closes nerdy-pro#153. Two halves, in the order the issue names them: The predicate: `needs_shaping` gains a second property — the Brahmic scripts whose OpenType shaping reorders glyphs relative to characters (Devanagari's prebase matra is stored after its consonant and drawn before it), which `Joining_Type` cannot see because those letters never join. The list is drawn from the shaping models themselves and excludes Thai and Lao (vowels stored in visual order; their widths must not move). The unit: §17.3.2.35 `w:spacing` and §17.3.1.13 `distribute` on a shaped run now operate on the shaped cluster — HarfBuzz cluster values, folded to UAX nerdy-pro#29 grapheme starts so an unreordered combining mark can never become a unit of its own. One HarfBuzz pass per shaped fragment measures the advance and the unit count; the count rides on the fragment for distribution to read without a measurer, and the painter groups its own shaped glyphs by the same values, shifting each visual rank by one spacing step in the same visual-gap model distribution reserves by. Along the way this repairs four standing defects on shaped runs: a `w:spacing` run in Arabic was measured and painted with no spacing at all; `distribute` stretched a shaped run's underline and borders while the painter left its glyphs unspread; §17.3.2.45 `w:w` scaled the reserved box but not the painted glyph positions (the painter now scales shaped positions); and the emergency splitter measured pieces of a shaped word by cmap while the painter drew them shaped — for an Indic conjunct the cmap sum runs half again wider than the ligature, so the splitter now measures pieces through the shaper (`SplitMeasure`). The 28-document corpus renders pixel-identical (the Latin fast path is untouched); the new devanagari.docx fixture pins the reorder, the conjunct-safe spacing, both distribute cases, and the Latin control.
ikashapov
force-pushed
the
issue-153-shaped-clusters
branch
from
September 22, 2026 09:39
e3c277d to
65739f6
Compare
ikashapov
added a commit
to ikashapov/dxpdf
that referenced
this pull request
Oct 14, 2026
One squashed commit carrying the tree of integration/2026-10-14, which was previously a chain of ten merges ending at ac5787e (kept locally as archive/integration-2026-10-14-merges). Base: upstream main 3a5c7ca. Integrated, each at the head of its open upstream pull request: - nerdy-pro#252 fix-layout-defects-228-232 (a40d5ce) — issues nerdy-pro#228–nerdy-pro#232: bookmark-only paragraphs, nested-table width, list instances over one abstract numbering, table width resolution (pct and tcW autofit), column balancing and separators - nerdy-pro#183 issue-150-wmf-svg (6e5aa99) — WMF/EMF and SVG images - nerdy-pro#184 issue-151-pgborders-docgrid (27204b3) — page borders, document grid - nerdy-pro#186 issue-153-shaped-clusters (65739f6) — shaped text clusters - nerdy-pro#187 issue-154-tracked-changes (6982b34) — tracked changes and comments - nerdy-pro#185 issue-155-smartart-charts (3683223) — SmartArt and charts - nerdy-pro#256 issue-253-del-instr-text (0656a52) — w:delInstrText tolerated - nerdy-pro#257 issue-250-image-hyperlinks (cb5ce2b) — a:hlinkClick on pictures - nerdy-pro#258 issue-188-font-absent-cost (9a63bf0) — no deep catalogue tier before the substitution table, no theme script-font preload - nerdy-pro#259 issue-158-auto-hyphenation (9bfbae2) — w:autoHyphenation Beyond the union of those branches, the integration itself changed: - README keeps one row per feature after the merges; the matrix tally is recomputed from the rows. - nerdy-pro#253's tests assert that a deleted field code is never drawn and never run text, because nerdy-pro#154 keeps w:del runs instead of dropping them at parse. - nerdy-pro#250's push_link_over sits after nerdy-pro#151's resolve_line_metrics; nerdy-pro#158's HyphenationCtx.measure and rejoin_hyphenated take &dyn SplitMeasure (nerdy-pro#153). Verified on this exact tree: cargo fmt, clippy and doc clean; cargo test 2977 passed / 0 failed (skipping the capi library freshness guards); --no-default-features and --features capi build; the twelve-document harness passes 12/12.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Shape Indic scripts, with the shaped cluster as the spacing unit
Closes #153.
What
Two halves, in the order the issue names them:
The predicate.
needs_shapinggains a second property: the Brahmicscripts whose OpenType shaping reorders glyphs relative to characters — a
Devanagari prebase matra is stored after its consonant and drawn before it,
which
Joining_Typecannot see because those letters never join. The list(Devanagari, Bengali, Gurmukhi, Gujarati, Oriya, Tamil, Telugu, Kannada,
Malayalam, Sinhala, Khmer, Myanmar) is drawn from the OpenType shaping models
themselves; Thai and Lao are deliberately excluded — their vowels are stored
in visual order, so the cmap walk is already right and their measured widths
must not move. New dependency:
unicode-script.The unit. §17.3.2.35
w:spacingand §17.3.1.13distributeon a shapedrun now operate on the shaped cluster — HarfBuzz cluster values collected
from Skia's shaper and folded to UAX #29 grapheme starts, so an unreordered
combining mark can never become a spacing unit of its own (Skia reports
character-level clusters; unfolded,
distributeopened a gap between ह andits own matra). One HarfBuzz pass per shaped fragment measures both the
advance and the unit count; the count rides on the fragment so distribution
can count units without a measurer, and the painter groups its own shaped
glyphs by the same values, shifting each visual rank by one spacing step in
the same visual-gap model distribution reserves by.
Standing defects repaired along the way
w:spacingrun in Arabic was measured and painted with no spacing atall (the documented pre-Complex-script shaping: make the spacing unit the shaped cluster so Indic reordering works #153 state).
distributestretched a shaped run's underline, borders and hyperlink rectwhile the painter left its glyphs unspread.
w:wscaled the reserved box but not the painted glyphpositions of a shaped run — glyphs overlapped under expansion and the
underline overran them.
painter drew them shaped; for an Indic conjunct the cmap sum runs ~1.8× the
ligature width. Pieces are now measured through the shaper (
SplitMeasuretrait, implemented by
TextMeasurer).Verification
Latin/Cyrillic/CJK/Thai fast path is untouched.
test-files/devanagari.docx(built byscripts/make_devanagari_fixture.py, OPC-verified) pins: the matrareorder, conjunct-safe
w:spacing,distributeover Devanagari and Arabiclines, and a Latin
w:spacingcontrol that must stay off the shaping path.Exercised by
tests/devanagari_shaping.rswith structural assertions (nohost font names) and graceful skips for hosts without Devanagari coverage
(ubuntu CI).
the shaped split measure and the RTL tie-break each carry a
mutation-checked test — every deliberate break was caught by exactly the
tests written for it.
cargo fmt/clippy -D warnings/test --all/ doc /--no-default-featuresallgreen.
README coverage table updated (76 ✅ / 12⚠️ / 10 ❌; the Indic row flips to
✅, the FAQ gap list drops it), AGENTS.md corpus table and entry-point notes
updated.