Skip to content

Render SmartArt diagrams and charts instead of dropping them - #185

Open
ikashapov wants to merge 2 commits into
nerdy-pro:mainfrom
ikashapov:issue-155-smartart-charts
Open

ikashapov wants to merge 2 commits into
nerdy-pro:mainfrom
ikashapov:issue-155-smartart-charts

Conversation

@ikashapov

Copy link
Copy Markdown
Contributor

Render SmartArt diagrams and charts instead of dropping them

Closes #155. Based on main (b2c0174, v0.5.1); one commit, branch
issue-155-smartart-charts. Independent of the other open branches.

Both are DrawingML parts referenced from a:graphicData, and both died at
the same seam: the graphicData schema knew only pic and wsp, so a
dgm:relIds or c:chart child parsed to graphic = None and the drawing
vanished — no space reserved, nothing drawn, not even a warning.

SmartArt — render the baked layout, not the algorithm

The issue guessed SmartArt "may need routing more than new rendering", and
that is exactly how it lands. Word ≥ 2007 SP2 writes a dsp: drawing
part
([MS-ODRAWXML]) into every SmartArt — "the last successful layout of
the diagram": literal shapes with baked fills and fitted text, in absolute
EMU on a canvas the size of the hosting wp:extent. LibreOffice's own
import uses it and throws its approximate layout engine's result away when
it's present; implementing §21.4's layout algorithm instead is a
multi-year effort that would still be approximate. So: parse the part,
route it through the existing shape pipeline.

Two wiring quirks are load-bearing and recorded in the module doc, both
verified against a Word-produced file during research: the drawing part is
named by the data part's dsp:dataModelExt/@relId extension, and both
that id and dgm:relIds/@r:dm resolve against the host part's rels
(document or header), not the data part's own. The dsp: wrappers contain
plain a: DrawingML, so dsp:spPr parses through the existing SpPrXml,
dsp:style through ShapeStyleXml, and rendering is per-shape
build_geometry → resolve_shape_visuals → the shared shape-text
sub-layout, with labels placed in the shape's own dsp:txXfrm rectangle
(SmartArt's way of keeping text upright in rotated shapes). Nested
dsp:grpSp levels fold into their leaves at parse via §20.1.7.5's
child-space affine. A diagram without a drawing part (Word 2007 RTM,
non-Office producers) warns and draws nothing — the recorded limit.

Found in passing: the color schema expected scRgbClr, but
§20.1.2.3.30's element — and what Word actually writes, e.g. in every
dsp:style placeholder — is scrgbClr. Every real SmartArt would have
failed the whole parse on it. Fixed with the spec spelling primary and the
camel-case form kept as an alias.

Charts — drawn from the cached data

c:numCache/c:strCache are "the last data shown on the chart", and
every producer that matters writes them on save — so the renderer never
opens the embedded workbook. Drawn: 2D bar/column (clustered, stacked,
percentStacked, gapWidth/overlap incl. Word's implicit 100 for
stacks), line (blank cells break the polyline, per dispBlanksAs's
default), pie/doughnut (per-point accent colors under varyColors,
dPt overrides, firstSliceAng, holeSize, white slice separators),
area (incl. stacked), scatter (markers always, line only when the
series asks). Axes follow Excel's automatic bounds — 5% margin snapped
outward to a 1/2/5×10ᵏ unit, zero pinned unless the data floats above ⅚
of its max — plus manual min/max, maxMin reversal, c:delete
(scales the plot but isn't drawn), and majorGridlines. Legend on any
edge (categories for a one-series pie, series names otherwise), rich-text
titles and the auto series-name title. Text and chrome defaults come from
Word's own chart-style part: 9pt labels / 14pt titles, theme minor face,
#595959 text, #D9D9D9 0.75pt chrome, 2.25pt series lines, the accent1–6
cycle darkened per completed lap.

Declines are visible, not silent: radar/surface/bubble/stock warn with
their element name; 3D kinds flatten to their 2D twins at parse (the
LibreOffice reading — flattening beats refusing).

What the pipeline gained (plain shapes benefit too)

  • Preset geometries beyond rect/line: roundRect, ellipse, the
    four block arrows, chevron, homePlate, diamond, triangle — the
    shapes SmartArt's layouts actually emit, from their spec guide formulas,
    with arcs as cubic Béziers and each generator's text rectangle.
  • DrawingML text (a:txBody): shared schema + model, converted to
    blocks at build (where the theme can resolve scheme colors), consumed by
    SmartArt labels and chart titles alike.
  • Fragment::Scene — the inline vector-scene channel. Inline drawings
    other than pictures previously rendered nothing at all; scenes now
    occupy their extent like an image and carry pre-built commands.
  • Anchored scenes ride the existing floating-shape channel (empty
    geometry + scene commands), keeping text wrapping, distances and
    behind-text for free.

Tests and fixtures

  • test-files/smartart.docx — a three-node process + arrow wired exactly
    as Word wires it, including the host-rels quirk. test-files/charts.docx
    — a clustered column chart (title/legend/gridlines), a varyColors pie,
    a line with a blank-cell gap and markers. Both byte-reproducible
    (scripts/make_issue155_fixtures.py) and carrying the stock Office
    theme, so accent RGBs are deterministic literals in the tests.
  • tests/smartart_charts.rs (8): parse payloads; per-node accent colors
    and EMU→Pt positions; label containment and fill-under-label z-order;
    bar count and 7:4/5:4 height ratios; pie slice count/colors and its
    category legend; the polyline break and the four markers; an anchored
    chart landing at its anchor offset with wrapping intact.
  • Unit tests: preset geometry tables (corner counts, adjust clamps, arrow
    tips per orientation), the diagram group-affine composition, the
    dataModelExt extraction, the chart schema (CT_Boolean's bare-element
    trap, sparse caches, 3D flattening, scatter pairing), the auto-scale
    table (zero pinning, the high-float release, the 1/2/5 ladder, manual
    bounds, percent), stacking ranges, gap runs.
  • Mutation-checked: four deliberate breaks (scenes never populated,
    diagram transforms dropped, bar value scale constant, anchored arm
    severed) each fail their own tests.
  • Corpus: all 28 pre-existing fixtures render byte-identical PDFs
    against main; the new fixtures verified visually (CoreGraphics
    raster) — accent-colored rounded nodes with white centered labels and
    an orange arrow; bars at the right heights with legend and gridlines,
    pie shares at 40/30/20/10, the line gap at the blank cell.

Gates

cargo fmt --check, clippy --all-targets -- -D warnings,
RUSTDOCFLAGS="-D warnings" cargo doc --no-deps,
cargo build --no-default-features and the full 45-target suite pass.
No new dependencies; public API unchanged (two new Document fields).

@ikashapov
ikashapov force-pushed the issue-155-smartart-charts branch 6 times, most recently from dc941f2 to 3826f2c Compare September 12, 2026 09:48
@ikashapov
ikashapov force-pushed the issue-155-smartart-charts branch 2 times, most recently from 3fe9702 to e24b990 Compare September 18, 2026 02:49
Both are DrawingML parts referenced from a:graphicData, and both vanished
at the same seam: GraphicDataXml knew only pic and wsp, so a dgm:relIds
or c:chart child parsed to graphic=None and the drawing dropped without
so much as a warning — no space reserved, nothing drawn (issue nerdy-pro#155).

SmartArt takes the LibreOffice route: render the pre-laid-out dsp:
drawing part ([MS-ODRAWXML]) that Word ≥2007 SP2 writes — "the last
successful layout of the diagram" — rather than implement the §21.4
layout algorithm (which LO, after years, still only approximates). The
part's wrappers are dsp: but everything inside is plain a: DrawingML,
so dsp:spPr parses through the existing SpPrXml, dsp:style through
ShapeStyleXml, and rendering walks the ordinary shape pipeline:
build_geometry, resolve_shape_visuals, the shared shape-text
sub-layout — with the label placed in the shape's own dsp:txXfrm
rectangle. Coordinates are absolute EMU on a canvas equal to wp:extent;
nested dsp:grpSp levels fold into their leaves at parse via the
§20.1.7.5 child-space affine. Two wiring quirks are load-bearing and
recorded: the drawing part hangs off the *data* part's
dsp:dataModelExt/@relid, and both that id and r:dm resolve against the
HOST part's rels, not the data part's. A diagram without the drawing
part (2007 RTM, non-Office producers) warns and draws nothing.

Charts render from the part's cached data — c:numCache/c:strCache,
"the last data shown on the chart", which every real producer writes —
so the embedded workbook is never opened. Drawn: 2D bar/column
(clustered, stacked, percentStacked with Word's implicit overlap 100),
line (blank cells break the polyline per dispBlanksAs' default), pie
and doughnut (per-point accent colors under varyColors, dPt overrides,
firstSliceAng, white slice separators), area, and scatter; axes with
Excel's automatic bounds (5% margin snapped to a 1/2/5×10^k unit, zero
pinned unless the data floats above ⅚ of its max), manual min/max,
maxMin reversal, c:delete (scales but does not draw), majorGridlines;
legend (r/l/t/b/tr, overlay), rich-text and auto series-name titles.
Defaults follow Word's own chart-style part: 9pt labels and 14pt titles
in the theme minor face at #595959, 2.25pt series lines, 0.75pt #D9D9D9
chrome, the accent1–6 cycle darkened per completed lap. 3D kinds
flatten to their 2D twins at parse (the LO reading); radar, surface,
bubble and stock decline with a warning naming them.

Plumbing both needed and plain wps shapes gained: a shared a:txBody
(DrawingML text) schema and model, converted to blocks with scheme
colors resolved at build; preset geometries beyond rect/line —
roundRect, ellipse, the four block arrows, chevron, homePlate, diamond,
triangle — as spec-formula generators (arcs as cubics, k=0.5523);
Fragment::Scene, the inline vector-scene channel (inline drawings other
than pictures previously rendered nothing at all); anchored scenes ride
the floating-shape channel, keeping wrapping and behind-text; and the
color schema now accepts §20.1.2.3.30's actual element spelling
scrgbClr (the camel-case scRgbClr it expected is not what Word writes —
every real dsp:style would have failed the parse).

Fixtures (scripts/make_issue155_fixtures.py, byte-reproducible, stock
Office theme so accent RGBs are deterministic literals): smartart.docx
wires a three-node process + arrow exactly as Word does; charts.docx
carries a clustered column chart with title/legend/gridlines, a
varyColors pie, and a line with a gap and markers. tests/
smartart_charts.rs pins parse payloads, per-node colors and EMU→Pt
positions, label containment and z-order, bar-height ratios, slice
counts and colors, the polyline break, and anchored placement at the
anchor offset. Four deliberate breaks (scenes unpopulated, diagram
transform dropped, bar scale constant, anchored arm severed) each fail
their own tests. The 28 existing corpus documents render
byte-identically before and after.

Closes nerdy-pro#155
…n it

Per ECMA-376 §20.1.9.3 the pen lies on the ellipse at stAng; the old code
used the pen as the ellipse center, displacing every arc by a radius and
adding a spurious pen-to-arc line. Also converts stAng/swAng ray angles to
Skia's parametric angles (matters when wR != hR) and splits >=360deg sweeps
so full circles draw instead of collapsing modulo 360.

Verified against Apache POI and LibreOffice; adds spec-derived unit tests
and the custgeom-arc.docx fixture.
@ikashapov
ikashapov force-pushed the issue-155-smartart-charts branch from e24b990 to 3683223 Compare September 22, 2026 09:39
ikashapov added a commit to ikashapov/dxpdf that referenced this pull request Oct 14, 2026
One squashed commit carrying the tree of integration/2026-10-14, which was
previously a chain of ten merges ending at ac5787e (kept locally as
archive/integration-2026-10-14-merges). Base: upstream main 3a5c7ca.

Integrated, each at the head of its open upstream pull request:

- nerdy-pro#252  fix-layout-defects-228-232 (a40d5ce) — issues nerdy-pro#228–nerdy-pro#232: bookmark-only
        paragraphs, nested-table width, list instances over one abstract
        numbering, table width resolution (pct and tcW autofit), column
        balancing and separators
- nerdy-pro#183  issue-150-wmf-svg (6e5aa99) — WMF/EMF and SVG images
- nerdy-pro#184  issue-151-pgborders-docgrid (27204b3) — page borders, document grid
- nerdy-pro#186  issue-153-shaped-clusters (65739f6) — shaped text clusters
- nerdy-pro#187  issue-154-tracked-changes (6982b34) — tracked changes and comments
- nerdy-pro#185  issue-155-smartart-charts (3683223) — SmartArt and charts
- nerdy-pro#256  issue-253-del-instr-text (0656a52) — w:delInstrText tolerated
- nerdy-pro#257  issue-250-image-hyperlinks (cb5ce2b) — a:hlinkClick on pictures
- nerdy-pro#258  issue-188-font-absent-cost (9a63bf0) — no deep catalogue tier before
        the substitution table, no theme script-font preload
- nerdy-pro#259  issue-158-auto-hyphenation (9bfbae2) — w:autoHyphenation

Beyond the union of those branches, the integration itself changed:
- README keeps one row per feature after the merges; the matrix tally is
  recomputed from the rows.
- nerdy-pro#253's tests assert that a deleted field code is never drawn and never run
  text, because nerdy-pro#154 keeps w:del runs instead of dropping them at parse.
- nerdy-pro#250's push_link_over sits after nerdy-pro#151's resolve_line_metrics; nerdy-pro#158's
  HyphenationCtx.measure and rejoin_hyphenated take &dyn SplitMeasure (nerdy-pro#153).

Verified on this exact tree: cargo fmt, clippy and doc clean; cargo test
2977 passed / 0 failed (skipping the capi library freshness guards);
--no-default-features and --features capi build; the twelve-document
harness passes 12/12.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Render SmartArt diagrams and charts

1 participant