add <track> for time-aligned media transcripts - #175
Conversation
- New `<track>` semantic element: `<bdiv/>`-delimited cue blocks, each with a mandatory start
time and optional end time, an optional `<frame>` and `<audio>`, and a transcript of
`<voice>`-attributed turns
- Semantics: cue blocks cover inclusive `[start, end]` intervals (missing end means
`[start, start]`) and are ordered by non-decreasing start time, overlaps allowed
- Validation: XSD elements/groups + `top_level_cat` entry + range facets; Schematron rules for
cue-block structure, timestamp order, and block sequence
- Archive: whole-track media in `audio/` and `video/` (`{N}.{ext}` by `<track>` position);
`pack()` and `doclang pack` gain `--audio`/`--video`; media content types added; archive-demo
gains a track and a spoken `audio/1.wav`
- Docs/tests: spec `### Tracks` section, regenerated `### Reference` (drops the abandoned
`<timestamp>`/`<hour>`/… draft rows), token vocab, and track + packaging fixtures
Signed-off-by: Panos Vagenas <[email protected]>
- `<chapter>`: an optional cue-block title, placed after the timestamps and before `<frame>`; it marks a chapter boundary at the cue block's start time only, and chapters form a flat non-overlapping partition of the timeline (no nesting, no explicit end — the next `<chapter>` or `</track>` closes it) - `<audio>` is now the recording over the cue block's `[start, end]` interval — a temporal crop, the way a `<picture>` crop corresponds to its page region; timestamps are authoritative, the clip is a best-effort fragment, and its exact duration is not checked. `<frame>` stays the still at the start time - Schematron: `track-audio-requires-end` (a cue block with `<audio>` must carry an end time) and `track-chapter-strictly-increasing` (two chapters cannot mark the same instant) - token vocabulary gains `<chapter>` / `</chapter>`; `### Reference` regenerated with the `<chapter>` entry and the revised `<frame>`/`<audio>` wording - tests: chapter fixtures (valid + invalid) and `nok_track_audio_without_end` Signed-off-by: Panos Vagenas <[email protected]>
Signed-off-by: Panos Vagenas <[email protected]>
Signed-off-by: Panos Vagenas <[email protected]>
ceberam
left a comment
There was a problem hiding this comment.
I added a couple of comments below.
And just a reflection on the design choice for <voice>, compared to WebvTT <v> span (not highlighting any defect though and going very fine-grain). In WebVTT, the <v speaker> span wraps the spoken text inline, so speaker and utterance are in the same node. In DocLang's <track>, the <voice> precedes the text as a sibling. This means <voice> has no explicit close: a turn runs until the next <voice> or </bdiv>. The structural intent is the same, but the representation differs and it may create some gaps:
- no XML structure encodes that boundary: a tree walker (from a consumer application) has to reconstruct it by walking forward from each
<voice>until the next one - mixed attributed and unattributed content within the same cue could not be captured, like in the 3rd block of WebVTT EXAMPLE 6.
- word-level timing is not expressible (this is not supported in Docling's WebVTT parser either, by the way), just at cue block interval.
Signed-off-by: Panos Vagenas <[email protected]>
|
@ceberam thanks for your comments
This is consistent with DocLang's delimiter + virtual-text pattern (e.g. in lists, tables)
It actually can with an empty |
This can indeed be done via cue blocks (with hardly any information loss), e.g. from #175 could become: <doclang>
<track>
<bdiv/>
<seconds value="0"/>
<msecs value="30"/>
<seconds value="0"/>
<msecs value="389"/>
the
<bdiv/>
<seconds value="0"/>
<msecs value="389"/>
<seconds value="0"/>
<msecs value="750"/>
quick
<bdiv/>
<seconds value="0"/>
<msecs value="750"/>
<seconds value="1"/>
<msecs value="110"/>
brown
<bdiv/>
<seconds value="1"/>
<msecs value="110"/>
<seconds value="2"/>
<msecs value="669"/>
fox
</track>
</doclang> |
<track>semantic element:<bdiv/>-delimited cue blocks, each with a mandatory start time and optional end time, an optional<frame>and<audio>, and a transcript of<voice>-attributed turns[start, end]intervals (missing end means[start, start]) and are ordered by non-decreasing start time, overlaps allowedtop_level_catentry + range facets; Schematron rules for cue-block structure, timestamp order, and block sequenceaudio/andvideo/({N}.{ext}by<track>position);pack()anddoclang packgain--audio/--video; media content types added; archive-demo gains a track and a spokenaudio/1.wav### Trackssection, regenerated### Reference(drops the abandoned<timestamp>/<hour>/… draft rows), token vocab, and track + packaging fixtures