DVB-TTML implementation - #1242
Draft
kierank wants to merge 12 commits into
Draft
Conversation
…nition The sub stored a new flow definition on its output the moment it arrived, while the subpictures in its schedule keep being output long after: pages decode ahead of their pts, so the pixels of the on-screen page went downstream under the flow definition of the next one. This was harmless while all subpictures of a flow had the same dimensions, but with the pic size and padding attributes changing per subpicture, downstream rescaled the current page to the geometry of the next page for the whole decode-ahead window, stretching it and putting it in the wrong place. Pair each buffered subpicture with the flow definition it was received under, and store that on the output when the subpicture starts being output. The output helper deduplicates identical flow definitions, so flows with constant dimensions behave as before. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The vertical scaling of the destination rectangle derived bpad from tpad, so an asymmetric padding put the picture at the wrong size and position. Only subpictures whose flow definition carries pic padding attributes are affected; for everything else both are zero. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The renderer read the coverage of a glyph at buffer[j * width + i], which is only where it is when the bitmap has no padding between its lines. Both the small bitmap cache and a rendered glyph say what the distance between two lines is, so use it: with a padded bitmap the glyph was sheared by a pixel per line and read past the end of the buffer on the last ones. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The left of the box was taken from the first glyph under "if (!i)", but the string index is advanced before the test, so it was never zero and the box always started at zero however far into the line the first glyph sat. Take it from the first glyph that is actually looked up. The height was assigned twice, the second time without the guard the first one had, which made the guard do nothing; drop the second one. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The scaler handed to the cache manager is a stack structure with two fields left unset; the manager keeps a copy of it. They are the resolutions, which it only reads when the sizes are not in pixels, so nothing came of it, but zero them rather than hash and store whatever was on the stack. FT_Get_Advance leaves its output untouched when it fails, and the result was added to the total either way. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Three different mismatches shared one message that named none of them, so a subpicture being dropped told you only that it was. Split the unset destination rectangle and the unreadable picture into their own messages, and give the size mismatch the size that arrived and the size that was expected. The flow format the sub provides upstream is what decides that size, so log what it worked out and the canvas and padding it worked it out from. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
EN 303 560 signals a DVB TTML subtitle stream with stream_type 0x06 and extension descriptor 0x20 in the PMT, so the descriptor is what makes the ES a subtitle stream, as for teletext and DVB subtitles. Give it the flow definition block.dvb_ttml_subtitle.pic.sub. and export the language, the subtitle purpose, the text-to-speech suitability, the processor profiles and the qualifier on the flow definition. Extension descriptors used to fall through to the default case and be copied verbatim onto the flow definition; the other tag extensions keep that behaviour. The maximum retention time is the maximum period of activation of a segment, 5 seconds (5.2.3.3): nothing in the stream is buffered for longer. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The PES payload of a DVB TTML subtitle stream is not an elementary stream: it is a PES_data_field (EN 303 560 5.2.2.2) carrying a segment media time, a list of subtitle segments and a CRC. Unwrap it into a stream of TTML documents, block.ttml.pic.sub., each dated with the PTS of its PES packet and carrying its segment media time, which is what converts TTML times to the MPEG timeline (5.2.4.1). The PES decapsulator hands over the payload of one transport stream packet at a time, marking the first chunk of a PES packet and its last and dating the first, so the packet is put back together before it is parsed: a PES_data_field only has a checkable CRC once all of it is there. A packet that never ends is handed over at the next one and left to the CRC, and one that outgrows a PES packet is dropped. An uncompressed segment is output by moving the block window onto the document, so the payload is never copied. A gzip segment (type 0x02) is inflated straight into a block taken from the ubuf manager, when zlib is available. Nothing is allocated per packet. A segment whose CRC fails is dropped rather than passed on, which leaves the previously received segment active until it expires, as 5.2.4.2 requires. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The pipe rendered one string, in one colour, at one size, on a baseline the application picked. A subtitle page is a handful of regions, each holding runs with their own colour, size and background, wrapped and aligned inside the region: describe one with the uref_text attributes and let the renderer measure the runs, break them into lines and lay them out. The picture is cropped to what is drawn and the distance to each edge of the canvas goes on the flow definition as a pic padding, the convention the DVB subtitle decoder already uses: downstream rebuilds the canvas from the size and the paddings, so two lines of subtitle are not rescaled and alpha blended over a whole frame-sized picture on every frame. Nothing is advertised until there is something to describe: a first negotiation at the size of the canvas settles on a chain that does not scale, and no later crop gets a chance to undo it. Compositing divides by the resulting alpha rather than mixing the source colour straight in. On the opaque background of the single string path the two are the same expression, but a page starts out transparent, and mixing in the colour of a transparent pixel darkens the edge of every glyph. Neither of the two ends of that needs the division: nothing underneath leaves the source colour as it stands, and an opaque result is a mix by 255, which is a pair of shifts. Subsampling is a shift too, so a pixel costs no division at all unless it is translucent over translucent. Only one font file is configured, so bold and italic are synthesized by emboldening and shearing a copy of the cached outline; the plain style still comes from the small bitmap cache. Glyph blitting honours the pitch of the bitmap, which is not its width for a glyph the small bitmap cache would not hold. A page is bounded, so the scratch it is laid out in comes with the pipe and rendering one allocates nothing. The smallest picture is the floor the DVB subtitle decoder uses for an empty subtitle: a couple of pixels is not a legal size once the chroma is subsampled. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The pipe reads the TTML documents the demuxer produces and turns each of them into the styled text pages of uref_text, which upipe_freetype renders. A document is a sequence of intermediate synchronic documents: whenever a piece of content becomes active or inactive one ends and the next begins. Collect the times at which that happens inside the activation window of the segment, and emit one page per interval, dated with PTS(Tx) = Pi + (Tx - Ti) as EN 303 560 5.2.4.1 defines, and given the duration of the interval so the scheduler holds it for exactly that long. An interval with no active content emits a page with no region, which clears the screen. The times in a document are on the timeline of the document, which begins at zero, and the segment media time is the point of that timeline the PTS corresponds to. Resolving them against the activation window instead would offset every one of them by the media time and leave nothing inside it. Styling covers what DVB TTML subtitles use: referential and nested styles, regions with an origin, an extent and a padding, displayAlign and textAlign, per span colour, background colour, font size, weight, style and decoration, line height, wrapping, and the EBU-TT-D linePadding, multiRowAlign and fillLineGap. A referenced style is applied onto the inherited context rather than resolved on its own, so a font size given as a percentage resolves against the size the referencing element inherits. Alignment and line styling are content properties that EBU-TT-D puts on the paragraph as often as on the region, so the region takes what its first paragraph gives and falls back on its own. The canvas the regions are positioned on has to be set by the application: there is no default, because a canvas of the wrong size puts every region in the wrong place rather than merely rendering them at the wrong resolution. The document is parsed out of the block the demuxer hands over rather than a copy of it, and a page is laid out in scratch that comes with the pipe. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The PES decapsulator hands over the payload of one transport stream packet at a time, so a PES_data_field only exists, and only has a checkable CRC, once the chunks marked with the block start and end flags have been put back together. That is what a framer does, and DVB subtitles and teletext, both PES carried, segment structured and no more an elementary stream than this one, are already framed by upipe_dvbsubf and upipe_telxf. The CRC is the only structural check EN 303 560 offers, and it is what says the reassembly was right, so it belongs in the framer as well. It is taken one ubuf at a time over the chain the reassembly builds, so the framer needs no ubuf manager and never has to merge. The framer also sets what the decapsulator never did: the flow definition is marked complete, and the DTS to PTS delay is zeroed because the PES header of a TTML subtitle stream carries a PTS and no DTS. The T-STD octet rate and buffer size are left unset for now. autof allocates it on block.dvb_ttml_subtitle., as it does for the other two.
The pipe reassembled the PES packets itself, checked their CRC and then unwrapped the TTML document out of them. upipe_ttmlf does the first two now, so drop them here: what is left is the decapsulation proper, the PES data field header, the segment walk, the gzip inflation and the media time, and the pipe receives one whole data field per uref. ts_demux therefore plumbs the TTML output as pesd, autof and ts_ttmld, with autof picking the framer, and takes that branch only once an autof manager has been set.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.