Split large artwork images across multiple messages - #188
Merged
Conversation
An artwork image larger than one Noise frame (65518 bytes of payload) currently relies on transport fragmentation, which admits no interleaving: the sender must finish the fragmented message before any other frame, so the whole image is an indivisible burst that audio chunks cannot preempt. A player holding only min_buffer_ms of audio can underrun behind it; on a live stream that truncates audio. With BMP gone (#168) the worst case shrank, but photographic JPEG still exceeds one frame beyond roughly 500x500 (the spec's own stream/start example, 800x800 JPEG, is 3-5 frames), and PNG exceeds it at almost any artwork size. Changes: - An image is transferred as a fixed 14-byte announce message carrying the timestamp and the image's total_size (uint32), followed by parts carrying only image data, all on the channel's message type; a flags byte after the type byte distinguishes the message kinds, following the shape #172 gave fragmentation - The announce carries no image data, so a client can allocate its image buffer before any image bytes arrive and write each part directly into it; the transfer completes when the received data reaches total_size - Every artwork message is capped at 65518 bytes after the type byte so it never needs transport fragmentation; any other messages MAY be sent between the messages of a transfer, and transfers on different channels are independent - The pending image is now the most recently announced image, from its announce until it becomes current: an announce discards any held pending image, partly received or complete, and the pending image becomes current once its transfer completes and its timestamp is reached. The newest-wins rule from #135 thus applies at the announce, as it did at message arrival before chunking, and a client needs at most two image buffers per channel, receiving the transfer directly into the pending one - A rejected announce (no active stream, or client not available) still starts its transfer, so its parts are rejected rather than treated as malformed - The clear message becomes an announce with total_size 0, which completes immediately with no parts - A two-byte cancel message (flags bit 1) discards the channel's pending image, partly received or complete, leaving the current image showing; it replaces resend-the-current-image as the cancel recipe in the scheduled-artwork rules, which under chunking would cost a full image retransfer - Malformed sequences (a message under 2 bytes or over the frame cap, an announce whose length is not 14 bytes, a cancel longer than 2 bytes, a part with no transfer in flight, data extending past total_size, a nonzero reserved flag bit, both flag bits set) close the connection, matching fragmentation Every image costs one extra small message, including images that fit in a single frame; the uniform format was judged worth ~31 bytes of ciphertext per image. There is no explicit last-part flag; total_size already determines completion, and a redundant end marker would add mismatch states to define. Every artwork message changes shape: the flags byte shifts the timestamp, the header and image data travel in separate messages, and the empty clear message becomes a bare announce with total_size 0. Out of scope: the fragmentation rules in messaging.md, which stay as they are, and the metadata/color cancel recipe, which remains resend-the-current-state since those are single small JSON messages. Closes #134
…only knows an arwork binary is complete based on counting bytes received vs expected.
…art or stream/end message
…tart ro stream/end Co-authored-by: Maxim Raznatovski <[email protected]>
maximmaxim345
approved these changes
Sep 1, 2026
maximmaxim345
left a comment
Member
There was a problem hiding this comment.
Looks good now (I have only one last nit, but that's more to give server implementers a hint so they avoid the pitfall this tries to solve).
Thanks @kahrendt!
Contributor
Author
|
@maximmaxim345 Thanks for the thorough feedback on this one, it improved the PR greatly! |
maximmaxim345
approved these changes
Sep 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An artwork image larger than one Noise frame (65518 bytes of payload) currently relies on transport fragmentation, which admits no interleaving: the sender must finish the fragmented message before any other frame, so the whole image is an indivisible burst that audio chunks cannot preempt. A player holding only
min_buffer_msof audio can underrun behind it; on a live stream that truncates audio. With BMP gone (#168) the worst case shrank, but photographic JPEG still exceeds one frame beyond roughly 500x500 (the spec's ownstream/startexample, 800x800 JPEG, is 3-5 frames), and PNG exceeds it at almost any artwork size.Changes:
total_size(uint32), followed by parts carrying only image data, all on the channel's message type; a flags byte after the type byte distinguishes the message kinds, following the shape Give fragmentation a single ID #172 gave fragmentationtotal_sizetotal_size0, which completes immediately with no partstotal_size, a nonzero reserved flag bit, both flag bits set) close the connection, matching fragmentationThe pending-image redefinition keeps the newest-wins rule from #135 applying at the announce, as it did at message arrival before chunking, and means a client needs at most two image buffers per channel, receiving the transfer directly into the pending one.
Every image costs one extra small message, including images that fit in a single frame; the uniform format was judged worth ~31 bytes of ciphertext per image to give clients flexibility to transfer directly into a proper sized buffer.
There is no explicit last-part flag;
total_sizealready determines completion, and a redundant end marker would add mismatch states to define.Every artwork message changes shape: the flags byte shifts the timestamp, the header and image data travel in separate messages, and the empty clear message becomes a bare announce with
total_size0.Closes #134
Breaking changes:
The wire format for artwork is completely changed. All servers and clients will have to re-implement to handle: