Optimize how SSE data is written - #826
Open
pR0Ps wants to merge 1 commit into
Open
Conversation
The `SseConnection` now writes queued events directly to the response's output stream from the connection-handling thread itself. Previously each SSE connection has its own virtual thread that would create and push the events into a pipe for the actual connection-handling thread to read from. Per SSE connection, this new method saves the creation of a virtual thread, a `PipedOutputStream`/`PipedInputStream` pair and their internal buffer, and the overhead of having to push all the SSE data across a thread boundary. Additionally, because the `SseConnection` now handles writing the data to the output stream itself and can handle flushing it as needed, the `HttpResponseOutputStream` no longer needs to flush the output stream after every buffer read just in case the data it was sending was part of an SSE stream. To support this change, the `HttpResponseStreamWriter` interface was added to allow `HttpResponse` bodies to be streamed incrementally and dynamically chunked using the new `ChunkedOutputStream` rather than only being able to be read fixed-size chunks from an `InputStream`. Note that because the pipe buffer is now gone, event buffering in the `SseConnection` occurs entirely in the queue. To compensate for this, the `QUEUE_CAPACITY` was increased from 16 to 64.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a followup of #819 (comment)
The
SseConnectionnow writes queued events directly to the response's output stream from the connection-handling thread itself. Previously each SSE connection has its own virtual thread that would create and push the events into a pipe for the actual connection-handling thread to read from.Per SSE connection, this new method saves the creation of a virtual thread, a
PipedOutputStream/PipedInputStreampair and their internal buffer, and the overhead of having to push all the SSE data across a thread boundary.Additionally, because the
SseConnectionnow handles writing the data to the output stream itself and can handle flushing it as needed, theHttpResponseOutputStreamno longer needs to flush the output stream after every buffer read just in case the data it was sending was part of an SSE stream.To support this change, the
HttpResponseStreamWriterinterface was added to allowHttpResponsebodies to be streamed incrementally and dynamically chunked using the newChunkedOutputStreamrather than only being able to be read fixed-size chunks from anInputStream.Note that because the pipe buffer is now gone, event buffering in the
SseConnectionoccurs entirely in the queue. To compensate for this, theQUEUE_CAPACITYwas increased from 16 to 64.