[pcbeingused333]

💡 Motivation and Context

TrackingStream in PostHogOpenAIHandler decodes the streamed SSE response chunk by chunk with Encoding.UTF8.GetString(buffer, offset, count). When a multibyte UTF-8 character (accented letters, emoji, CJK) straddles two reads, each half is decoded on its own and becomes U+FFFD (�), so $ai_output_choices contains corrupted text even though the stream passed to the caller is intact.

The tracker now keeps a Decoder (UTF8Encoding.GetDecoder()) for the lifetime of the stream, so a partial character is buffered until its remaining bytes arrive.

💚 How did you test it?

Added SendAsyncDecodesMultibyteCharactersSplitAcrossStreamReads in PostHogOpenAIHandlerTests: the SSE body (Café , 😀 ñ) is served one byte per read, so every multibyte character is split, and the captured $ai_output_choices content must be Café 😀 ñ. It uses OneByteAtATimeStream, a small MemoryStream subclass in the test file. Edge cases covered by the one-byte reads: a 2-byte accented letter, a 4-byte emoji, and a character split with the rest of a line still buffered. A changeset for PostHog.AI (patch) is included.

The test was run locally in a .NET SDK container (dotnet test tests/PostHog.AI.Tests/PostHog.AI.Tests.csproj --filter FullyQualifiedName~SendAsyncDecodesMultibyteCharactersSplitAcrossStreamReads): it fails on main, where the captured content no longer matches, and passes with the change (Failed: 0, Passed: 1). The rest of the suite was left to CI. The handler was not exercised against the live OpenAI API, since that needs credentials; the test drives the real handler pipeline end to end through HttpClient with a byte-at-a-time response stream.

📝 Checklist

• All commits are signed (see Commit signing)
• I reviewed the submitted code.
• I added tests to verify the changes.
• I updated the docs if needed.
• No breaking change or entry added to the changelog.

If releasing new changes

• Added a changeset file (.changeset/utf8-openai-stream-decoder.md)

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Prepared with an AI coding assistant (Claude Code); the work was directed by the PR author, who owns the change. PostHog/sdk-specs was checked as AGENTS.md asks: the capture-ai spec says nothing about SSE decoding or character encoding, so there is no cross-SDK contract this change diverges from. The decoder choice is a plain Decoder rather than buffering raw bytes until a line boundary: it keeps the existing char-based line/SSE buffering unchanged and is the standard way to decode a UTF-8 stream incrementally.

[marandaneto]

The code looks good; no qualifying findings. Verdict: correct.