fix: drop the oldest AI events while too many wait to be written to disk by turnipdabeets · Pull Request #901 · PostHog/posthog-android

turnipdabeets · 2026-10-28T22:39:41Z

Stacked on #881.

💡 Motivation and Context

One single-threaded executor both writes AI events to disk and sends them. During a slow upload (up to captureAiTimeout, 60 s, per request), new captureAi events wait in the executor's task queue, in memory, with no limit. The 100-event / 50 MiB caps only applied once an event was written, so a chat app capturing multi-MiB generations on a slow network could exhaust the heap. PostHog Review flagged this on #881 and #882.

AI events now wait in a bounded list until the executor writes them. Once more than captureAiMaxQueueSize events (100 by default) or 50 MiB wait, the oldest waiting events are dropped and the new one is kept, the same as the disk queue already does (and as the posthog-ios 4.0 AI queue and the sdk-specs capture-ai client note describe). These are the disk queue's limits, so holding more in memory couldn't deliver more events; it only delayed the eviction. captureAi still returns the new event's uuid.

PostHogQueue.addMeasured(record, bytes): adds the record to the waiting list, drops the oldest waiting records while over the count or bytes, and queues an executor task that writes the oldest waiting record, only while there are fewer tasks than waiting records, so dropped events leave no tasks behind. Only the AI lane uses it; analytics, replay and logs keep add.
captureAi passes the size it already measures for the per-event limit, so there's no second serialization.
A drop is logged with the count only, never event content.
The changeset bullet, the migration guide and captureAiMaxQueueSize describe the drop.

How the other SDKs bound waiting AI events

SDK	Waiting AI events	Limit	When full
posthog-ios 4.0	None in memory: `captureAi` writes to disk on the caller's thread	Disk: 100 events, 50 MiB	Evicts the oldest on disk
posthog-python v1	In-memory queue	1000 events	Drops the new event
posthog-go v1	In-memory channel	1000 events	Drops the new event, `ErrQueueFull`
posthog-rs v1	In-memory queue	1000 events	Drops the new event, one warning
posthog-android (this PR)	Waiting list before the disk write	100 events, 50 MiB	Drops the oldest waiting events

This follows posthog-ios and the disk queue (drop the oldest) with mobile limits: a count-only cap of 100 would still allow ~800 MiB of 8 MiB events. A synchronous write like iOS would put disk I/O on Android's main thread. posthog-flutter needs no change: mobile calls the native SDKs, and the desktop Dart lane enforces its caps when it queues.

Splitting writing from sending into separate threads would remove the waiting entirely, but it changes the queue shared by analytics, replay and logs, so it's left out of this PR.

💚 How did you test it?

PostHogQueueByteLimitsTest: with the executor blocked, going over the waiting bytes and then the waiting count each drops the oldest waiting record; the queue ends with the newest three.
PostHogCaptureAiTest: with the AI executor blocked and captureAiMaxQueueSize = 2, the third captureAi still returns a uuid and the oldest waiting event is dropped.
With the eviction removed, both tests fail.
./gradlew apiCheck spotlessCheck :posthog:test :posthog-android:testReleaseUnitTest :posthog-server:test pass locally.

📝 Checklist

All commits are signed (see Commit signing)
I reviewed the submitted code.
I added tests to verify the changes.
I updated the docs if needed.
No breaking change or entry added to the changelog.

If releasing new changes

Ran pnpm changeset to generate a changeset file (updated the unreleased v4 major changeset's captureAi bullet)

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Claude Code compared how posthog-ios, posthog-python, posthog-go and posthog-rs bound AI events waiting to be sent, from their v1/4.0 branch sources, and implemented the cap as a follow-up to #881; after review against sdk-specs #147, the maintainer switched it from drop-the-newest to drop-the-oldest. Rejected: a count-only cap (too much memory on a device), a synchronous write (main-thread I/O), separate write and send threads (touches every queue; possible later).

🤖 Generated with Claude Code

One executor both writes and sends AI events, so during a slow upload new multi-MiB events waited in memory with no limit. captureAi now returns null while captureAiMaxQueueSize events or 50 MiB are waiting, the same limits the disk queue keeps. Co-Authored-By: Claude Opus 5.5 <[email protected]>

github-actions · 2026-10-28T22:52:02Z

posthog-android Compliance Report

Date: 2026-10-29T21:56:55.027090+00:00
Duration: 474805ms

✅ All Tests Passed!

232/232 tests passed

Capture_V1 Tests

✅ 105/105 tests passed

View Details

Test	Status	Duration
Endpoint And Method.Targets V1 Endpoint	✅	325ms
Endpoint And Method.Does Not Use Legacy Endpoints	✅	36ms
Required Headers.Has Authorization Bearer Header	✅	24ms
Required Headers.Has Content Type Json	✅	22ms
Required Headers.Has Posthog Sdk Info Format	✅	21ms
Required Headers.Has Posthog Attempt Header	✅	25ms
Required Headers.Has Posthog Request Id	✅	20ms
Required Headers.Has Posthog Request Timestamp	✅	23ms
Required Headers.Has User Agent	✅	17ms
Body Format.Body Has Created At And Batch	✅	20ms
Body Format.No Api Key In Body	✅	20ms
Body Format.No Sent At In Body	✅	16ms
Event Format.Event Has Required Root Fields	✅	18ms
Event Format.Event Uuid Is Valid	✅	18ms
Event Format.Event Timestamp Is Rfc3339	✅	20ms
Event Format.Non Utc Event Timestamp Is Converted To Utc	✅	20ms
Event Format.Distinct Id Is String	✅	16ms
Event Format.Distinct Id At Root Not Properties	✅	20ms
Event Format.Custom Properties Preserved	✅	18ms
Event Format.Set Properties Preserved	✅	18ms
Event Format.Set Once Properties Preserved	✅	19ms
Event Format.Groups Properties Preserved	✅	21ms
Event Format.Sdk Generates Uuid If Not Provided	✅	18ms
Event Format.Event Has Required Root Fields Batch	✅	28ms
Event Format.Event Uuid Is Valid Batch	✅	23ms
Event Format.Event Timestamp Is Rfc3339 Batch	✅	24ms
Event Format.Distinct Id Is String Batch	✅	21ms
Event Format.Distinct Id At Root Not Properties Batch	✅	21ms
Event Format.Custom Properties Preserved Batch	✅	27ms
Event Format.Set Properties Preserved Batch	✅	24ms
Event Format.Set Once Properties Preserved Batch	✅	24ms
Event Format.Groups Properties Preserved Batch	✅	22ms
Event Format.Sdk Generates Uuid If Not Provided Batch	✅	36ms
Batch Behavior.Multiple Events In Single Batch	✅	27ms
Batch Behavior.Batch Envelope Smoke	✅	24ms
Batch Behavior.Flush With No Events Sends Nothing	✅	13ms
Batch Behavior.Flush At Triggers Batch	✅	1022ms
Batch Behavior.Created At Reflects Batch Creation Time	✅	19ms
Deduplication.Generates Unique Uuids	✅	39ms
Deduplication.Different Events Same Content Different Uuids	✅	22ms
Deduplication.Preserves Uuid On Retry	✅	7015ms
Deduplication.Preserves Timestamp On Retry	✅	7019ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry	✅	7021ms
Deduplication.No Duplicate Events In Batch	✅	32ms
Header Behavior On Retry.Attempt Header Starts At One	✅	15ms
Header Behavior On Retry.Attempt Header Increments On Retry	✅	12023ms
Header Behavior On Retry.Request Id Preserved On Retry	✅	7017ms
Header Behavior On Retry.Different Requests Have Different Request Ids	✅	2030ms
Header Behavior On Retry.Request Timestamp Changes On Retry	✅	7017ms
Response Format Validation.Success Response Has Uuid Keyed Results	✅	18ms
Response Format Validation.Success Response Has Ok For Each Event	✅	20ms
Response Format Validation.Success No Retry After When All Ok	✅	16ms
Response Format Validation.Success Retry After Present When Retry Events	✅	2009ms
Response Format Validation.Success No Retry After When Drop Only	✅	20ms
Response Format Validation.Response Echoes Request Id	✅	14ms
Retry Behavior.Retries On 408	✅	7014ms
Retry Behavior.Retries On 500	✅	7016ms
Retry Behavior.Retries On 503	✅	7021ms
Retry Behavior.Retries On 504	✅	7018ms
Retry Behavior.Retries On 404	✅	7016ms
Retry Behavior.Retries On 429	✅	6017ms
Retry Behavior.Retries On 502	✅	6017ms
Retry Behavior.Retryable Errors Have Retry After	✅	4015ms
Retry Behavior.Respects Retry After On Retryable Error	✅	10023ms
Retry Behavior.Does Not Retry On 400	✅	4020ms
Retry Behavior.Does Not Retry On 401	✅	4018ms
Retry Behavior.Does Not Retry On 402	✅	4019ms
Retry Behavior.Does Not Retry On 413	✅	4019ms
Retry Behavior.Does Not Retry On 415	✅	4021ms
Retry Behavior.Non Retryable Errors Have No Retry After	✅	4020ms
Retry Behavior.Implements Backoff	✅	14029ms
Partial Batch Handling.Handles 200 Full Success	✅	2018ms
Partial Batch Handling.Handles 200 With All Ok	✅	3027ms
Partial Batch Handling.Does Not Retry Dropped Events	✅	3023ms
Partial Batch Handling.Does Not Retry Warning Events	✅	3022ms
Partial Batch Handling.Prunes Ok Events On Partial Retry	✅	7016ms
Partial Batch Handling.Prunes Dropped Events On Partial Retry	✅	7016ms
Partial Batch Handling.Retries Only Retry Events From Partial	✅	7014ms
Partial Batch Handling.Partial Retry Preserves Uuids	✅	7017ms
Partial Batch Handling.Partial Retry Attempt Header Increments	✅	7015ms
Partial Batch Handling.Partial Retry Request Id Preserved	✅	7017ms
Partial Batch Handling.Respects Retry After On Partial	✅	7024ms
Partial Batch Handling.Unknown Result Treated As Terminal	✅	3022ms
Partial Batch Handling.Mixed Ok Drop Warning No Retry	✅	3027ms
Compression.Sends Gzip Content Encoding	✅	16ms
Compression.No Content Encoding When Disabled	✅	14ms
Compression.Compressed Body Is Decompressible	✅	12ms
Error Handling.Does Not Retry On Unknown 4Xx	✅	4014ms
Event Options.Cookieless Mode Override	✅	16ms
Event Options.Disable Skew Correction Override	✅	12ms
Event Options.Process Person Profile Override	✅	13ms
Event Options.Product Tour Id Override	✅	12ms
Event Options.Unset Options Omitted	✅	14ms
Event Options.Options Override In Batch	✅	18ms
Event Options.Unknown Option Passes Through	✅	13ms
Event Options.Option Value Is Not Converted	✅	19ms
Event Options.Lenient Bool Option Values Are Not Converted	✅	20ms
Event Options.Option Wins Over Legacy Property	✅	18ms
Event Options.Legacy Property Fills Unset Option	✅	17ms
Event Options.Null Option Falls Back To Legacy Property	✅	14ms
Property Layers.Caller Property Beats Registered Property	✅	13ms
Property Layers.Before Send Runs Last	✅	14ms
Fail Open.Empty Api Key Sends Nothing	✅	3011ms
Fail Open.Personal Api Key Sends Nothing	✅	3013ms
Geoip And Historical Migration.Historical Migration Absent By Default	✅	14ms

Capture_Ai_V1 Tests

✅ 110/110 tests passed

View Details

Test	Status	Duration
Endpoint And Method.Targets V1 Endpoint	✅	14ms
Endpoint And Method.Does Not Use Legacy Endpoints	✅	13ms
Endpoint And Method.Capture Does Not Reroute Ai Named Events	✅	14ms
Endpoint And Method.Capture Ai Does Not Reroute Non Ai Events	✅	15ms
Lane Isolation.Failing Ai Lane Does Not Delay Analytics	✅	7023ms
Required Headers.Has Authorization Bearer Header	✅	16ms
Required Headers.Has Content Type Json	✅	16ms
Required Headers.Has Posthog Sdk Info Format	✅	15ms
Required Headers.Has Posthog Attempt Header	✅	15ms
Required Headers.Has Posthog Request Id	✅	11ms
Required Headers.Has Posthog Request Timestamp	✅	12ms
Required Headers.Has User Agent	✅	12ms
Body Format.Body Has Created At And Batch	✅	13ms
Body Format.No Api Key In Body	✅	12ms
Body Format.No Sent At In Body	✅	11ms
Event Format.Event Has Required Root Fields	✅	11ms
Event Format.Event Uuid Is Valid	✅	11ms
Event Format.Event Timestamp Is Rfc3339	✅	11ms
Event Format.Non Utc Event Timestamp Is Converted To Utc	✅	12ms
Event Format.Distinct Id Is String	✅	13ms
Event Format.Distinct Id At Root Not Properties	✅	12ms
Event Format.Custom Properties Preserved	✅	16ms
Event Format.Set Properties Preserved	✅	19ms
Event Format.Set Once Properties Preserved	✅	12ms
Event Format.Groups Properties Preserved	✅	15ms
Event Format.Sdk Generates Uuid If Not Provided	✅	14ms
Event Format.Event Has Required Root Fields Batch	✅	16ms
Event Format.Event Uuid Is Valid Batch	✅	16ms
Event Format.Event Timestamp Is Rfc3339 Batch	✅	18ms
Event Format.Distinct Id Is String Batch	✅	16ms
Event Format.Distinct Id At Root Not Properties Batch	✅	16ms
Event Format.Custom Properties Preserved Batch	✅	17ms
Event Format.Set Properties Preserved Batch	✅	17ms
Event Format.Set Once Properties Preserved Batch	✅	19ms
Event Format.Groups Properties Preserved Batch	✅	23ms
Event Format.Sdk Generates Uuid If Not Provided Batch	✅	16ms
Batch Behavior.Multiple Events In Single Batch	✅	23ms
Batch Behavior.Batch Envelope Smoke	✅	19ms
Batch Behavior.Flush At Triggers Batch	✅	1014ms
Batch Behavior.Created At Reflects Batch Creation Time	✅	16ms
Deduplication.Generates Unique Uuids	✅	28ms
Deduplication.Different Events Same Content Different Uuids	✅	17ms
Deduplication.Preserves Uuid On Retry	✅	7012ms
Deduplication.Preserves Timestamp On Retry	✅	7014ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry	✅	7015ms
Deduplication.No Duplicate Events In Batch	✅	26ms
Header Behavior On Retry.Attempt Header Starts At One	✅	12ms
Header Behavior On Retry.Attempt Header Increments On Retry	✅	12021ms
Header Behavior On Retry.Request Id Preserved On Retry	✅	7014ms
Header Behavior On Retry.Different Requests Have Different Request Ids	✅	2023ms
Header Behavior On Retry.Request Timestamp Changes On Retry	✅	7016ms
Response Format Validation.Success Response Has Uuid Keyed Results	✅	20ms
Response Format Validation.Success Response Has Ok For Each Event	✅	18ms
Response Format Validation.Success No Retry After When All Ok	✅	14ms
Response Format Validation.Success Retry After Present When Retry Events	✅	2009ms
Response Format Validation.Success No Retry After When Drop Only	✅	16ms
Response Format Validation.Response Echoes Request Id	✅	12ms
Retry Behavior.Retries On 408	✅	7013ms
Retry Behavior.Retries On 500	✅	7016ms
Retry Behavior.Retries On 503	✅	7021ms
Retry Behavior.Retries On 504	✅	7016ms
Retry Behavior.Retries On 404	✅	6012ms
Retry Behavior.Retries On 429	✅	7011ms
Retry Behavior.Retries On 502	✅	7015ms
Retry Behavior.Retryable Errors Have Retry After	✅	4013ms
Retry Behavior.Respects Retry After On Retryable Error	✅	10023ms
Retry Behavior.Does Not Retry On 400	✅	4017ms
Retry Behavior.Does Not Retry On 401	✅	4016ms
Retry Behavior.Does Not Retry On 402	✅	4017ms
Retry Behavior.Does Not Retry On 413	✅	4018ms
Retry Behavior.Does Not Retry On 415	✅	4018ms
Retry Behavior.Non Retryable Errors Have No Retry After	✅	4016ms
Retry Behavior.Implements Backoff	✅	14026ms
Partial Batch Handling.Handles 200 Full Success	✅	2018ms
Partial Batch Handling.Handles 200 With All Ok	✅	3022ms
Partial Batch Handling.Does Not Retry Dropped Events	✅	3023ms
Partial Batch Handling.Does Not Retry Warning Events	✅	3022ms
Partial Batch Handling.Prunes Ok Events On Partial Retry	✅	7015ms
Partial Batch Handling.Prunes Dropped Events On Partial Retry	✅	7014ms
Partial Batch Handling.Retries Only Retry Events From Partial	✅	7016ms
Partial Batch Handling.Partial Retry Preserves Uuids	✅	7015ms
Partial Batch Handling.Partial Retry Attempt Header Increments	✅	7015ms
Partial Batch Handling.Partial Retry Request Id Preserved	✅	7014ms
Partial Batch Handling.Respects Retry After On Partial	✅	7016ms
Partial Batch Handling.Unknown Result Treated As Terminal	✅	3021ms
Partial Batch Handling.Mixed Ok Drop Warning No Retry	✅	3024ms
Compression.Sends Gzip Content Encoding	✅	16ms
Compression.No Content Encoding When Disabled	✅	18ms
Compression.Compressed Body Is Decompressible	✅	13ms
Error Handling.Does Not Retry On Unknown 4Xx	✅	4014ms
Event Options.Cookieless Mode Override	✅	13ms
Event Options.Disable Skew Correction Override	✅	11ms
Event Options.Process Person Profile Override	✅	11ms
Event Options.Product Tour Id Override	✅	11ms
Event Options.Unset Options Omitted	✅	12ms
Event Options.Options Override In Batch	✅	17ms
Event Options.Unknown Option Passes Through	✅	11ms
Event Options.Option Value Is Not Converted	✅	12ms
Event Options.Lenient Bool Option Values Are Not Converted	✅	13ms
Event Options.Option Wins Over Legacy Property	✅	10ms
Event Options.Legacy Property Fills Unset Option	✅	12ms
Event Options.Null Option Falls Back To Legacy Property	✅	12ms
Event Size.Event Over Ceiling Is Not Sent	✅	3314ms
Event Size.Over Ceiling Drop Is Reported Or Logged Without Content	✅	2047ms
Event Size.Batches Close Before Byte Target	✅	6081ms
Geoip And Historical Migration.Historical Migration Absent By Default	✅	15ms
Property Layers.Caller Property Beats Registered Property	✅	12ms
Property Layers.Before Send Runs Last	✅	12ms
Fail Open.Empty Api Key Sends Nothing	✅	1008ms
Fail Open.Personal Api Key Sends Nothing	✅	1010ms

Feature_Flags Tests

✅ 17/17 tests passed

View Details

Test	Status	Duration
Request Payload.Request With Person Properties Device Id	✅	30ms
Request Payload.Flags Request Uses V2 Query Param	✅	18ms
Request Payload.Flags Request Hits Flags Path Not Decide	✅	16ms
Request Payload.Flags Request Omits Authorization Header	✅	18ms
Request Payload.Token In Flags Body Matches Init	✅	17ms
Request Payload.Groups Round Trip	✅	19ms
Request Payload.Groups Default To Empty Object	✅	19ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False	✅	15ms
Request Payload.Disable Geoip Omitted Defaults To False	✅	13ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key	✅	13ms
Request Lifecycle.No Flags Request On Init Alone	✅	7ms
Request Lifecycle.No Flags Request On Normal Capture	✅	11ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests	✅	18ms
Request Lifecycle.Mock Response Value Is Returned To Caller	✅	17ms
Retry Behavior.Retries Flags On 502	✅	317ms
Retry Behavior.Retries Flags On 504	✅	318ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event	✅	14ms

While a slow upload holds the AI executor, captureAi now keeps the new event and drops the oldest events still waiting to be written once captureAiMaxQueueSize events or 50 MiB wait, the same as the disk queue already does. captureAi returns null again only for disabled, opted-out, beforeSend-dropped or oversized events. Co-Authored-By: Claude Opus 5.5 <[email protected]>

veria-ai · 2026-10-28T23:52:02Z

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

Dropping a waiting AI event no longer leaves its executor task behind for nothing: a dropped event's task writes a newer one instead, so the executor never holds more tasks than events are waiting. Co-Authored-By: Claude Opus 5.5 <[email protected]>

posthog

PostHog Review · Standard · reviewed cf53ab6: 1 Should fix.

See this review and its findings in PostHog Review. Comment @posthog review --deep for a Deep review. A Deep review takes longer and looks deeper than a Standard review.

…ai-pending-cap Moves this branch's queue changes into enqueueEvent(), where #904 put capture's queueing step.

…ai-pending-cap

turnipdabeets self-assigned this Oct 28, 2026

turnipdabeets requested a review from a team as a code owner October 28, 2026 22:39

dustinbyrne approved these changes Oct 28, 2026

View reviewed changes

turnipdabeets changed the title ~~fix: drop an AI event while too many wait to be written to disk~~ fix: drop the oldest AI events while too many wait to be written to disk Oct 28, 2026

veria-ai Bot reviewed Oct 28, 2026

View reviewed changes

Comment thread posthog/src/main/java/com/posthog/internal/PostHogQueue.kt Outdated

posthog Bot reviewed Oct 29, 2026

View reviewed changes

Comment thread posthog/src/main/java/com/posthog/internal/PostHogQueue.kt Outdated

turnipdabeets added 3 commits October 29, 2026 10:31

fix: queue AI write tasks in the order their events were added

e93f6ff

Merge remote-tracking branch 'origin/feat/v4-capture-v1' into fix/v4-…

3216a40

…ai-pending-cap Moves this branch's queue changes into enqueueEvent(), where #904 put capture's queueing step.

Merge remote-tracking branch 'origin/feat/v4-capture-v1' into fix/v4-…

a95948a

…ai-pending-cap

Navigation Menu

Repository navigation

fix: drop the oldest AI events while too many wait to be written to disk - #901

fix: drop the oldest AI events while too many wait to be written to disk#901
turnipdabeets wants to merge 6 commits into
feat/v4-capture-v1from
fix/v4-ai-pending-cap

turnipdabeets commented Oct 28, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 28, 2026 •

edited

Loading

Uh oh!

Uh oh!

veria-ai Bot commented Oct 28, 2026 •

edited

Loading

Uh oh!

posthog Bot left a comment

Uh oh!

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

turnipdabeets commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💡 Motivation and Context

How the other SDKs bound waiting AI events

💚 How did you test it?

📝 Checklist

If releasing new changes

🤖 Agent context

Uh oh!

github-actions Bot commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

posthog-android Compliance Report

✅ All Tests Passed!

Capture_V1 Tests

Capture_Ai_V1 Tests

Feature_Flags Tests

Uh oh!

Uh oh!

veria-ai Bot commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR overview

Security review

Uh oh!

posthog Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

turnipdabeets commented Oct 28, 2026 •

edited

Loading

github-actions Bot commented Oct 28, 2026 •

edited

Loading

veria-ai Bot commented Oct 28, 2026 •

edited

Loading