Repository navigation
feat(eap-items): Add eap_items_2_dlq_replay storage - #8570
Closed
sentry-junior[bot] wants to merge 1 commit into
Closed
sentry-junior[bot] wants to merge 1 commit into
sentry-junior[bot] wants to merge 1 commit into
Conversation
Co-Authored-By: Ryan Broughan <[email protected]>
Contributor
|
Closing in favor of #8575 |
tryangul
added a commit
that referenced
this pull request
Oct 20, 2026
) Repoints the existing `eap_items_dlq_replay` storage from `eap_items_1_local` / `eap_items_1_dist` to `eap_items_2_local` / `eap_items_2_dist`. The main `eap-items-consumer` in us, de and s4s2 now writes `eap_items_2`, so DLQ replays should land there too. The schema and `UUIDColumnProcessor` columns are synced with `eap_items_2`, which adds `conversation_id`, `session_id` and `ai_conversation_id`. They had drifted from the primary storage. The storage name, the input topic (`snuba-dead-letter-items`) and the lack of a `dlq_topic` don't change. Because the name doesn't change, `eap_items_dlq_grace_period_min` still has no entry for it, so replayed rows aren't DLQ'd again. This applies everywhere the replay consumer runs with the default storage. Any replay consumer that's running when this deploys starts writing `eap_items_2` on its next restart. No ops change is needed for regions that use the default storage. This replaces #8570, which added a separate `eap_items_2_dlq_replay` storage instead. <!-- junior-request-attribution:start --> via **Ryan Broughan**. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:slack%3AC0CAC22GMKM%3A1792468188.049119 --> -- [View Junior Session](https://junior-prod.sentry.dev/conversations/slack%3AC0CAC22GMKM%3A1792468188.049119) [[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC0CAC22GMKM%3A1792468188.049119/?project=4510944073809921) <!-- junior-session-footer:end --> Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Ryan Broughan <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an
eap_items_2_dlq_replaywritable storage: theeap_items_2counterpart toeap_items_dlq_replay(#7944). It readssnuba-dead-letter-itemsand writes toeap_items_2_local/eap_items_2_dist.eap_items_dlq_replaystill writes toeap_items_1_*, so a region whose main consumer has moved toeap_items_2would replay its DLQ into the old table.Running the replay consumer with
--storage eap_items_2and the DLQ as the raw topic doesn't work in regions whereeap_items_dlq_grace_period_minhas aneap_items_2entry (us currently haseap_items_2: 5). The grace period is looked up by storage name, so replayed prior-partition rows would get DLQ'd back into the topic being read.eap_items_2also sets adlq_topic, so other processing failures would loop the same way.This storage has its own name, which keeps it out of the grace-period map, and like
eap_items_dlq_replayit has nodlq_topic. Schema and query processors are copied fromeap_items_2(so it includesconversation_id,session_idandai_conversation_id);deletion_settingsand subscription config are left out.The ops change pointing the us
eap-items-dlq-replay-consumerat this storage depends on this being deployed first.via Ryan Broughan.
--
View Junior Session [Sentry]