Skip to content

ref(producers): Use arroyo KafkaProducer for lw deletions and querylog - #8584

Open
lvthanh03 wants to merge 1 commit into
masterfrom
tony/search-issues-arroyo
Open

lvthanh03 wants to merge 1 commit into
masterfrom
tony/search-issues-arroyo

Conversation

@lvthanh03

Copy link
Copy Markdown
Member

Switches the lightweight deletions producer and the querylog producer from confluent_kafka.Producer to arroyo's KafkaProducer.

  • KafkaProducer polls in its own thread, so the lw deletions flush_producers thread and the poll(0) calls are removed
  • delivery callbacks are attached to the returned future
  • metrics and warning logs are unchanged
  • state.flush_producer() (only called at shutdown by the subscriptions executors) now also closes the producer

Refs STREAM-1929

@lvthanh03
lvthanh03 requested a review from a team as a code owner October 20, 2026 23:33
@linear-code

linear-code Bot commented Oct 20, 2026

Copy link
Copy Markdown

STREAM-1929

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant