When a streaming client disconnects abnormally (no audio-stop / end-marker — killed app, dropped connection, host restart mid-stream), the leftover audio:stream:* key becomes a permanent STT reconnect generator. Three interacting behaviors, verified on a self-hosted deployment that accumulated 17 such streams after a night of forced reconnects:
- Completion knowledge is a 5-minute-TTL key; stream keys are deliberately immortal.
streaming_consumer gates re-discovery on transcription:complete:{session_id}, which expires after 5 min. Any surviving audio:stream:* key (intentionally preserved by the reaper until Mongo durability is proven) is then re-discovered as active un-transcribed work every tick, opening a fresh STT provider connection roughly every 5 minutes per stream, forever. Against a serial-handshake provider this is a wedge (CLOSE_WAIT pile-up); against a cloud provider it is unbounded spend.
- Zombie completion is not durable. The 300s idle-without-end-marker path logs 'treating as zombie and ending processing' and sets the completion signal, but neither XACKs its pending entries nor writes a terminal session status — so the reclaim sweep can never prove the session closed.
- Progress tracking resurrects deleted session hashes. The consumer's progress
hset (e.g. transcription_seconds_sent) recreates audio:session:{id} with no status field. session_append_closed correctly treats a statusless hash as ambiguous, so POST /api/streaming/reclaim retains the stream as session_may_still_append indefinitely — the supported cleanup surface is structurally unable to converge to zero.
Repro: start a streaming session, kill the client without an end-marker, wait past the 5-min completion TTL. The stream is re-discovered and re-connected every ~5 min; /api/streaming/reclaim retains it forever once any progress write has recreated the session hash.
Suggested direction (maintainer's choice): on zombie completion, durably mark the session terminal (so session_append_closed answers truthfully); and/or gate re-discovery on something reclaim deletes rather than a TTL clock. Pending-entry ACKs arguably should stay fail-closed as-is.
When a streaming client disconnects abnormally (no
audio-stop/ end-marker — killed app, dropped connection, host restart mid-stream), the leftoveraudio:stream:*key becomes a permanent STT reconnect generator. Three interacting behaviors, verified on a self-hosted deployment that accumulated 17 such streams after a night of forced reconnects:streaming_consumergates re-discovery ontranscription:complete:{session_id}, which expires after 5 min. Any survivingaudio:stream:*key (intentionally preserved by the reaper until Mongo durability is proven) is then re-discovered as active un-transcribed work every tick, opening a fresh STT provider connection roughly every 5 minutes per stream, forever. Against a serial-handshake provider this is a wedge (CLOSE_WAIT pile-up); against a cloud provider it is unbounded spend.hset(e.g.transcription_seconds_sent) recreatesaudio:session:{id}with nostatusfield.session_append_closedcorrectly treats a statusless hash as ambiguous, soPOST /api/streaming/reclaimretains the stream assession_may_still_appendindefinitely — the supported cleanup surface is structurally unable to converge to zero.Repro: start a streaming session, kill the client without an end-marker, wait past the 5-min completion TTL. The stream is re-discovered and re-connected every ~5 min;
/api/streaming/reclaimretains it forever once any progress write has recreated the session hash.Suggested direction (maintainer's choice): on zombie completion, durably mark the session terminal (so
session_append_closedanswers truthfully); and/or gate re-discovery on something reclaim deletes rather than a TTL clock. Pending-entry ACKs arguably should stay fail-closed as-is.