core: async-nats behind one module, and dead letters announce themselves

.cargo/audit.toml

@@ -4,38 +4,6 @@
 
 [advisories]
 ignore = [
-    # rustls 0.22.4 and rustls-webpki 0.102.8 come with the sync `nats` 0.25
-    # client (core, connectors/amqp, connectors/mq); connectors/amqp also uses
-    # rustls 0.22 directly for amqps://. nats 0.25 is the last release of the
-    # sync client, so this line gets no patched version; moving to async-nats
-    # removes all seven. Ignores go by advisory ID, so these would also hide
-    # 0346/0349 on rustls 0.23: keep core on >= 0.23.46.
-    # None is reachable the way the crates are used here:
-
-    # Debug output of client session values leaks secrets only through a
-    # custom ClientSessionStore; neither nats nor vejas-amqp installs one.
-    "RUSTSEC-2026-0346",
-    # An unauthenticated ServerHello can drop later connections from hybrid
-    # post-quantum to classical key exchange. rustls 0.22 on ring never offers
-    # a post-quantum group, so there is nothing to downgrade.
-    "RUSTSEC-2026-0349",
-    # CRL distribution-point matching and CRL parsing: no CRL is ever loaded.
-    "RUSTSEC-2026-0049",
-    "RUSTSEC-2026-0104",
-    # URI and wildcard name constraints: reachable only after signature
-    # verification, with a certificate misissued by a CA the client already
-    # trusts, and only on a TLS connection (a nats:// URL to a server that does
-    # not require TLS stays plaintext).
-    "RUSTSEC-2026-0098",
-    "RUSTSEC-2026-0099",
-    # Certificate paths through the distrusted emSign intermediates: rustls-webpki
-    # 0.102 has no distrust list. Reachable only on a tls:// NATS_URL, from a
-    # network position that can impersonate the NATS server, with a chain the
-    # machine's OS trust store accepts (nats 0.25 loads the native roots). A NATS
-    # server on a plain or private-CA connection is outside it. This ID also
-    # covers rustls-webpki 0.103: keep core on >= 0.103.16, webpki-roots >= 1.0.10.
-    "RUSTSEC-2026-0382",
-
     # idna 0.1.5 accepts Punycode labels that decode to plain ASCII. It comes
     # through url 1.7 <- amq-protocol 1.4 <- amiquip 0.4.2 (the last amiquip
     # release), in connectors/amqp only, where it parses the broker URL the

.github/workflows/ci.yml

@@ -96,6 +96,8 @@ jobs:
         run: e2e/emit-subject.sh
       - name: http-in survives a burst and a thread limit
         run: e2e/http-in-bound.sh
+      - name: Dead letters announce themselves, without feeding themselves
+        run: e2e/dlq-events.sh
       - name: Docs book builds (broken includes fail here, not on deploy)
         run: |
           curl -sL https://github.com/rust-lang/mdBook/releases/download/v0.5.4/mdbook-v0.5.4-x86_64-unknown-linux-gnu.tar.gz | tar xz

CHANGELOG.md

@@ -7,12 +7,53 @@ is `0`, minor versions may carry breaking changes — they are called out here.
 
 ## [Unreleased]
 
+### Added
+- **Dead letters announce themselves** on `vx.dlq.events`, an ordinary subject,
+  so that a flow or a sink can wake a person or an agent (ADR-0015 Amendment A).
+  The event carries metadata only, never the payload or the error text, and
+  `VEJAS_DLQ_EVENT_SECS` (default 60) groups a burst into one event per unit,
+  with `count` for the letters it stands for. `docs/examples/dlq_webhook.vjs.example`
+  posts it to a URL.
+
+### Security
+- **Seven advisories leave `.cargo/audit.toml`**: RUSTSEC-2026-0049, -0098, -0099,
+  -0104, -0346, -0349 and -0382, all held by the sync `nats` client's rustls 0.22
+  and rustls-webpki 0.102. Every lockfile is now on rustls 0.23.46 and
+  rustls-webpki 0.103.16, `connectors/amqp` included.
+
 ### Fixed
 - **A burst of connections during boot could end the version watch for good.** Its
   first connection panicked when the OS had no thread to give: nothing showed it,
   and a promoted version no longer reached that instance until a restart. It
   needs a thread limit and a burst at that moment. The watch and the control loop
   now carry a name and start again after a panic.
+- **A consumer deleted under a running flow no longer leaves a zombie.** The flow
+  stayed `running` for good, consuming nothing, and two empty dead letters from an
+  `_INBOX` subject landed in the queue. It now restarts and recreates the consumer.
+- **An emit the server cannot take no longer redelivers forever.** A publish
+  refused before it leaves, for a payload over the server's limit, left its source
+  message un-acked with no cap on deliveries. It is now a failed run, dead-lettered
+  after `MAX_DELIVERIES` like any other. Detect units keep the old behaviour here:
+  their emits are small and their engine state makes a redelivery a different case.
+
+### Changed
+- **The NATS client is now `async-nats`, behind one module (ADR-0035).** `nats` 0.25
+  is deprecated. `core/src/bus.rs` runs `async-nats` on one tokio worker and gives
+  the rest of the runtime a blocking API: no other module sees tokio or a NATS type,
+  and flows, connectors and supervisors stay threads. A process now holds one
+  connection, where it held two per flow. At 200 flows that is 210 threads against
+  1 011, 1 connection against 401, and 26 MB against 140. A flow is running 11 ms
+  after spawn, against 1.7 s. The flow hop holds its rate, ten flows run about 10 %
+  faster, and the 5 ms flusher floor is gone, with `publish_confirmed`,
+  `fetch_iter_flushed` and the second connection for acks. The costs: the binary
+  grows from 9.9 to 11.2 MB, RSS at one flow from 9.2 to 10.2 MB, and the MQTT
+  loopback falls from about 2 400 to 2 070 round-trips/s. Existing streams,
+  consumers, buckets and snapshots work as they are, and v0.3.6 reads back what this
+  version writes. **To upgrade:** a NATS user with a publish allow-list needs
+  `_INBOX.>` on it. Without it nothing is ever acked and everything redelivers; the
+  log names the permissions violation.
+- **`connectors/mq` and `connectors/amqp` compile the same module**, and
+  `connectors/amqp` moves to rustls 0.23.
 
 ## [0.4.0] — 2026-11-06
 

README.md

@@ -33,10 +33,10 @@ end-to-end test ([the script](e2e/bridge-film.mjs)) — the video cannot lie.
 ![The expert corrects the meaning; the pipes never move](docs/demo/bridge-teaser.gif)
 
 **Measured, not claimed** — reproducible on an 8-core dev machine
-([methodology and comparisons](bench/)): cold start **11–13 ms**, **6–8 MB**
-RSS under load (49 MB with fifty live flows), **6.2 MB** binary / **201 MB**
+([methodology and comparisons](bench/)): cold start **11–13 ms**, **10 MB**
+RSS under load (38 MB with fifty live flows), **11.2 MB** binary / **201 MB**
 image, end-to-end p50 **2 ms** uncongested / p50 14 ms at a paced 1 900/s
-sustained with **every event hop persisted** in JetStream, **2 285/s** through a
+sustained with **every event hop persisted** in JetStream, **2 070/s** through a
 real MQTT broker and back (QoS 1 both ways, PUBACK-gated). Same order of
 magnitude as engines that persist nothing, in ~25× less memory.
 

bench/README.md

@@ -31,12 +31,12 @@ All five ceilings fell (#4+#3 `5cffb7b`, #2 `b34d9f3`, #1 `75fad83`,
 | Metric | v0 | all five fixes in |
 |---|---|---|
 | Cold start | 13–15 ms | **11–13 ms** |
-| Runtime RSS under load | 6–8 MB | **6–8 MB** |
-| Binary size | 3.9 MB | 6.2 MB (rustls) |
+| Runtime RSS under load | 6–8 MB | **10 MB** |
+| Binary size | 3.9 MB | 11.2 MB (rustls, async-nats) |
 | e2e sustained, paced ~1 900/s (`bench/paced.sh 2000 15`) | — | **p50 14 ms, p99 36 ms**, 20 000/20 000 |
 | e2e saturated (32 conns) | 65/s delivered | ~4 900/s ingest, **~2 650/s delivered** (sink-bound, queue absorbs the rest) |
 | e2e latency, uncongested (`bench/paced.sh 20 15`) | p50 859 ms | **p50 2 ms, p99 3 ms** |
-| MQTT broker loopback, QoS 1 both ways (`bench/broker-mqtt.sh 5000`) | — | **2 285 rt/s**, 5 000/5 000 |
+| MQTT broker loopback, QoS 1 both ways (`bench/broker-mqtt.sh 5000`) | — | **2 070 rt/s**, 5 000/5 000 |
 | Isolated flow hop | 171/s | **8 110/s** |
 
 The old "sustained (12 conns) 1 701/s, p50 18 ms" row died with ceiling #5:
@@ -47,6 +47,31 @@ any concurrency, so steady-state latency is now measured **paced**
 
 Every hop persisted in JetStream throughout — the guarantee never moved.
 
+### Since ADR-0035
+
+The sync `nats` client left for `async-nats` behind `core/src/bus.rs`, so findings
+#5 and #6 below describe a client that is gone, along with `publish_confirmed`,
+`fetch_iter_flushed` and the second connection of #7. Against the v0.3.6 binary,
+alternating in one session on this box:
+
+| | v0.3.6 | now |
+|---|---:|---:|
+| Threads at 1 / 10 / 50 / 200 idle flows | 16 / 61 / 261 / 1 011 | 11 / 20 / 60 / 210 |
+| NATS connections, same | 3 / 21 / 101 / 401 | 1 |
+| RSS, same, MB | 8.9 / 16.7 / 46.9 / 139.8 | 9.6 / 10.5 / 14.2 / 26.4 |
+| Flow hop (`PUBS=16`, 64 000 events) | 13 100 – 13 700/s | 13 200 – 13 500/s |
+| Flow hop, `bench/pub.py` as publisher | 20 100 – 20 500/s | 19 400 – 21 400/s |
+| One detect unit | 30 800 – 35 400/s, 11 MB | 34 700 – 41 100/s, 12.7 MB |
+| MQTT loopback | 2 370 – 2 430 rt/s | 2 050 – 2 090 rt/s |
+| Spawn to a flow running | 1 714 ms | 10.9 ms |
+| Paced 2 000/s, p50 / p99 | 3–4 / 14–17 ms | 3 / 14–15 ms |
+| Binary | 9.9 MB | 11.2 MB |
+
+The paced row is why the p50 14 ms above stands as published: both binaries are far
+below it today, so this change did not move it. The detect tables further down
+predate this: add about 1.7 MB to their RSS. The method and the MQTT cost are in
+ADR-0035.
+
 The footprint numbers are the thesis, measured. The throughput and latency
 numbers are **known ceilings of the v0 I/O paths, not of the interpreter** —
 each one matches its cause exactly:
@@ -134,13 +159,13 @@ they publish). Scaling the number of flows (`bench/multi-flow.sh`):
 
 | Flows | Aggregate rate | Runtime RSS |
 |---|---|---|
-| 1 | 7–8 k/s | 6 MB |
-| 10 | **13 746/s** | 16.6 MB |
-| 50 | 8 410/s | 49.3 MB |
+| 1 | 7–8 k/s | 10 MB |
+| 10 | **13 700 – 14 700/s** | 14.5 MB |
+| 50 | 11 100 – 12 100/s | 38 MB |
 
 Throughput *rises* with flow count (consumers parallelize; the bus, not the
-interpreter, is the bound) and memory stays ~1 MB per running flow — fifty
-live, persisted flows in under 50 MB.
+interpreter, is the bound) and memory stays around half a megabyte per running
+flow under load — fifty live, persisted flows in under 40 MB.
 
 ## Detect units (`bench/detect.sh`)
 
@@ -252,7 +277,7 @@ one unit handles that at around 25 000 events/s in 18 MB.
 - **One instance saturates near 44 000 events/s** whatever the unit count.
   Beyond it, add instances: they share the durable consumers and the work
   (see *Clustering*).
-- **Artefacts:** the binary is 8.2 MiB, the image 37.6 MiB, and a unit that is
+- **Artefacts:** the binary is 10.7 MiB, the image 37.6 MiB, and a unit that is
   idle costs nothing measurable.
 
 Measured on the dev box in this file's header — 8 cores, WSL2 — two runs of