From ff6d22e22eec21ed69a9f4e6cbf60103370d3197 Mon Sep 17 00:00:00 2001 From: Cyril Poder Date: Fri, 6 Nov 2026 19:47:41 +0100 Subject: [PATCH 1/9] docs(adr-0035): the NATS client moves to async-nats, behind one synchronous module nats 0.25 is deprecated and pins rustls 0.22, and its threads, connections and flusher floor had shaped the hot path. The decision: one module hosts async-nats on one tokio worker and everything else stays synchronous, with no async rewrite. The ADR records what the numbers decided (one connection, one worker), what the module has to rebuild because the new client does not give it (a real barrier, a reconnect that fails it, URL credentials), and what stays the same on the wire so that an existing JetStream store keeps working. Status: Proposed. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_018wXgxT2UoFSZfqhAE1Tb5V --- docs/adr/0035-async-nats-bus-island.md | 219 +++++++++++++++++++++++++ 1 file changed, 219 insertions(+) create mode 100644 docs/adr/0035-async-nats-bus-island.md diff --git a/docs/adr/0035-async-nats-bus-island.md b/docs/adr/0035-async-nats-bus-island.md new file mode 100644 index 0000000..7f1aff8 --- /dev/null +++ b/docs/adr/0035-async-nats-bus-island.md @@ -0,0 +1,219 @@ +# 0035 — The NATS client moves to async-nats, behind one synchronous module + +- Status: Proposed +- Date: 2026-11-06 + +## Context + +The runtime is synchronous and thread-based, and it stays that way: a flow, a +connector and a supervisor are OS threads, and no async runtime was in the tree. +That was also true of the NATS client, `nats` 0.25, which is the problem. The +crate is deprecated. `nats` 0.26 does not help: it keeps rustls 0.22.4 and the +same advisories, and seven entries of `.cargo/audit.toml` existed only for it. + +It also cost more than its size. The client runs two threads per connection, and +the runtime opened a connection per use, 25 sites in all. Measured on v0.3.6: 11 +threads idle, then 5 threads and 2 connections per flow. Two hundred flows were +1 011 threads, 401 connections and 140 MB. + +And a hard-coded 5 ms floor in its flusher thread shaped the hot path. Findings +#5 and #6 of `bench/README.md` come from it, and three workarounds followed: +`publish_confirmed`, `fetch_iter_flushed`, and a second connection per flow for +the acks (#7). + +## Decision + +One new module, `core/src/bus.rs`, hosts `async-nats` on one tokio runtime and +exposes a synchronous API. Nothing else in the runtime imports tokio or a NATS +type. Flows, the interpreter, connectors and supervisors do not change shape. +`Engine::live` and `Engine::shadow` are untouched and the interpreter has no bus +access, so `Program::writes_bus()` means what it meant. + +This is not an async rewrite. The rewrite was never on the table: threads are +what make a flow simple to read, and the cost above was the client's, not the +threads'. + +The eight points that make the module. + +### 1. One connection per process + +`bus::connect` returns the one `Bus` of a URL, whoever asks. Pull rounds, +publishes, requests, subscriptions and acks all share it. + +Whether data and acks need a connection each was measured, because finding #7 +said they did. They do not, at any flow count, and that decides it: + +| Events/s, drained with `bench/pub.py` | v0.3.6 | 2 conns, 2 workers | 1 conn, 2 workers | 2 conns, 1 worker | 1 conn, 1 worker | +|---|---:|---:|---:|---:|---:| +| 1 flow | 20 100 – 20 500 | 19 500 – 19 800 | 20 700 | 20 000 – 20 400 | 19 400 – 21 400 | +| 10 flows | 35 000 – 35 200 | 36 500 – 37 600 | 37 900 – 38 400 | 36 600 | 35 200 – 36 000 | +| 50 flows | 22 100 | 26 900 | 27 000 | 26 900 | 27 400 | +| 1 detect unit | 33 200 – 35 400 | 37 700 – 40 500 | 35 500 – 37 300 | not run | 40 700 – 41 100 | + +### 2. One worker + +One connection is one task, and the runtime runs nothing that blocks. In the table +above a second worker is about 5 % ahead at ten flows, about 10 % behind on a detect +unit and level elsewhere, for one more thread. The simpler one stays. + +`wait` parks the calling OS thread on a future. It is never called from inside +the runtime: tokio panics rather than deadlocks if it ever is, and a test pins +that. The event callback and the echo task are the only code of ours that runs +there. + +### 3. The barrier is rebuilt, because `flush` does not mean what it meant + +The at-least-once rule of a batch is: emits leave, the server confirms it has read +them, then the source messages are acked. The sync client's `flush` was a PING and +a PONG. `async_nats::Client::flush` only writes the socket. Using it would have +compiled, passed every test and acked messages the server never saw. + +There is no public PING, so the barrier publishes to a subject only this +connection subscribes to and waits to hear it back. A connection's messages are +handled in order, so the echo cannot overtake them. Two alternatives were +rejected. A request to a subject nobody listens to, relying on the 503, stalls +every flow as soon as someone runs `nats sub '>'`. Draining a subscription does +write a PING, but the client does not wait for the PONG. + +### 4. A reconnect fails the barrier + +On a reconnect `async-nats` replaces the connection and its write buffer with it: +what was not yet written is dropped, in silence, and what comes next leaves on the +new connection. A round-trip alone would then confirm a batch that lost its first +half. `Statistics::connects` moves on every connection, before any later command +is handled, so the barrier reads it before the batch leaves and again after the +echo. If it moved, the batch fails and redelivers, whether or not anything was +lost. The sync client could complete a pending PONG on reconnect, so this is +stricter than what it replaces. + +### 5. Emits are held per batch + +A first version handed each emit to the connection as it was made. It ran the flow +hop at 12 600 evt/s against 20 300 for v0.3.6, and the round phases on `/metrics` +put all of it in `process`: 33 µs more per message, CPU tripled. Each emit cost a +cross-thread wake and a write of its own. The 5 ms floor had been coalescing those +all along, as a side effect. So `Batch` holds a round's emits and hands them over +with the barrier: one wake and one write per round, and throughput came back. + +A batch is whatever has arrived, so a quiet flow's emit still leaves at once: the +server ends a round as soon as the consumer has caught up. Subject and size are +checked when an emit is held, so one bad emit fails its own message and not the +batch. + +### 6. Pull rounds keep their wire format + +`async-nats`'s pull streams manage their own requests and thresholds, which is not +what "one request in flight ahead of the round being processed" asks for. So +`Puller` owns an inbox on the shared connection and sends the same request as +before: +`{"batch":64,"expires":500000000,"no_wait":true}`. `Round` is the same iterator, +with one change: a 503, or a 409 saying the consumer was deleted, now fails the +next round so the supervisor recreates it. The old iterator handed those statuses +to the flow as empty messages. Deleting a consumer under a running flow on v0.3.6 +put two dead letters from an `_INBOX` subject in the queue and left the flow +`running`, consuming nothing, for good. Here it restarts once and carries on. + +While the connection is down no request is sent. The round times out empty and the +loop keeps its pace and its stop flag, then picks up where it was. + +### 7. Administration keeps its wire format too + +Streams, buckets and consumers are created through raw JetStream API requests, +with the bodies the sync client sent. The typed configs of `async-nats` carry +defaults of their own: a bucket would have come out with `allow_direct` on. KV and +object stores then open through `async-nats` on top of those streams. + +The result was checked three ways. `nats stream info` and `nats consumer info` on +a fresh store are identical to v0.3.6 for all eight streams and every durable; only +the ordered consumers behind a KV watch differ, and they hold no state. An upgrade +test ran v0.3.6 against a real store, then this build on it, then v0.3.6 again: +durable positions, the version overlay, a pending proposal, a dead letter, offsets +and the lease survive both ways, and a sequence opened under one binary closed +under the other, from the other client's detect snapshot. + +### 8. Two things the client does not do + +`async-nats` parses `nats://user:pass@host` and `nats://token@host` and connects +without them, where the sync client used them. `bus.rs` hands them over. No test +in the repo connects with credentials, so this would have failed in production. + +And `connect` refuses while the connection is down, as `nats::connect` did: a unit +that starts during an outage backs off and comes back, and an HTTP route says so. +Threads already running stay in place and resume on reconnect. + +## Measured + +v0.3.6 and this build alternating, one session on the 8-core WSL2 box of +`bench/README.md`, each run started under a load of 6. + +| | v0.3.6 | this build | +|---|---:|---:| +| Threads at 0 / 1 / 10 / 50 / 200 flows | 11 / 16 / 61 / 261 / 1 011 | 10 / 11 / 20 / 60 / 210 | +| NATS connections, same | 1 / 3 / 21 / 101 / 401 | 1 / 1 / 1 / 1 / 1 | +| Idle RSS, same, MB | 7.1 / 8.9 / 16.7 / 46.9 / 139.8 | 9.1 / 9.6 / 10.5 / 14.2 / 26.4 | +| Flow hop, `bench/flow-only.sh`, 16 publishers | 13 100 – 13 700/s | 13 200 – 13 500/s | +| Flow hop, drained with `bench/pub.py` | 20 100 – 20 500/s | 19 400 – 21 400/s | +| RSS under load, one flow | 9.2 MB | 10.1 – 10.3 MB | +| `bench/multi-flow.sh`, 10 flows | 12 300 – 13 000/s, 18.6 MB | 13 700 – 14 700/s, 14.5 MB | +| `bench/multi-flow.sh`, 50 flows | 11 300/s, 58 MB | 11 100 – 12 100/s, 38 MB | +| One detect unit, `bench/detect.sh` | 30 800 – 35 400/s, 11 MB | 34 700 – 41 100/s, 12.7 MB | +| Paced 2 000/s, p50 / p99 | 3–4 / 14–17 ms | 3 / 14–15 ms | +| Paced 20/s, p50 / p99 | 2 / 3–4 ms | 2 / 3 ms | +| `bench/run.sh 15 32`, ingest / delivered per s | 4 930 – 4 960 / 2 850 – 2 890 | 4 640 – 5 110 / 2 570 – 2 910 | +| MQTT loopback, `bench/broker-mqtt.sh 5000` | 2 370 – 2 430 rt/s | 2 050 – 2 090 rt/s | +| Spawn to `/healthz` 200 | 4.0 ms | 3.9 ms | +| Spawn to a flow running | 1 714 ms | 10.9 ms | +| Binary | 9.90 MB | 11.24 MB | + +`bench/flow-only.sh` is bound by its `nats` publishers on both sides, which is why +the second line is there. `bench/cluster.sh` delivers 20 000/20 000 through a +`kill -9` with the timer staying single, `bench/cluster-promote.sh` converges in +10 ms with 40 000/40 000 and no interleave, and all the e2e suites pass, the +connector admission job included. + +The MQTT loopback is the one number that fell, by 14 %. Both its legs work one +message at a time, a pub-ack on `mqtt-in` and an ack on `mqtt-out`, and each now +crosses to the worker and back where the sync client wrote from the calling thread. +That is the likely cost, not an isolated one. A batch has no such hop to lose, so +the flow hop did not feel it. + +## Connectors + +`connectors/mq` and `connectors/amqp` hold their own NATS client (ADR-0023, +ADR-0026) and move too, by compiling the same file: `#[path]` on +`core/src/bus.rs`, which depends on nothing else in core. Their `publish_confirmed` +and `fetch_round` go away. `connectors/amqp` also moves from rustls 0.22 to 0.23, +with no source change to its TLS adapter. Both are sync programs; "no tokio" is no +longer true of them, in the same contained sense as the runtime. + +## Consequences + +- `.cargo/audit.toml` loses seven entries, and every lockfile is on rustls 0.23.46 + and rustls-webpki 0.103.16. `core/Cargo.lock` goes from 251 packages to 201. +- An emit subject with whitespace or a line break is refused. The sync client + wrote it into the `PUB` line, so event data could inject protocol: reproduced on + the v0.3.6 binary, see the changelog. The refused message stays un-acked and + redelivers. It does not reach the dead-letter queue, because a failed emit was + always treated as transient; a payload over the server's limit behaves the same. +- A runtime whose NATS user has a publish allow-list needs `_INBOX.>` on it, for + the barrier. It already needed to subscribe there, and to answer `exec-rpc` and + control requests. Without it the emits leave, no batch is ever confirmed, nothing + is acked, and everything redelivers; the log names the permissions violation. No + deployment in this repo sets such a list, so this is an upgrade note. +- tokio is in the tree. It stays inside one file, and the runtime has one more + thing it depends on behaving. What `bus.rs` rebuilt (points 3, 4 and 8) is what a + client upgrade could break. The `bus::live` tests hold it against a real server; + like every test that needs one they are ignored in CI, so run them on an upgrade: + `cargo test bus::live -- --ignored`. +- One connection is one fate: when it drops, every flow waits at once, and a server + that closes it as a slow consumer stops the whole process. A pull bounds what a + flow can have in flight to 128 messages, which is what keeps that far away. +- The binary grows by 1.3 MB and RSS at one flow by about 1 MB. Past a few flows + the runtime is smaller than it was. + +## Not measured + +TLS performance (`tls://` was checked for behaviour: a private CA in the trust +store, an upgrade from `nats://`, an unknown CA refused). Failover between the +servers of a NATS cluster, or more than one server in `NATS_URL`: the runs here +restart one server in place. From 8c218dab10b27150116ff6e44be267699412d4ed Mon Sep 17 00:00:00 2001 From: Cyril Poder Date: Fri, 6 Nov 2026 19:47:50 +0100 Subject: [PATCH 2/9] core: async-nats behind one module, one connection for the process nats 0.25 is deprecated, 0.26 keeps rustls 0.22, and the client cost more than its size: two threads per connection and a connection per use, 1 011 threads and 401 connections at 200 flows, and a 5 ms flusher floor that three workarounds lived around. core/src/bus.rs hosts async-nats on one tokio worker and gives everything else a blocking API. No other module sees tokio or a NATS type, flows and connectors stay threads, and the interpreter still has no bus access. One connection carries the whole process, acks included: measured at 1, 10 and 50 flows, a second one buys nothing. publish_confirmed, fetch_iter_flushed and ack_nc go with the floor. Three things in the module are not obvious and matter. Client::flush only writes the socket, so it cannot stand for the sync client's PING. The barrier publishes to a subject only this connection subscribes to and waits to hear it back. And on a reconnect async-nats drops its unwritten buffer, so a round-trip alone could confirm a batch that lost its first half: a connection replaced while a batch was leaving fails the barrier, and the batch redelivers. Emits are held per batch. Handing each one over as it was made cost a cross-thread wake and a write per message, which the flusher floor had been coalescing without anyone asking: the flow hop fell from 20 300 to 12 600 evt/s, all of it in `process`. Held and sent with the barrier they cost one of each per round, and the rate came back. A quiet flow's emit still leaves at once. async-nats ignores credentials in the URL. bus.rs hands nats://user:pass@host and nats://token@host over itself; no test in the repo connects with credentials. Streams, buckets and durables are created with the JSON the sync client sent, so a fresh store comes out identical to v0.3.6's and an old one opens as it is. v0.3.6, then this build, then v0.3.6 again on one store: durable positions, a version overlay, a proposal, a dead letter, offsets, the lease and a detect snapshot all cross both ways. An emit subject with whitespace or a line break is now refused. v0.3.6 wrote it into the PUB line, so a flow building a subject from an event field let the event inject protocol: reproduced on that binary. The bus::live tests start their own nats-server and are ignored in CI like every test that needs one: cargo test bus::live -- --ignored Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_018wXgxT2UoFSZfqhAE1Tb5V --- core/Cargo.lock | 765 +++++--------------- core/Cargo.toml | 7 +- core/src/bus.rs | 1551 ++++++++++++++++++++++++++++++++++++++++ core/src/cluster.rs | 22 +- core/src/connectors.rs | 470 +++--------- core/src/control.rs | 8 +- core/src/main.rs | 290 +++----- core/src/proposals.rs | 21 +- core/src/versions.rs | 17 +- 9 files changed, 1953 insertions(+), 1198 deletions(-) create mode 100644 core/src/bus.rs diff --git a/core/Cargo.lock b/core/Cargo.lock index e20a5fd..127da93 100644 --- a/core/Cargo.lock +++ b/core/Cargo.lock @@ -33,77 +33,56 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d92bec98840b8f03a5ff5413de5293bfcd8bf96467cf5452609f939ec6f5de16" [[package]] -name = "async-channel" -version = "2.5.0" +name = "async-nats" +version = "0.52.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "924ed96dd52d1b75e9c1a3e6275715fd320f5f9439fb5a4a11fa51f4221158d2" +checksum = "015f0a7a3b0b29701b78c76a4d4a122624c97c71fa30363381664eb729d619c7" dependencies = [ - "concurrent-queue", - "event-listener-strategy", - "futures-core", - "pin-project-lite", + "base64", + "bytes", + "futures-util", + "memchr", + "pin-project", + "portable-atomic", + "rand", + "regex", + "ring", + "rustls-native-certs", + "rustls-pki-types", + "rustls-webpki", + "serde", + "serde_json", + "serde_nanos", + "serde_repr", + "thiserror", + "time", + "tokio", + "tokio-rustls", + "tokio-stream", + "tokio-util", + "tracing", + "tryhard", + "url", ] -[[package]] -name = "async-task" -version = "4.7.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8b75356056920673b02621b35afd0f7dda9306d03c79a30f5c56c44cf256e3de" - -[[package]] -name = "atomic-waker" -version = "1.1.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1505bd5d3d116872e7271a6d4e16d81d0c8570876c8de68093a09ac269d8aac0" - [[package]] name = "autocfg" version = "1.5.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53" -[[package]] -name = "base64" -version = "0.13.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9e1b586273c5702936fe7b7d6896644d8be71e6314cfe09d3167c95f712589e8" - [[package]] name = "base64" version = "0.22.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6" -[[package]] -name = "base64-url" -version = "1.4.13" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "67a99c239d0c7e77c85dddfa9cebce48704b3c49550fcd3b84dd637e4484899f" -dependencies = [ - "base64 0.13.1", -] - -[[package]] -name = "base64ct" -version = "1.8.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06" - [[package]] name = "bitflags" version = "2.13.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da" -[[package]] -name = "block-buffer" -version = "0.10.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71" -dependencies = [ - "generic-array", -] - [[package]] name = "block-buffer" version = "0.12.1" @@ -122,25 +101,21 @@ dependencies = [ "objc2", ] -[[package]] -name = "blocking" -version = "1.6.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e83f8d02be6967315521be875afa792a316e28d57b5a2d401897e2a7921b7f21" -dependencies = [ - "async-channel", - "async-task", - "futures-io", - "futures-lite", - "piper", -] - [[package]] name = "bumpalo" version = "3.20.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649" +[[package]] +name = "bytes" +version = "1.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fc652a48c352aef3ea3aed32080501cf3ef6ed5da78602a020c991775b0aff04" +dependencies = [ + "serde", +] + [[package]] name = "cc" version = "1.4.3" @@ -170,8 +145,8 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "65c35e4b699c7e15ccbe7ee35c005e4fc0a278d22238a2857e6ce2dadeda1b06" dependencies = [ "cfg-if", - "cpufeatures 0.3.1", - "rand_core 0.10.1", + "cpufeatures", + "rand_core", ] [[package]] @@ -194,21 +169,6 @@ version = "1.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "6e4de3bc4ea267985becf712dc6d9eed8b04c953b3fcfb339ebc87acd9804901" -[[package]] -name = "concurrent-queue" -version = "2.5.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "4ca0197aee26d1ae37445ee532fefce43251d24cc7c166799f4d46817f1d3973" -dependencies = [ - "crossbeam-utils", -] - -[[package]] -name = "const-oid" -version = "0.9.6" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c2459377285ad874054d797f3ccebf984978aa39129f6eafde5cdc8315b612f8" - [[package]] name = "const-oid" version = "0.10.2" @@ -217,9 +177,9 @@ checksum = "a6ef517f0926dd24a1582492c791b6a4818a4d94e789a334894aa15b0d12f55c" [[package]] name = "core-foundation" -version = "0.9.4" +version = "0.10.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "91e195e091a93c46f7102ec7818a2aa394e1e1771c3ab4825963fa03e45afb8f" +checksum = "b2a6cd9ae233e7f62ba4e9353e81a88df7fc8a5987b8d445b4d90c879bd156f6" dependencies = [ "core-foundation-sys", "libc", @@ -231,15 +191,6 @@ version = "0.8.7" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b" -[[package]] -name = "cpufeatures" -version = "0.2.17" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280" -dependencies = [ - "libc", -] - [[package]] name = "cpufeatures" version = "0.3.1" @@ -258,31 +209,6 @@ dependencies = [ "cfg-if", ] -[[package]] -name = "crossbeam-channel" -version = "0.5.16" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d85363c37faeca707aef026efa9f3b34d077bce547e48f770770625c6013679e" -dependencies = [ - "crossbeam-utils", -] - -[[package]] -name = "crossbeam-utils" -version = "0.8.22" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "61803da095bee82a81bb1a452ecc25d3b2f1416d1897eb86430c6159ef717c17" - -[[package]] -name = "crypto-common" -version = "0.1.7" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a" -dependencies = [ - "generic-array", - "typenum", -] - [[package]] name = "crypto-common" version = "0.2.2" @@ -303,49 +229,6 @@ dependencies = [ "windows-sys 0.61.2", ] -[[package]] -name = "curve25519-dalek" -version = "4.1.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "97fb8b7c4503de7d6ae7b42ab72a5a59857b4c937ec27a3d4539dba95b5ab2be" -dependencies = [ - "cfg-if", - "cpufeatures 0.2.17", - "curve25519-dalek-derive", - "digest 0.10.7", - "fiat-crypto", - "rustc_version", - "subtle", -] - -[[package]] -name = "curve25519-dalek-derive" -version = "0.1.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f46882e17999c6cc590af592290432be3bce0428cb0d5f8b6715e4dc7b383eb3" -dependencies = [ - "proc-macro2", - "quote", - "syn 2.0.119", -] - -[[package]] -name = "data-encoding" -version = "2.11.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "4583a4551df46e2792f82ceeac45e850d2e2d5debba0b91f102385cda5b11f06" - -[[package]] -name = "der" -version = "0.7.10" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e7c1832837b905bbfb5101e07cc24c8deddf52f93225eee6ead5f4d63d53ddcb" -dependencies = [ - "const-oid 0.9.6", - "pem-rfc7468", - "zeroize", -] - [[package]] name = "deranged" version = "0.5.8" @@ -355,25 +238,15 @@ dependencies = [ "serde_core", ] -[[package]] -name = "digest" -version = "0.10.7" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292" -dependencies = [ - "block-buffer 0.10.4", - "crypto-common 0.1.7", -] - [[package]] name = "digest" version = "0.11.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2" dependencies = [ - "block-buffer 0.12.1", - "const-oid 0.10.2", - "crypto-common 0.2.2", + "block-buffer", + "const-oid", + "crypto-common", ] [[package]] @@ -405,75 +278,12 @@ version = "1.0.20" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d0881ea181b1df73ff77ffaaf9c7544ecc11e82fba9b5f27b262a3c73a332555" -[[package]] -name = "ed25519" -version = "2.2.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "115531babc129696a58c64a4fef0a8bf9e9698629fb97e9e40767d235cfbcd53" -dependencies = [ - "signature", -] - -[[package]] -name = "ed25519-dalek" -version = "2.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "70e796c081cee67dc755e1a36a0a172b897fab85fc3f6bc48307991f64e4eca9" -dependencies = [ - "curve25519-dalek", - "ed25519", - "sha2 0.10.9", - "signature", - "subtle", -] - [[package]] name = "equivalent" version = "1.0.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f" -[[package]] -name = "event-listener" -version = "5.4.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5a23add41df1562121a9393cb065eab5146a1242410f23a644851e90cfd669d2" -dependencies = [ - "parking", - "pin-project-lite", -] - -[[package]] -name = "event-listener-strategy" -version = "0.5.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8be9f3dfaaffdae2972880079a491a1a8bb7cbed0b8dd7a347f668b4150a3b93" -dependencies = [ - "event-listener", - "pin-project-lite", -] - -[[package]] -name = "fastrand" -version = "1.9.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e51093e27b0797c359783294ca4f0a911c270184cb10f85783b118614a1501be" -dependencies = [ - "instant", -] - -[[package]] -name = "fastrand" -version = "2.5.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "da7c62ceae207dd37ea5b845da6a0696c799f85e97da1ab5b7910be3c1c80223" - -[[package]] -name = "fiat-crypto" -version = "0.2.9" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "28dea519a9695b9977216879a3ebfddf92f1c08c05d984f8996aecd6ecdc811d" - [[package]] name = "find-msvc-tools" version = "0.1.11" @@ -518,20 +328,10 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "92d699e522242e69e3003b94ecc1f960f3a5e015aa7c5d7486e65ad01dd94f5e" [[package]] -name = "futures-io" +name = "futures-sink" version = "0.3.34" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "53c0fa8157de1303bfffdaa1cc2a673bfffb60102f76b0ef4441659124373fed" - -[[package]] -name = "futures-lite" -version = "2.6.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f78e10609fe0e0b3f4157ffab1876319b5b0db102a2c60dc4626306dc46b44ad" -dependencies = [ - "futures-core", - "pin-project-lite", -] +checksum = "1944426bf7d03f1d14f708785e4b33efd750b36d48a157b836b3efc15ede8e1d" [[package]] name = "futures-task" @@ -546,21 +346,12 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0d50a92467f8ba5dd6e3ee5d4bd04d73ab2e4e1c44474a0674821dfce14b79bc" dependencies = [ "futures-core", + "futures-sink", "futures-task", "pin-project-lite", "slab", ] -[[package]] -name = "generic-array" -version = "0.14.7" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a" -dependencies = [ - "typenum", - "version_check", -] - [[package]] name = "getrandom" version = "0.2.17" @@ -581,7 +372,7 @@ dependencies = [ "cfg-if", "libc", "r-efi", - "rand_core 0.10.1", + "rand_core", ] [[package]] @@ -769,15 +560,6 @@ dependencies = [ "serde_core", ] -[[package]] -name = "instant" -version = "0.1.13" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e0242819d153cba4b4b05a5a8f2a7e9bbf97b6055b2a002b395c96b5ff3c0222" -dependencies = [ - "cfg-if", -] - [[package]] name = "itoa" version = "1.0.18" @@ -795,18 +577,6 @@ dependencies = [ "wasm-bindgen", ] -[[package]] -name = "json" -version = "0.12.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "078e285eafdfb6c4b434e0d31e8cfcb5115b651496faca5749b88fafd4f23bfd" - -[[package]] -name = "lazy_static" -version = "1.5.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe" - [[package]] name = "libc" version = "0.2.189" @@ -819,15 +589,6 @@ version = "0.8.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "47d9d19d1d6efa0109d2f65ff4c85cddd50bd572e5a00127ab10987290bcefae" -[[package]] -name = "lock_api" -version = "0.4.14" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "224399e74b87b5f3557511d98dff8b14089b3dadafcab6bb93eab67d3aace965" -dependencies = [ - "scopeguard", -] - [[package]] name = "log" version = "0.4.33" @@ -905,40 +666,14 @@ dependencies = [ ] [[package]] -name = "nats" -version = "0.25.0" +name = "mio" +version = "1.2.4" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "19090dd06e27eb59adfd16f332b85818a3e7dbbdbc13c448676c9bb69f964208" +checksum = "1788edb87fdc09c7e26304471e2f5be8cdefb1b6930d6e3985fc02ff53bf86ee" dependencies = [ - "base64 0.13.1", - "base64-url", - "blocking", - "crossbeam-channel", - "fastrand 1.9.0", - "itoa", - "json", - "lazy_static", "libc", - "log", - "memchr", - "nkeys", - "nuid", - "once_cell", - "parking_lot", - "portable-atomic", - "regex", - "ring", - "rustls 0.22.4", - "rustls-native-certs", - "rustls-pemfile", - "rustls-webpki 0.102.8", - "serde", - "serde_json", - "serde_nanos", - "serde_repr", - "time", - "url", - "winapi", + "wasi", + "windows-sys 0.61.2", ] [[package]] @@ -953,31 +688,6 @@ dependencies = [ "libc", ] -[[package]] -name = "nkeys" -version = "0.4.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "879011babc47a1c7fdf5a935ae3cfe94f34645ca0cac1c7f6424b36fc743d1bf" -dependencies = [ - "data-encoding", - "ed25519", - "ed25519-dalek", - "getrandom 0.2.17", - "log", - "rand 0.8.7", - "signatory", -] - -[[package]] -name = "nuid" -version = "0.3.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "20c1bb65186718d348306bf1afdeb20d9ab45b2ab80fb793c0fdcf59ffbb4f38" -dependencies = [ - "lazy_static", - "rand 0.8.7", -] - [[package]] name = "num-conv" version = "0.2.2" @@ -1016,47 +726,9 @@ checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" [[package]] name = "openssl-probe" -version = "0.1.6" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d05e27ee213611ffe7d6348b942e8f942b37114c00cc03cec254295a4a17852e" - -[[package]] -name = "parking" -version = "2.2.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f38d5652c16fde515bb1ecef450ab0f6a219d619a7274976324d5e377f7dceba" - -[[package]] -name = "parking_lot" -version = "0.12.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "93857453250e3077bd71ff98b6a65ea6621a19bb0f559a85248955ac12c45a1a" -dependencies = [ - "lock_api", - "parking_lot_core", -] - -[[package]] -name = "parking_lot_core" -version = "0.9.12" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2621685985a2ebf1c516881c026032ac7deafcda1a2c9b7850dc81e3dfcb64c1" -dependencies = [ - "cfg-if", - "libc", - "redox_syscall", - "smallvec", - "windows-link", -] - -[[package]] -name = "pem-rfc7468" -version = "0.7.0" +version = "0.2.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "88b39c9bfcfc231068454382784bb460aae594343fb030d46e9f50a645418412" -dependencies = [ - "base64ct", -] +checksum = "7e036b411754132fd6b74e94b74e9a13c5cf606fa93d1885c35860d3060820b1" [[package]] name = "percent-encoding" @@ -1107,31 +779,30 @@ dependencies = [ ] [[package]] -name = "pin-project-lite" -version = "0.2.17" +name = "pin-project" +version = "1.1.13" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd" +checksum = "2466b2336ed02bcdca6b294417127b90ec92038d1d5c4fbeac971a922e0e0924" +dependencies = [ + "pin-project-internal", +] [[package]] -name = "piper" -version = "0.2.5" +name = "pin-project-internal" +version = "1.1.13" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c835479a4443ded371d6c535cbfd8d31ad92c5d23ae9770a61bc155e4992a3c1" +checksum = "c96395f0a926bc13b1c17622aaddda1ecb55d49c8f1bf9777e4d877800a43f8b" dependencies = [ - "atomic-waker", - "fastrand 2.5.0", - "futures-io", + "proc-macro2", + "quote", + "syn 2.0.119", ] [[package]] -name = "pkcs8" -version = "0.10.2" +name = "pin-project-lite" +version = "0.2.17" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f950b2377845cebe5cf8b5165cb3cc1a5e0fa5cfa3e1f7f55707d8fd82e0a7b7" -dependencies = [ - "der", - "spki", -] +checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd" [[package]] name = "portable-atomic" @@ -1154,15 +825,6 @@ version = "0.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "439ee305def115ba05938db6eb1644ff94165c5ab5e9420d1c1bcedbba909391" -[[package]] -name = "ppv-lite86" -version = "0.2.21" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "85eae3c4ed2f50dcfe72643da4befc30deadb458a9b590d720cde2f2b1e97da9" -dependencies = [ - "zerocopy", -] - [[package]] name = "proc-macro2" version = "1.0.107" @@ -1187,17 +849,6 @@ version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf" -[[package]] -name = "rand" -version = "0.8.7" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "22f6172bdec972074665ed81ed53b71da00bfc44b65a753cfde883ec4c702a1a" -dependencies = [ - "libc", - "rand_chacha", - "rand_core 0.6.4", -] - [[package]] name = "rand" version = "0.10.3" @@ -1206,26 +857,7 @@ checksum = "65c9fb96cbc91e3478eaae79a69fcd3f1ae4ad052e471fe6732fff548984b4af" dependencies = [ "chacha20", "getrandom 0.4.3", - "rand_core 0.10.1", -] - -[[package]] -name = "rand_chacha" -version = "0.3.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e6c10a63a0fa32252be49d21e7709d4d4baf8d231c2dbce1eaa8141b9b127d88" -dependencies = [ - "ppv-lite86", - "rand_core 0.6.4", -] - -[[package]] -name = "rand_core" -version = "0.6.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ec0be4795e2f6a28069bec0b5ff3e2ac9bafc99e6a9a7dc3547996c5c816922c" -dependencies = [ - "getrandom 0.2.17", + "rand_core", ] [[package]] @@ -1234,15 +866,6 @@ version = "0.10.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "63b8176103e19a2643978565ca18b50549f6101881c443590420e4dc998a3c69" -[[package]] -name = "redox_syscall" -version = "0.5.18" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d" -dependencies = [ - "bitflags", -] - [[package]] name = "ref-cast" version = "1.0.27" @@ -1312,29 +935,6 @@ version = "2.1.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "6b1e7f9a428571be2dc5bc0505c13fb6bf936822b894ec87abf8a08a4e51742d" -[[package]] -name = "rustc_version" -version = "0.4.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "cfcb3a22ef46e85b45de6ee7e79d063319ebb6594faafcf1c225ea92ab6e9b92" -dependencies = [ - "semver", -] - -[[package]] -name = "rustls" -version = "0.22.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "bf4ef73721ac7bcd79b2b315da7779d8fc09718c6b3d2d1b2d94850eb8c18432" -dependencies = [ - "log", - "ring", - "rustls-pki-types", - "rustls-webpki 0.102.8", - "subtle", - "zeroize", -] - [[package]] name = "rustls" version = "0.23.46" @@ -1345,33 +945,23 @@ dependencies = [ "once_cell", "ring", "rustls-pki-types", - "rustls-webpki 0.103.16", + "rustls-webpki", "subtle", "zeroize", ] [[package]] name = "rustls-native-certs" -version = "0.7.3" +version = "0.8.5" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e5bfb394eeed242e909609f56089eecfe5fda225042e8b171791b9c95f5931e5" +checksum = "4a073f5dc7469f984c52ad2752b63b0807745133b6de880b7b64c1ac4c48aec4" dependencies = [ "openssl-probe", - "rustls-pemfile", "rustls-pki-types", "schannel", "security-framework", ] -[[package]] -name = "rustls-pemfile" -version = "2.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "dce314e5fee3f39953d46bb63bb8a46d40c2f8fb7cc5a3b6cab2bde9721d6e50" -dependencies = [ - "rustls-pki-types", -] - [[package]] name = "rustls-pki-types" version = "1.15.1" @@ -1381,17 +971,6 @@ dependencies = [ "zeroize", ] -[[package]] -name = "rustls-webpki" -version = "0.102.8" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "64ca1bc8749bd4cf37b5ce386cc146580777b4e8572c7b97baf22c83f444bee9" -dependencies = [ - "ring", - "rustls-pki-types", - "untrusted", -] - [[package]] name = "rustls-webpki" version = "0.103.16" @@ -1444,17 +1023,11 @@ dependencies = [ "syn 3.0.3", ] -[[package]] -name = "scopeguard" -version = "1.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49" - [[package]] name = "security-framework" -version = "2.11.1" +version = "3.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "897b2245f0b511c87893af39b033e5ca9cce68824c4d7e7630b5a1d339658d02" +checksum = "b7f4bc775c73d9a02cde8bf7b2ec4c9d12743edf609006c7facc23998404cd1d" dependencies = [ "bitflags", "core-foundation", @@ -1473,12 +1046,6 @@ dependencies = [ "libc", ] -[[package]] -name = "semver" -version = "1.0.28" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8a7852d02fc848982e0c167ef163aaff9cd91dc640ba85e263cb1ce46fae51cd" - [[package]] name = "serde" version = "1.0.229" @@ -1553,17 +1120,6 @@ dependencies = [ "syn 3.0.3", ] -[[package]] -name = "sha2" -version = "0.10.9" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283" -dependencies = [ - "cfg-if", - "cpufeatures 0.2.17", - "digest 0.10.7", -] - [[package]] name = "sha2" version = "0.11.0" @@ -1571,8 +1127,8 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "446ba717509524cb3f22f17ecc096f10f4822d76ab5c0b9822c5f9c284e825f4" dependencies = [ "cfg-if", - "cpufeatures 0.3.1", - "digest 0.11.3", + "cpufeatures", + "digest", ] [[package]] @@ -1581,28 +1137,6 @@ version = "2.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba" -[[package]] -name = "signatory" -version = "0.27.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c1e303f8205714074f6068773f0e29527e0453937fe837c9717d066635b65f31" -dependencies = [ - "pkcs8", - "rand_core 0.6.4", - "signature", - "zeroize", -] - -[[package]] -name = "signature" -version = "2.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "77549399552de45a898a580c1b41d445bf730df867cc44e6c0233bbc4b8329de" -dependencies = [ - "digest 0.10.7", - "rand_core 0.6.4", -] - [[package]] name = "simd-adler32" version = "0.3.10" @@ -1622,13 +1156,13 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8ed6a63f02c8539c91a8685a86f4099661ba3da017932f6ebbea6de3f0fa7c90" [[package]] -name = "spki" -version = "0.7.3" +name = "socket2" +version = "0.6.5" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d91ed6c858b01f942cd56b37a94b3e0a1798290327d1236e4d9cf4eaca44d29d" +checksum = "c3d1e2c7f27f8d4cb10542a02c49005dbd6e93095799d6f3be745fae9f8fedd4" dependencies = [ - "base64ct", - "der", + "libc", + "windows-sys 0.61.2", ] [[package]] @@ -1748,6 +1282,66 @@ dependencies = [ "zerovec", ] +[[package]] +name = "tokio" +version = "1.54.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b52fd3255258d26fac7264ab5a76b53c13367136dc011d3d6ad4f8ed070396a3" +dependencies = [ + "bytes", + "libc", + "mio", + "pin-project-lite", + "socket2", + "tokio-macros", + "windows-sys 0.61.2", +] + +[[package]] +name = "tokio-macros" +version = "2.7.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "78773a2a397f451582ce068015985c33193cf6dea8b74d2a639fe457b2f07b0e" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.3", +] + +[[package]] +name = "tokio-rustls" +version = "0.26.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c9cc2678c2cdd569ef8215e2afd7954ada2ae20b4fdd2c5fe6139a3b02d105db" +dependencies = [ + "rustls", + "tokio", +] + +[[package]] +name = "tokio-stream" +version = "0.1.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a3d06f0b082ba57c26b79407372e57cf2a1e28124f78e9479fe80322cf53420b" +dependencies = [ + "futures-core", + "pin-project-lite", + "tokio", +] + +[[package]] +name = "tokio-util" +version = "0.7.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e464cf451ba96ebfc6f9b6542f17ee8b8956e33f1e40d9690624e59d7a7f8a4b" +dependencies = [ + "bytes", + "futures-core", + "futures-sink", + "pin-project-lite", + "tokio", +] + [[package]] name = "tracing" version = "0.1.44" @@ -1779,6 +1373,16 @@ dependencies = [ "once_cell", ] +[[package]] +name = "tryhard" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9fe58ebd5edd976e0fe0f8a14d2a04b7c81ef153ea9a54eebc42e67c2c23b4e5" +dependencies = [ + "pin-project-lite", + "tokio", +] + [[package]] name = "typenum" version = "1.20.1" @@ -1815,11 +1419,11 @@ version = "2.12.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "02d1a66277ed75f640d608235660df48c8e3c19f3b4edb6a263315626cc3c01d" dependencies = [ - "base64 0.22.1", + "base64", "flate2", "log", "once_cell", - "rustls 0.23.46", + "rustls", "rustls-pki-types", "url", "webpki-roots 0.26.11", @@ -1927,12 +1531,12 @@ dependencies = [ "hashlink", "hex", "indexmap", - "rand 0.10.3", + "rand", "regex", "rustc-hash", "serde", "serde_json", - "sha2 0.11.0", + "sha2", "smallvec", "thiserror", "tracing", @@ -1978,22 +1582,19 @@ dependencies = [ name = "vejas-runtime" version = "0.4.0" dependencies = [ + "async-nats", + "bytes", "ctrlc", - "nats", - "rustls 0.23.46", + "futures-util", + "rustls", "serde_json", "tiny_http", + "tokio", "ureq", "varpulis-engine", "webpki-roots 0.26.11", ] -[[package]] -name = "version_check" -version = "0.9.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a" - [[package]] name = "wasi" version = "0.11.1+wasi-snapshot-preview1" @@ -2063,28 +1664,6 @@ dependencies = [ "rustls-pki-types", ] -[[package]] -name = "winapi" -version = "0.3.9" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5c839a674fcd7a98952e593242ea400abe93992746761e38641405d28b00f419" -dependencies = [ - "winapi-i686-pc-windows-gnu", - "winapi-x86_64-pc-windows-gnu", -] - -[[package]] -name = "winapi-i686-pc-windows-gnu" -version = "0.4.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6" - -[[package]] -name = "winapi-x86_64-pc-windows-gnu" -version = "0.4.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f" - [[package]] name = "windows-core" version = "0.62.2" @@ -2255,26 +1834,6 @@ dependencies = [ "synstructure", ] -[[package]] -name = "zerocopy" -version = "0.8.56" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "556764e583adb45a9f8d413c2a147fa7e8d821e48e12b14fd560b607998b75eb" -dependencies = [ - "zerocopy-derive", -] - -[[package]] -name = "zerocopy-derive" -version = "0.8.56" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f2ab42fc20575779bd240faa45f94a74256f755c0fa9e89f0ede20d91d0cdfc1" -dependencies = [ - "proc-macro2", - "quote", - "syn 2.0.119", -] - [[package]] name = "zerofrom" version = "0.1.8" diff --git a/core/Cargo.toml b/core/Cargo.toml index ff681cc..37c6768 100644 --- a/core/Cargo.toml +++ b/core/Cargo.toml @@ -11,7 +11,12 @@ serde_json = "1" # `termination` so SIGTERM/SIGHUP (a k8s rolling restart) hit the graceful drain # path, not default immediate kill — needed for lease handoff (ADR-0020). ctrlc = { version = "3.4", features = ["termination"] } -nats = "0.25" +# The NATS client and what it runs on. All four stay inside src/bus.rs (ADR-0035): +# the rest of the runtime is synchronous and sees neither tokio nor a NATS type. +async-nats = { version = "0.52", default-features = false, features = ["jetstream", "kv", "object-store", "ring", "server_2_10"] } +tokio = { version = "1", default-features = false, features = ["rt-multi-thread", "time"] } +bytes = "1" +futures-util = { version = "0.3", default-features = false, features = ["std"] } # Pooled, blocking HTTP(S) client for the http-out/http-poll/oauth-poll drivers # (replaces one curl process per request). Pure-Rust TLS (rustls), self-contained. ureq = "2" diff --git a/core/src/bus.rs b/core/src/bus.rs new file mode 100644 index 0000000..1da151f --- /dev/null +++ b/core/src/bus.rs @@ -0,0 +1,1551 @@ +// The bus island (ADR-0035): every NATS byte of the runtime goes through here. +// +// The runtime is synchronous and thread-based; the NATS client is not. So +// `async-nats` lives on one small tokio runtime owned by this module, and the +// rest of the code sees a blocking API and no NATS type. A flow, a connector or +// a supervisor is still a plain OS thread that calls a plain function. +// +// Three rules keep the island honest: +// +// * Every call that leaves a thread for the island goes through `wait`, which +// parks that OS thread. `wait` is never called from inside the runtime: the +// event callback and the echo task below are the only code that runs there, +// and neither calls back into this API. tokio panics rather than deadlocks +// if that ever changes (a test pins it). +// * The connection is shared. `connect` returns the one `Bus` of a URL, whoever +// asks, so a thousand flows are a thousand threads on one socket instead of +// two thousand sockets. +// * What the sync client gave for free is rebuilt here on purpose, because +// `async-nats` does not give it: see `Batch::confirm`. + +use std::cell::RefCell; +use std::collections::HashMap; +use std::future::Future; +use std::sync::atomic::{AtomicU64, Ordering}; +use std::sync::{Arc, Mutex, OnceLock}; +use std::time::{Duration, Instant}; + +use async_nats::client::{Request, Statistics}; +use async_nats::connection::State; +use async_nats::jetstream::{self, kv, object_store}; +use async_nats::{Client, ConnectOptions, Event, ServerAddr, StatusCode, Subject, Subscriber}; +use bytes::Bytes; +use futures_util::{FutureExt, StreamExt}; +use serde_json::{json, Value}; +use tokio::runtime::Runtime; +use tokio::sync::oneshot; +use tokio::time::timeout; + +// ───────────────────────── the runtime ───────────────────────── + +/// Worker threads of the island. One connection is one task, and the worker runs +/// nothing that blocks: a second one bought nothing at 1, 10 or 50 flows (ADR-0035). +const WORKERS: usize = 1; + +/// How long a publish may wait for room in a connection's queue. Past it the +/// connection is gone for good enough reasons to say so to the caller. +const SEND_WAIT: Duration = Duration::from_secs(5); + +/// How long a barrier may wait for its echo (the sync client's flush timeout). +const BARRIER_WAIT: Duration = Duration::from_secs(10); + +/// After a failed first connect, later callers get the same answer for this +/// long instead of queueing on the registry lock behind a dead address. +const CONNECT_MEMO: Duration = Duration::from_secs(1); + +fn runtime() -> &'static Runtime { + static RUNTIME: OnceLock = OnceLock::new(); + RUNTIME.get_or_init(|| { + tokio::runtime::Builder::new_multi_thread() + .worker_threads(WORKERS) + .thread_name("vejas-bus") + .enable_all() + .build() + .expect("the bus runtime") + }) +} + +/// Park this OS thread on a future of the island. Never call it from a task of +/// the runtime itself. +fn wait(future: F) -> F::Output { + runtime().block_on(future) +} + +/// `wait`, for at most `within`; `None` if the time ran out. The timer has to be +/// built inside the runtime, which is why this is not `wait(timeout(..))`. +fn wait_for(within: Duration, future: F) -> Option { + wait(async { timeout(within, future).await.ok() }) +} + +/// `future`, bounded by `SEND_WAIT`, with its error as text. +async fn sent( + future: impl Future>, +) -> Result { + match timeout(SEND_WAIT, future).await { + Ok(result) => result.map_err(|e| e.to_string()), + Err(_) => Err("timed out: the bus connection is not taking writes".into()), + } +} + +// ───────────────────────── connections ───────────────────────── + +/// One NATS connection, and the way to prove the server has read it. +struct Link { + client: Client, + stats: Arc, + /// A subject only this connection listens to, for the barrier. + echo: Subject, + next_id: AtomicU64, + waiting: Arc>>>, +} + +impl Link { + async fn open(servers: Vec) -> Result, String> { + let client = with_url_credentials(ConnectOptions::new(), &servers) + .name("vejas-runtime") + .event_callback(|event| async move { + match event { + Event::Connected | Event::Closed => {} + Event::Disconnected => eprintln!("[bus] disconnected, reconnecting"), + other => eprintln!("[bus] {other}"), + } + }) + .connect(servers) + .await + .map_err(|e| e.to_string())?; + let echo: Subject = client.new_inbox().into(); + let mut echoes = client + .subscribe(echo.clone()) + .await + .map_err(|e| e.to_string())?; + let waiting: Arc>>> = Arc::default(); + let waiters = waiting.clone(); + tokio::spawn(async move { + while let Some(echoed) = echoes.next().await { + let id = <[u8; 8]>::try_from(&echoed.payload[..]).map(u64::from_be_bytes); + if let Ok(id) = id { + if let Some(waiter) = waiters.lock().unwrap().remove(&id) { + let _ = waiter.send(()); + } + } + } + }); + Ok(Arc::new(Link { + stats: client.statistics(), + client, + echo, + next_id: AtomicU64::new(0), + waiting, + })) + } + + fn up(&self) -> bool { + self.client.connection_state() == State::Connected + } + + /// Which connection this link is on: it moves on every reconnect. + fn epoch(&self) -> u64 { + self.stats.connects.load(Ordering::Relaxed) + } + + /// Prove the server has read everything sent on this link so far: publish + /// to ourselves and wait to hear it. A connection's messages are handled in + /// the order they were sent, so the echo cannot overtake them. + async fn barrier(&self) -> Result<(), String> { + let id = self.next_id.fetch_add(1, Ordering::Relaxed); + let (tx, rx) = oneshot::channel(); + self.waiting.lock().unwrap().insert(id, tx); + let ask = self + .client + .publish(self.echo.clone(), Bytes::copy_from_slice(&id.to_be_bytes())); + let heard = match sent(ask).await { + Err(error) => Err(error), + Ok(()) => match timeout(BARRIER_WAIT, rx).await { + Ok(Ok(())) => Ok(()), + Ok(Err(_)) => Err("the barrier was dropped".to_string()), + Err(_) => Err("timed out waiting for the server".to_string()), + }, + }; + if heard.is_err() { + self.waiting.lock().unwrap().remove(&id); + } + heard + } + + /// The barrier, on the connection `epoch` names: one that was replaced in + /// the meantime fails it, whether or not anything was lost with it. + async fn barrier_since(&self, epoch: u64) -> Result<(), String> { + self.barrier().await?; + if self.epoch() != epoch { + return Err("the connection was replaced during the batch".into()); + } + Ok(()) + } +} + +/// The process's connection to one NATS deployment: a cheap handle, shared by +/// every thread that talks to it. +#[derive(Clone)] +pub struct Bus(Arc); + +struct Shared { + /// One connection for everything: publishes, requests, subscriptions, pull + /// rounds and acks. A connection of their own for the acks bought nothing + /// here at any flow count (ADR-0035). + link: Arc, + js: jetstream::Context, +} + +#[derive(Default)] +struct Registry { + open: HashMap, + failed: HashMap, +} + +/// The `Bus` of `url`, connecting on first use. `url` is what `NATS_URL` holds: +/// one server or a comma-separated list, with or without a scheme. Once it has +/// connected, a `Bus` reconnects on its own for as long as the process lives, +/// and threads already holding it pick up where they were. While it is down +/// this answers an error: a unit that starts now backs off and comes back, +/// as it did when every start dialled its own connection. +pub fn connect(url: &str) -> Result { + static BUSES: OnceLock> = OnceLock::new(); + let mut registry = BUSES + .get_or_init(Default::default) + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + if let Some(bus) = registry.open.get(url) { + if !bus.0.link.up() { + return Err("the connection to NATS is down".into()); + } + return Ok(bus.clone()); + } + if let Some((at, error)) = registry.failed.get(url) { + if at.elapsed() < CONNECT_MEMO { + return Err(error.clone()); + } + } + match open(url) { + Ok(bus) => { + registry.open.insert(url.to_string(), bus.clone()); + Ok(bus) + } + Err(error) => { + registry + .failed + .insert(url.to_string(), (Instant::now(), error.clone())); + Err(error) + } + } +} + +/// `nats://user:pass@host` and `nats://token@host`: `async-nats` parses these and +/// then connects without them, so they are handed over here. One set for the +/// connection: the first server that names any. +fn with_url_credentials(options: ConnectOptions, servers: &[ServerAddr]) -> ConnectOptions { + let named = servers + .iter() + .find_map(|server| Some((server.username()?, server.password()))); + match named { + Some((user, Some(pass))) => options.user_and_password(user.to_string(), pass.to_string()), + Some((token, None)) => options.token(token.to_string()), + None => options, + } +} + +fn servers_of(url: &str) -> Result, String> { + url.split(',') + .map(|server| { + server + .trim() + .parse::() + .map_err(|e| e.to_string()) + }) + .collect() +} + +fn open(url: &str) -> Result { + let servers = servers_of(url)?; + wait(async move { + let link = Link::open(servers).await?; + let js = jetstream::new(link.client.clone()); + Ok(Bus(Arc::new(Shared { link, js }))) + }) +} + +impl Bus { + pub fn jetstream(&self) -> JetStream { + JetStream(self.clone()) + } + + /// Publish one message now, with no promise: for what stands for nothing to + /// ack. An error means it was never handed to the connection. + pub fn publish(&self, subject: &str, payload: impl Into) -> Result<(), String> { + let (client, payload) = (&self.0.link.client, payload.into()); + wait(sent(client.publish(subject.to_string(), payload))) + } + + /// A batch of publishes that leave together, see [`Batch`]. + pub fn batch(&self) -> Batch { + Batch { + bus: self.clone(), + held: Vec::new(), + } + } + + /// Ack every message, in one go. An ack that does not make it only means a + /// redelivery. + pub fn ack_all<'a>( + &self, + messages: impl IntoIterator, + ) -> Result<(), String> { + let client = &self.0.link.client; + wait(async { + for message in messages { + if let Some(reply) = &message.reply { + sent(client.publish(reply.clone(), Bytes::new())).await?; + } + } + Ok(()) + }) + } + + /// A core request: the answer, or an error when nobody answers within `within`. + pub fn request( + &self, + subject: &str, + payload: impl Into, + within: Duration, + ) -> Result { + let request = Request::new().payload(payload.into()).timeout(Some(within)); + let answer = wait( + self.0 + .link + .client + .send_request(subject.to_string(), request), + ) + .map_err(|e| e.to_string())?; + Ok(Message::new(answer, self.0.link.clone())) + } + + pub fn subscribe(&self, subject: &str) -> Result { + let client = &self.0.link.client; + let sub = wait(sent(client.subscribe(subject.to_string())))?; + Ok(Subscription { + sub, + link: self.0.link.clone(), + }) + } + + pub fn queue_subscribe(&self, subject: &str, group: &str) -> Result { + let client = &self.0.link.client; + let sub = wait(sent( + client.queue_subscribe(subject.to_string(), group.to_string()), + ))?; + Ok(Subscription { + sub, + link: self.0.link.clone(), + }) + } +} + +/// The publishes of one batch, held back until [`Batch::confirm`]. A loop that +/// publishes before it acks keeps its emits here, one batch at a time. +/// +/// Handing each emit to the connection as it is made costs a cross-thread wake +/// and a write per message, the most expensive thing on the hot path (~33 µs +/// each on the flow hop, measured: 20 000 evt/s fell to 12 500). Held until the +/// end of the batch they cost one of each per batch, and the batch is still +/// whatever has arrived, so a quiet flow's emit leaves as soon as it is made. +pub struct Batch { + bus: Bus, + held: Vec<(Subject, Bytes)>, +} + +impl Batch { + /// Hold a publish. An error means this one can never leave (a subject the + /// protocol cannot carry, a payload over the server's limit), and only this + /// one: the caller leaves its source message un-acked, and the rest of the + /// batch goes on. + pub fn publish(&mut self, subject: &str, payload: impl Into) -> Result<(), String> { + let payload = payload.into(); + if subject.is_empty() || subject.contains([' ', '\t', '\r', '\n']) { + return Err(format!("invalid subject {subject:?}")); + } + let limit = self.bus.0.link.client.max_payload(); + if payload.len() > limit { + return Err(format!( + "{} bytes is over the server's limit of {limit}", + payload.len() + )); + } + self.held.push((subject.to_string().into(), payload)); + Ok(()) + } + + /// Send what is held and prove it arrived: the publish-before-ack barrier. + /// Ok means the server has read every publish of the batch, so their source + /// messages may be acked; a batch with nothing held has nothing to prove. + /// The batch is empty afterwards, either way. Two things stand in here for + /// what the sync client's `flush` was. + /// + /// `async_nats::Client::flush` only writes the socket; it proves nothing + /// about the server. The barrier is a round-trip through it instead, see + /// [`Link::barrier`]. + /// + /// And on a reconnect `async-nats` drops what it had not yet written: the + /// old connection's buffer goes with it, in silence, and whatever comes next + /// leaves on the new one. A round-trip alone would then confirm a batch that + /// lost its first half. So a connection replaced while the batch was leaving + /// fails the barrier, and the batch redelivers. + pub fn confirm(&mut self) -> Result<(), String> { + if self.held.is_empty() { + return Ok(()); + } + let link = &self.bus.0.link; + if !link.up() { + self.held.clear(); + return Err("the connection is down".into()); + } + let epoch = link.epoch(); + let held = &mut self.held; + wait(async { + for (subject, payload) in held.drain(..) { + sent(link.client.publish(subject, payload)).await?; + } + link.barrier_since(epoch).await + }) + } +} + +/// A plain subscription: the messages arrive whether or not anyone is waiting, +/// and go back with `respond`. +pub struct Subscription { + sub: Subscriber, + link: Arc, +} + +impl Subscription { + /// The next message, or an error if none arrives within `within`. + pub fn next_timeout(&mut self, within: Duration) -> Result { + let next = match self.sub.next().now_or_never() { + Some(next) => next, + None => wait_for(within, self.sub.next()).ok_or("timed out")?, + }; + match next { + Some(message) => Ok(Message::new(message, self.link.clone())), + None => Err("subscription closed".into()), + } + } +} + +/// A message off the bus. +pub struct Message { + pub subject: String, + pub data: Bytes, + reply: Option, + link: Arc, +} + +/// What JetStream says about a delivery, read from its reply subject. +#[derive(Debug, PartialEq, Eq)] +pub struct Delivery { + /// How many times this message has been delivered, this time included. + pub delivered: i64, + pub stream_seq: u64, +} + +impl Message { + fn new(message: async_nats::Message, link: Arc) -> Message { + Message { + subject: message.subject.to_string(), + data: message.payload, + reply: message.reply, + link, + } + } + + /// The JetStream delivery behind this message, if it is one. + pub fn delivery(&self) -> Option { + parse_ack_subject(self.reply.as_deref()?) + } + + pub fn ack(&self) -> Result<(), String> { + self.answer(Bytes::new()) + } + + pub fn nak(&self) -> Result<(), String> { + self.answer(Bytes::from_static(b"-NAK")) + } + + /// Answer a request. A message that expects no answer is left alone. + pub fn respond(&self, payload: impl AsRef<[u8]>) -> Result<(), String> { + self.answer(Bytes::copy_from_slice(payload.as_ref())) + } + + fn answer(&self, payload: Bytes) -> Result<(), String> { + let Some(reply) = &self.reply else { + return Ok(()); + }; + wait(sent(self.link.client.publish(reply.clone(), payload))) + } +} + +/// `$JS.ACK.......`, +/// or the same with `.` in front and a token behind. +fn parse_ack_subject(reply: &str) -> Option { + let tokens: Vec<&str> = reply.strip_prefix("$JS.ACK.")?.split('.').collect(); + let at = match tokens.len() { + 7 => 2, + n if n >= 9 => 4, + _ => return None, + }; + Some(Delivery { + delivered: tokens[at].parse().ok()?, + stream_seq: tokens[at + 1].parse().ok()?, + }) +} + +// ───────────────────────── pull rounds ───────────────────────── + +/// Messages asked for in one pull request. +const PULL_BATCH: usize = 64; + +/// The idle poll window (`expires`). One outstanding pull covers this whole span, +/// so a message arriving any time inside it is delivered at once: there is NO gap +/// between rounds where a message would sit unpolled. It is therefore two things +/// at once: the idle pull cadence (one pull per window when quiet) AND the +/// worst-case stop-response latency (the loop re-checks its `alive` flag each +/// time a pull returns). 500 ms keeps idle chatter to ~2 pulls/s per consumer +/// while leaving stop far inside the transport suite's 3 s bound (T5). +const PULL_EXPIRES_MS: u64 = 500; + +/// A continuous long-poll. `no_wait` returns the instant messages are available +/// (a single message is delivered at once, no batch-fill latency, and a high-rate +/// drain still fills 64-batches in one round). When the subject is idle the pull +/// is HELD server-side for `expires` and comes back empty, so a loop paces itself +/// on the server with no client sleep and no coverage gap. Nothing parks in +/// `max_waiting`: every request resolves within `expires`, so a stopped loop is +/// never wedged on one. (An unbounded fetch parks forever on an idle subject: a +/// stopped flow's thread stayed in `recv()`, reported "running", and processed +/// one more message before dying, the zombie-consumer trap caught while +/// rehearsing the demo.) +const PULL_REQUEST: &[u8] = b"{\"batch\":64,\"expires\":500000000,\"no_wait\":true}"; + +/// Client-side backstop, comfortably past the server's `expires`, so that the +/// server's own status ends a round and not this timeout. +const PULL_WAIT: Duration = Duration::from_millis(PULL_EXPIRES_MS + 250); + +/// A pull consumer, bound to one inbox of its own on the shared connection. The +/// messages of every outstanding request land in that inbox, in order, so a +/// caller can keep the next request in flight while it works through a round. +pub struct Puller { + next: Subject, + inbox: Subject, + sub: RefCell, + /// Set when a round learned that the consumer is gone: the next round + /// fails, and the supervisor's attempt starts over from the top. + fault: RefCell>, + link: Arc, + /// The stream and name of an ephemeral consumer, deleted with this handle. + owned: Option<(String, String)>, + js: JetStream, +} + +impl Puller { + /// Send one pull request and return its round. The request leaves now, and + /// the messages come back one by one as they arrive. + /// + /// With the connection down no request is sent: the round times out empty, + /// and the caller's loop keeps its pace and its stop flag until the + /// connection is back, when it picks up where it was. + pub fn round(&self) -> Result, String> { + if let Some(fault) = self.fault.borrow_mut().take() { + return Err(fault); + } + if self.link.up() { + let request = self.link.client.publish_with_reply( + self.next.clone(), + self.inbox.clone(), + Bytes::from_static(PULL_REQUEST), + ); + wait(sent(request))?; + } + Ok(Round { + puller: self, + taken: 0, + }) + } + + fn receive(&self, deadline: Instant) -> Option { + let mut sub = self.sub.borrow_mut(); + match sub.next().now_or_never() { + Some(next) => next, + None => wait_for(deadline - Instant::now(), sub.next()).flatten(), + } + } +} + +impl Drop for Puller { + fn drop(&mut self) { + if let Some((stream, consumer)) = self.owned.take() { + if self.link.up() { + let _ = self.js.delete_consumer(&stream, &consumer); + } + } + } +} + +/// The messages of one pull request: it ends with the batch, on the server's own +/// status, or at the backstop. With a request in flight behind it, a status may +/// be that request's rather than this one's: both mean "nothing more for now". +pub struct Round<'a> { + puller: &'a Puller, + taken: usize, +} + +impl Iterator for Round<'_> { + type Item = Message; + + fn next(&mut self) -> Option { + let deadline = Instant::now() + PULL_WAIT; + while self.taken < PULL_BATCH { + let message = self.puller.receive(deadline)?; + match classify(&message) { + Pulled::Message => { + self.taken += 1; + return Some(Message::new(message, self.puller.link.clone())); + } + Pulled::Nothing => {} + Pulled::Over => return None, + Pulled::Gone(why) => { + *self.puller.fault.borrow_mut() = Some(why); + return None; + } + } + } + None + } +} + +enum Pulled { + Message, + /// 404: a request came back empty; keep waiting for the rest of the round. + Nothing, + /// 408, or a 409 that leaves the consumer standing: this round is over. + Over, + /// The consumer or JetStream is not there any more. + Gone(String), +} + +fn classify(message: &async_nats::Message) -> Pulled { + let description = message.description.as_deref().unwrap_or(""); + match message.status { + None | Some(StatusCode::OK) => Pulled::Message, + Some(StatusCode::NOT_FOUND) => Pulled::Nothing, + Some(StatusCode::TIMEOUT) => Pulled::Over, + Some(StatusCode::REQUEST_TERMINATED) if !description.contains("Deleted") => Pulled::Over, + Some(status) => Pulled::Gone(format!("consumer gone: {status} {description}")), + } +} + +// ───────────────────────── JetStream ───────────────────────── + +/// The persisted side of a `Bus`. +#[derive(Clone)] +pub struct JetStream(Bus); + +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] +pub enum Discard { + #[default] + Old, + New, +} + +/// What a stream is created with. The fields not named here keep the server's +/// defaults, and `add_stream` sends these and only these, so a stream comes out +/// the same as one made by any earlier release of the runtime. +#[derive(Clone, Default)] +pub struct StreamConfig { + pub name: String, + pub subjects: Vec, + pub max_msgs: i64, + pub max_msgs_per_subject: i64, + pub max_age: Duration, + pub discard: Discard, + pub allow_rollup: bool, + pub deny_delete: bool, +} + +/// A key-value bucket. The stream behind it is the one the sync client made, +/// `allow_direct` off included, so buckets of earlier releases and new ones are +/// the same thing. +#[derive(Clone, Default)] +pub struct KvConfig { + pub bucket: String, + pub history: i64, + pub max_age: Duration, +} + +/// A consumer that is not kept: named by the server, deleted with its `Puller`, +/// reaped by the server if the process dies first. +pub struct EphemeralConfig<'a> { + pub filter: &'a str, + /// Start at this stream sequence; `None` follows the tail. + pub start_seq: Option, + pub inactive_threshold: Duration, +} + +#[derive(Clone, Copy, Debug)] +pub struct StreamState { + pub first_seq: u64, + pub last_seq: u64, + pub messages: u64, +} + +pub struct StoredMessage { + pub sequence: u64, + pub data: Bytes, +} + +impl JetStream { + fn shared(&self) -> &Shared { + &(self.0).0 + } + + /// The JetStream API, as the server speaks it: a request and a JSON reply. + /// Administration goes this way rather than through the client's typed + /// configs because each of those is free to carry defaults of its own, and + /// an in-place upgrade must find the same streams and consumers it left. + fn api(&self, verb: String, body: Bytes) -> Result { + let client = &self.shared().link.client; + let reply = + wait(client.request(format!("$JS.API.{verb}"), body)).map_err(|e| e.to_string())?; + let answer: Value = + serde_json::from_slice(&reply.payload).map_err(|e| format!("unreadable reply: {e}"))?; + match answer.get("error") { + None => Ok(answer), + Some(error) => { + let what = error + .get("description") + .and_then(Value::as_str) + .unwrap_or("error"); + Err(what.to_string()) + } + } + } + + /// Publish and wait for the stream to say it stored the message. Ok means + /// persisted; the stream sequence comes back. + pub fn publish(&self, subject: &str, payload: impl AsRef<[u8]>) -> Result { + let bytes = Bytes::copy_from_slice(payload.as_ref()); + wait(async { + let ack = self.shared().js.publish(subject.to_string(), bytes).await; + let ack = ack.map_err(|e| e.to_string())?.await; + ack.map(|ack| ack.sequence).map_err(|e| e.to_string()) + }) + } + + /// Create a stream, or leave it as it is: one that exists is never changed, + /// and a different config on it is an error the caller may ignore. + pub fn add_stream(&self, config: &StreamConfig) -> Result<(), String> { + let discard = match config.discard { + Discard::Old => "old", + Discard::New => "new", + }; + let mut body = json!({ + "name": config.name, + "subjects": config.subjects, + "max_msgs": config.max_msgs, + "max_msgs_per_subject": config.max_msgs_per_subject, + "max_age": config.max_age.as_nanos() as u64, + "discard": discard, + "retention": "limits", + "storage": "file", + }); + if config.allow_rollup { + body["allow_rollup_hdrs"] = json!(true); + } + if config.deny_delete { + body["deny_delete"] = json!(true); + } + self.api( + format!("STREAM.CREATE.{}", config.name), + body.to_string().into(), + ) + .map(|_| ()) + } + + #[cfg(test)] + pub fn delete_stream(&self, stream: &str) -> Result<(), String> { + self.api(format!("STREAM.DELETE.{stream}"), Bytes::new()) + .map(|_| ()) + } + + pub fn stream_state(&self, stream: &str) -> Result { + let info = self.api(format!("STREAM.INFO.{stream}"), Bytes::new())?; + let seq = |field: &str| info["state"][field].as_u64().unwrap_or(0); + Ok(StreamState { + first_seq: seq("first_seq"), + last_seq: seq("last_seq"), + messages: seq("messages"), + }) + } + + /// The message at a stream sequence. + pub fn get_message(&self, stream: &str, sequence: u64) -> Result { + let js = &self.shared().js; + let message = wait(async { + let stream = js + .get_stream_no_info(stream) + .await + .map_err(|e| e.to_string())?; + stream + .get_raw_message(sequence) + .await + .map_err(|e| e.to_string()) + })?; + Ok(StoredMessage { + sequence: message.sequence, + data: message.payload, + }) + } + + pub fn delete_message(&self, stream: &str, sequence: u64) -> Result<(), String> { + let body = json!({"seq": sequence}).to_string(); + self.api(format!("STREAM.MSG.DELETE.{stream}"), body.into()) + .map(|_| ()) + } + + /// Empty a stream; how many messages went comes back. + pub fn purge_stream(&self, stream: &str) -> Result { + let reply = self.api(format!("STREAM.PURGE.{stream}"), Bytes::new())?; + Ok(reply["purged"].as_u64().unwrap_or(0)) + } + + /// Create (or verify) a durable pull consumer that delivers every one of + /// `filters` in stream order, from the beginning or, with `start_seq`, from + /// that stream sequence on. Idempotent: the same request on an existing + /// consumer is accepted, and 2.10 updates the filters of an existing one in + /// place (its ack floor stays). The reply's filters must be exactly the ones + /// asked for: a server that ignored `filter_subjects` would deliver the + /// whole stream. + pub fn create_pull_consumer( + &self, + stream: &str, + durable: &str, + filters: &[String], + ack_wait: Duration, + start_seq: Option, + ) -> Result<(), String> { + let mut config = json!({ + "durable_name": durable, + "ack_policy": "explicit", + "ack_wait": ack_wait.as_nanos() as u64, + "deliver_policy": "all", + "replay_policy": "instant", + }); + // Resuming after a snapshot: start at the sequence after it. The server + // cannot change an existing consumer's deliver policy, so the caller + // deletes the durable first. + if let Some(seq) = start_seq { + config["deliver_policy"] = Value::from("by_start_sequence"); + config["opt_start_seq"] = Value::from(seq); + } + match filters { + [one] => config["filter_subject"] = Value::String(one.clone()), + many => config["filter_subjects"] = Value::from(many.to_vec()), + } + let body = json!({ "stream_name": stream, "config": config }).to_string(); + let reply = self + .api( + format!("CONSUMER.DURABLE.CREATE.{stream}.{durable}"), + body.into(), + ) + .map_err(|e| format!("consumer {durable}: {e}"))?; + let mut kept = filters_of(&reply["config"]); + let mut asked = filters.to_vec(); + kept.sort(); + asked.sort(); + if kept != asked { + return Err(format!( + "consumer {durable}: the server kept the filters {kept:?}, not {asked:?} \ + (several `.from()` subjects need NATS 2.10 or newer)" + )); + } + Ok(()) + } + + pub fn delete_consumer(&self, stream: &str, consumer: &str) -> Result<(), String> { + self.api(format!("CONSUMER.DELETE.{stream}.{consumer}"), Bytes::new()) + .map(|_| ()) + } + + /// The stream sequence below which a consumer has acked everything. + pub fn ack_floor(&self, stream: &str, consumer: &str) -> Result { + let info = self.api(format!("CONSUMER.INFO.{stream}.{consumer}"), Bytes::new())?; + Ok(info["ack_floor"]["stream_seq"].as_u64().unwrap_or(0)) + } + + /// Bind to a durable consumer that must exist. With `filter`, it must also + /// be the one consuming that subject. + pub fn pull_durable( + &self, + stream: &str, + durable: &str, + filter: Option<&str>, + ) -> Result { + let info = self + .api(format!("CONSUMER.INFO.{stream}.{durable}"), Bytes::new()) + .map_err(|e| format!("provided durable consumer doesn't exist: {e}"))?; + if let Some(filter) = filter { + let filters = filters_of(&info["config"]); + if !filters.is_empty() && filters != [filter] { + return Err("subjects do not match".into()); + } + } + self.puller(stream, durable, None) + } + + /// A read-only cursor over a stream: a consumer of its own, never acked, so + /// nothing a live consumer does or sees changes. + pub fn pull_ephemeral(&self, stream: &str, config: &EphemeralConfig) -> Result { + let mut consumer = json!({ + "deliver_policy": "new", + "ack_policy": "explicit", + "filter_subject": config.filter, + "replay_policy": "instant", + "inactive_threshold": config.inactive_threshold.as_nanos() as u64, + }); + if let Some(seq) = config.start_seq { + consumer["deliver_policy"] = Value::from("by_start_sequence"); + consumer["opt_start_seq"] = Value::from(seq); + } + let body = json!({ "stream_name": stream, "config": consumer }).to_string(); + let info = self.api(format!("CONSUMER.CREATE.{stream}"), body.into())?; + let name = info["name"] + .as_str() + .ok_or("the server named no consumer")? + .to_string(); + self.puller(stream, &name, Some((stream.to_string(), name.clone()))) + } + + fn puller( + &self, + stream: &str, + consumer: &str, + owned: Option<(String, String)>, + ) -> Result { + let shared = self.shared(); + let inbox: Subject = shared.link.client.new_inbox().into(); + let sub = wait(sent(shared.link.client.subscribe(inbox.clone())))?; + Ok(Puller { + next: format!("$JS.API.CONSUMER.MSG.NEXT.{stream}.{consumer}").into(), + inbox, + sub: RefCell::new(sub), + fault: RefCell::new(None), + link: shared.link.clone(), + owned, + js: self.clone(), + }) + } + + /// Open a bucket, creating it if it is not there. None if JetStream KV is + /// unavailable. + pub fn key_value(&self, config: &KvConfig) -> Option { + let _ = self.add_stream(&StreamConfig { + name: format!("KV_{}", config.bucket), + subjects: vec![format!("$KV.{}.>", config.bucket)], + max_msgs_per_subject: config.history.max(1), + max_age: config.max_age, + discard: Discard::New, + allow_rollup: true, + deny_delete: true, + ..Default::default() + }); + let store = wait(self.shared().js.get_key_value(config.bucket.clone())).ok()?; + Some(Kv { store }) + } + + /// Open an object bucket, creating it if it is not there. None if JetStream + /// is unavailable. + pub fn object_store(&self, bucket: &str) -> Option { + let _ = self.add_stream(&StreamConfig { + name: format!("OBJ_{bucket}"), + subjects: vec![format!("$O.{bucket}.C.>"), format!("$O.{bucket}.M.>")], + discard: Discard::New, + allow_rollup: true, + ..Default::default() + }); + let store = wait(self.shared().js.get_object_store(bucket)).ok()?; + Some(Objects { store }) + } +} + +/// The subject filters of a consumer's config, however it holds them. +fn filters_of(config: &Value) -> Vec { + match config.get("filter_subjects").and_then(Value::as_array) { + Some(all) => all + .iter() + .filter_map(|x| x.as_str().map(str::to_string)) + .collect(), + None => config + .get("filter_subject") + .and_then(Value::as_str) + .map(|one| vec![one.to_string()]) + .unwrap_or_default(), + } +} + +// ───────────────────────── key-value ───────────────────────── + +/// A JetStream KV bucket. Revisions are stream sequences; `create` and `update` +/// are compare-and-set on them, which is what the singleton lease stands on +/// (ADR-0020). +#[derive(Clone)] +pub struct Kv { + store: kv::Store, +} + +impl Kv { + pub fn get(&self, key: &str) -> Result, String> { + wait(self.store.get(key)).map_err(|e| e.to_string()) + } + + pub fn put(&self, key: &str, value: impl AsRef<[u8]>) -> Result { + let value = Bytes::copy_from_slice(value.as_ref()); + wait(self.store.put(key, value)).map_err(|e| e.to_string()) + } + + /// Atomic create-if-absent: a deleted key counts as absent. The revision + /// comes back; a key that exists is an error. + pub fn create(&self, key: &str, value: impl AsRef<[u8]>) -> Result { + let value = Bytes::copy_from_slice(value.as_ref()); + wait(self.store.create(key, value)).map_err(|e| e.to_string()) + } + + /// Compare-and-set: fails unless `revision` is still the key's latest. + pub fn update(&self, key: &str, value: impl AsRef<[u8]>, revision: u64) -> Result { + let value = Bytes::copy_from_slice(value.as_ref()); + wait(self.store.update(key, value, revision)).map_err(|e| e.to_string()) + } + + pub fn delete(&self, key: &str) -> Result<(), String> { + wait(self.store.delete(key)).map_err(|e| e.to_string()) + } + + /// The keys that hold a value now. + pub fn keys(&self) -> Result, String> { + let collect = async { + let mut keys = self.store.keys().await.map_err(|e| e.to_string())?; + let mut all = Vec::new(); + while let Some(key) = keys.next().await { + all.push(key.map_err(|e| e.to_string())?); + } + Ok(all) + }; + wait_for(BARRIER_WAIT, collect).unwrap_or_else(|| Err("timed out listing keys".into())) + } + + /// Every change to the bucket, starting with its current keys. + pub fn watch_all(&self) -> Result { + let watch = wait(self.store.watch_with_history(">")).map_err(|e| e.to_string())?; + Ok(KvWatch { watch }) + } +} + +pub struct KvWatch { + watch: kv::Watch, +} + +impl KvWatch { + /// The next key that changed, `None` if nothing did within `within`, an + /// error when the watch has ended and must be opened again. + pub fn next_timeout(&mut self, within: Duration) -> Result, String> { + match wait_for(within, self.watch.next()) { + None => Ok(None), + Some(Some(Ok(entry))) => Ok(Some(entry.key)), + Some(Some(Err(e))) => Err(e.to_string()), + Some(None) => Err("the watch ended".into()), + } + } +} + +// ───────────────────────── objects ───────────────────────── + +/// A JetStream object bucket. +#[derive(Clone)] +pub struct Objects { + store: object_store::ObjectStore, +} + +impl Objects { + pub fn put(&self, name: &str, mut bytes: &[u8]) -> Result<(), String> { + wait(self.store.put(name, &mut bytes)) + .map(|_| ()) + .map_err(|e| e.to_string()) + } + + /// The object's bytes, or `None` if there is no such object. + pub fn get(&self, name: &str) -> Result>, String> { + use object_store::GetErrorKind; + use tokio::io::AsyncReadExt; + wait(async { + let mut object = match self.store.get(name).await { + Ok(object) => object, + Err(e) if e.kind() == GetErrorKind::NotFound => return Ok(None), + Err(e) => return Err(e.to_string()), + }; + let mut bytes = Vec::new(); + object + .read_to_end(&mut bytes) + .await + .map_err(|e| e.to_string())?; + Ok(Some(bytes)) + }) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn an_ack_subject_gives_its_delivery() { + let short = "$JS.ACK.VEJAS.flow_x.3.17.9.1793984542000000000.0"; + assert_eq!( + parse_ack_subject(short), + Some(Delivery { + delivered: 3, + stream_seq: 17 + }) + ); + let long = "$JS.ACK.dom.ACC.VEJAS.flow_x.2.41.9.1793984542000000000.5.token"; + assert_eq!( + parse_ack_subject(long), + Some(Delivery { + delivered: 2, + stream_seq: 41 + }) + ); + assert_eq!(parse_ack_subject("_INBOX.abc"), None); + assert_eq!(parse_ack_subject("$JS.ACK.too.few"), None); + } + + #[test] + fn the_pull_request_says_what_the_constants_say() { + let request: Value = serde_json::from_slice(PULL_REQUEST).unwrap(); + assert_eq!(request["batch"], PULL_BATCH); + assert_eq!(request["expires"], PULL_EXPIRES_MS * 1_000_000); + assert_eq!(request["no_wait"], true); + } + + #[test] + fn a_url_is_one_server_or_a_list() { + assert_eq!(servers_of("nats://127.0.0.1:4222").unwrap().len(), 1); + assert_eq!(servers_of("127.0.0.1:4222").unwrap().len(), 1); + assert_eq!(servers_of("nats://a:4222, nats://b:4222").unwrap().len(), 2); + assert!(servers_of("http://a:4222").is_err()); + } + + #[test] + fn statuses_end_a_round_or_a_consumer() { + let status = |code: StatusCode, description: &str| async_nats::Message { + subject: "_INBOX.x".into(), + reply: None, + payload: Bytes::new(), + headers: None, + status: Some(code), + description: Some(description.to_string()), + length: 0, + }; + let verdict = |m: async_nats::Message| match classify(&m) { + Pulled::Message => "message", + Pulled::Nothing => "nothing", + Pulled::Over => "over", + Pulled::Gone(_) => "gone", + }; + assert_eq!( + verdict(status(StatusCode::NOT_FOUND, "No Messages")), + "nothing" + ); + assert_eq!( + verdict(status(StatusCode::TIMEOUT, "Request Timeout")), + "over" + ); + assert_eq!( + verdict(status( + StatusCode::REQUEST_TERMINATED, + "Exceeded MaxWaiting" + )), + "over" + ); + assert_eq!( + verdict(status(StatusCode::REQUEST_TERMINATED, "Consumer Deleted")), + "gone" + ); + assert_eq!(verdict(status(StatusCode::NO_RESPONDERS, "")), "gone"); + } + + /// `wait` inside a task of the island would deadlock a worker; tokio refuses. + #[test] + fn blocking_inside_the_runtime_panics_instead_of_deadlocking() { + let task = runtime().spawn(async { wait(async {}) }); + assert!(wait(task).unwrap_err().is_panic()); + } +} + +/// Against a real server: each test starts a `nats-server` of its own, so that +/// it can also kill it. Excluded from CI like every test that needs one. Run: +/// `cargo test bus::live -- --ignored` +#[cfg(test)] +mod live { + use super::*; + use std::path::PathBuf; + use std::process::{Child, Command, Stdio}; + + struct Server { + child: Child, + port: u16, + store: PathBuf, + } + + impl Server { + fn start() -> Server { + Server::start_with(&[]) + } + + fn start_with(args: &[&str]) -> Server { + let port = std::net::TcpListener::bind("127.0.0.1:0") + .unwrap() + .local_addr() + .unwrap() + .port(); + let store = + std::env::temp_dir().join(format!("vejas-bus-{}-{port}", std::process::id())); + let child = Server::spawn(port, &store, args); + Server { child, port, store } + } + + fn spawn(port: u16, store: &PathBuf, args: &[&str]) -> Child { + let mut child = Command::new("nats-server") + .args(["-js", "-a", "127.0.0.1", "-p", &port.to_string()]) + .args(args) + .arg("-sd") + .arg(store) + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .spawn() + .expect("nats-server on PATH"); + for _ in 0..100 { + if std::net::TcpStream::connect(("127.0.0.1", port)).is_ok() { + return child; + } + std::thread::sleep(Duration::from_millis(20)); + } + let _ = child.kill(); + let _ = child.wait(); + panic!("nats-server did not come up"); + } + + fn url(&self) -> String { + format!("nats://127.0.0.1:{}", self.port) + } + + fn restart(&mut self) { + self.child.kill().unwrap(); + self.child.wait().unwrap(); + self.child = Server::spawn(self.port, &self.store, &[]); + } + } + + impl Drop for Server { + fn drop(&mut self) { + let _ = self.child.kill(); + let _ = self.child.wait(); + let _ = std::fs::remove_dir_all(&self.store); + } + } + + fn until(what: &str, done: impl Fn() -> bool) { + let start = Instant::now(); + while !done() { + assert!( + start.elapsed() < Duration::from_secs(15), + "never happened: {what}" + ); + std::thread::sleep(Duration::from_millis(10)); + } + } + + #[test] + #[ignore] + fn the_barrier_comes_back_after_everything_published_before_it() { + let server = Server::start(); + let bus = connect(&server.url()).unwrap(); + let mut sub = bus.subscribe("t.barrier").unwrap(); + let mut batch = bus.batch(); + for i in 0..5000 { + batch.publish("t.barrier", format!("{i}")).unwrap(); + } + batch.confirm().unwrap(); + // The echo left the server after these: they are all here, unwaited. + for i in 0..5000 { + let message = sub.next_timeout(Duration::ZERO).unwrap(); + assert_eq!(&message.data[..], format!("{i}").as_bytes()); + } + } + + #[test] + #[ignore] + fn a_batch_that_spans_a_reconnect_fails_its_barrier() { + let mut server = Server::start(); + let bus = connect(&server.url()).unwrap(); + let link = &bus.0.link; + let epoch = link.epoch(); + bus.publish("t.lost", "published before the crash").unwrap(); + server.restart(); + until("the bus is back", || link.up() && link.epoch() != epoch); + // The echo comes back on the new connection, and proves nothing about + // the old one. + assert!(wait(link.barrier()).is_ok()); + assert!( + wait(link.barrier_since(epoch)).is_err(), + "that publish may be gone" + ); + // The next batch stands on the new connection. + let mut batch = bus.batch(); + batch.publish("t.lost", "published after").unwrap(); + batch.confirm().unwrap(); + } + + #[test] + #[ignore] + fn one_unsendable_emit_fails_alone_and_nothing_held_needs_no_barrier() { + let server = Server::start(); + let bus = connect(&server.url()).unwrap(); + let mut batch = bus.batch(); + batch.confirm().unwrap(); + assert!(batch.publish("two words", "x").is_err()); + assert!( + batch.publish("a\r\nPUB b 0", "x").is_err(), + "no protocol injection" + ); + assert!(batch.publish("", "x").is_err()); + assert!(batch.publish("t.big", vec![0u8; 2 << 20]).is_err()); + batch.publish("t.fine", "x").unwrap(); + batch.confirm().unwrap(); + } + + #[test] + #[ignore] + fn connect_refuses_while_the_connection_is_down_and_recovers() { + let mut server = Server::start(); + let url = server.url(); + let bus = connect(&url).unwrap(); + server.child.kill().unwrap(); + server.child.wait().unwrap(); + until("the bus to notice", || !bus.0.link.up()); + assert!(connect(&url).is_err()); + let mut batch = bus.batch(); + batch.publish("t.x", "x").unwrap(); + assert!(batch.confirm().is_err()); + server.child = Server::spawn(server.port, &server.store, &[]); + until("the bus to reconnect", || bus.0.link.up()); + assert!(connect(&url).is_ok()); + } + + #[test] + #[ignore] + fn a_durable_survives_a_restart_and_keeps_its_place() { + let mut server = Server::start(); + let js = connect(&server.url()).unwrap().jetstream(); + js.add_stream(&StreamConfig { + name: "T".into(), + subjects: vec!["t.>".into()], + ..Default::default() + }) + .unwrap(); + let filter = ["t.in".to_string()]; + js.create_pull_consumer("T", "d", &filter, Duration::ZERO, None) + .unwrap(); + let puller = js.pull_durable("T", "d", Some("t.in")).unwrap(); + assert!( + js.pull_durable("T", "d", Some("t.other")).is_err(), + "bound to its subject" + ); + js.publish("t.in", "one").unwrap(); + let first: Vec = puller.round().unwrap().collect(); + assert_eq!(first.len(), 1); + assert_eq!( + first[0].delivery().unwrap(), + Delivery { + delivered: 1, + stream_seq: 1 + } + ); + first[0].ack().unwrap(); + // Let the server take the ack before it goes down. + wait(connect(&server.url()).unwrap().0.link.barrier()).unwrap(); + server.restart(); + until("the bus is back", || connect(&server.url()).is_ok()); + js.publish("t.in", "two").unwrap(); + until("the next message", || { + puller + .round() + .unwrap() + .next() + .is_some_and(|m| &m.data[..] == b"two") + }); + } + + #[test] + #[ignore] + fn a_round_ends_at_the_pull_expiry_and_a_deleted_consumer_fails_the_next() { + let server = Server::start(); + let js = connect(&server.url()).unwrap().jetstream(); + js.add_stream(&StreamConfig { + name: "T".into(), + subjects: vec!["t.>".into()], + ..Default::default() + }) + .unwrap(); + let filter = ["t.in".to_string()]; + js.create_pull_consumer("T", "d", &filter, Duration::ZERO, None) + .unwrap(); + let puller = js.pull_durable("T", "d", None).unwrap(); + let t0 = Instant::now(); + assert_eq!(puller.round().unwrap().count(), 0); + let idle = t0.elapsed(); + assert!( + idle >= Duration::from_millis(400) && idle < Duration::from_millis(700), + "{idle:?}" + ); + // A deleted consumer is told to the request waiting on it, as a 409. + let waiting = puller.round().unwrap(); + std::thread::sleep(Duration::from_millis(50)); + js.delete_consumer("T", "d").unwrap(); + assert_eq!(waiting.count(), 0); + let error = puller.round().err().expect("the consumer is gone"); + assert!(error.contains("consumer gone"), "{error}"); + } + + #[test] + #[ignore] + fn a_lease_is_compare_and_set() { + let server = Server::start(); + let js = connect(&server.url()).unwrap().jetstream(); + let kv = js + .key_value(&KvConfig { + bucket: "T".into(), + history: 1, + max_age: Duration::from_secs(60), + }) + .unwrap(); + let rev = kv.create("lease", "a").unwrap(); + assert!(kv.create("lease", "b").is_err(), "exactly one wins"); + let renewed = kv.update("lease", "a", rev).unwrap(); + assert!( + kv.update("lease", "a", rev).is_err(), + "a stale revision is fenced" + ); + assert_eq!(kv.get("lease").unwrap().as_deref(), Some(&b"a"[..])); + kv.delete("lease").unwrap(); + assert_eq!(kv.get("lease").unwrap(), None); + assert!( + kv.update("lease", "a", renewed).is_err(), + "a released lease is not renewed" + ); + kv.create("lease", "b").unwrap(); + assert_eq!(kv.keys().unwrap(), vec!["lease".to_string()]); + } + + #[test] + #[ignore] + fn a_watch_opens_with_what_is_there_and_follows() { + let server = Server::start(); + let js = connect(&server.url()).unwrap().jetstream(); + let kv = js + .key_value(&KvConfig { + bucket: "T".into(), + history: 4, + ..Default::default() + }) + .unwrap(); + assert!(kv.keys().unwrap().is_empty()); + kv.put("before", "1").unwrap(); + let mut watch = kv.watch_all().unwrap(); + let wait = Duration::from_secs(2); + assert_eq!(watch.next_timeout(wait).unwrap().as_deref(), Some("before")); + kv.put("after", "2").unwrap(); + assert_eq!(watch.next_timeout(wait).unwrap().as_deref(), Some("after")); + assert_eq!(watch.next_timeout(Duration::from_millis(50)).unwrap(), None); + } + + #[test] + #[ignore] + fn a_watch_goes_on_after_a_restart_or_says_it_ended() { + let mut server = Server::start(); + let url = server.url(); + let js = connect(&url).unwrap().jetstream(); + let kv = js + .key_value(&KvConfig { + bucket: "T".into(), + history: 4, + ..Default::default() + }) + .unwrap(); + let mut watch = kv.watch_all().unwrap(); + server.restart(); + until("the bus is back", || connect(&url).is_ok()); + kv.put("after", "1").unwrap(); + let start = Instant::now(); + loop { + assert!( + start.elapsed() < Duration::from_secs(30), + "the watch went silent" + ); + match watch.next_timeout(Duration::from_millis(500)) { + Ok(Some(key)) => { + assert_eq!(key, "after"); + break; + } + Ok(None) => continue, + // `version_watch` opens it again from here, and reads what is there + Err(_) => watch = kv.watch_all().unwrap(), + } + } + } + + #[test] + #[ignore] + fn credentials_in_the_url_are_used() { + let server = Server::start_with(&["--user", "alice", "--pass", "s3cret"]); + let at = format!("127.0.0.1:{}", server.port); + assert!( + connect(&format!("nats://{at}")).is_err(), + "the server wants credentials" + ); + assert!(connect(&format!("nats://alice:wrong@{at}")).is_err()); + connect(&format!("nats://alice:s3cret@{at}")).unwrap(); + let server = Server::start_with(&["--auth", "t0ken"]); + connect(&format!("nats://t0ken@127.0.0.1:{}", server.port)).unwrap(); + } + + #[test] + #[ignore] + fn an_object_comes_back_whole_and_a_missing_one_is_none() { + let server = Server::start(); + let js = connect(&server.url()).unwrap().jetstream(); + let objects = js.object_store("T").unwrap(); + assert_eq!(objects.get("detect:x").unwrap(), None); + let big: Vec = (0..300_000u32).map(|i| i as u8).collect(); + objects.put("detect:x", &big).unwrap(); + assert_eq!(objects.get("detect:x").unwrap().as_deref(), Some(&big[..])); + objects.put("detect:x", b"smaller").unwrap(); + assert_eq!( + objects.get("detect:x").unwrap().as_deref(), + Some(&b"smaller"[..]) + ); + } +} diff --git a/core/src/cluster.rs b/core/src/cluster.rs index 96ee8dc..16adf7a 100644 --- a/core/src/cluster.rs +++ b/core/src/cluster.rs @@ -6,11 +6,11 @@ //! instance, or N instances produce N× the events. Such a driver acquires a //! **lease** in a JetStream KV bucket before it runs: //! -//! - acquire = `Store::create` (atomic create-if-absent — exactly one wins) -//! - renew = `Store::update(key, id, revision)` (compare-and-set; a paused +//! - acquire = `Kv::create` (atomic create-if-absent — exactly one wins) +//! - renew = `Kv::update(key, id, revision)` (compare-and-set; a paused //! leader that wakes with a stale revision fails and stands down — //! fencing, so two instances never *keep* running the same unit) -//! - release = `Store::delete` on graceful shutdown (instant handoff) +//! - release = `Kv::delete` on graceful shutdown (instant handoff) //! - failover = the bucket's `max_age` ages the value out after a crash, so a //! stand-by's `create` then succeeds (bounded by the TTL) //! @@ -21,6 +21,8 @@ use std::sync::{Arc, OnceLock}; use std::thread; use std::time::Duration; +use crate::bus; + const LEASE_BUCKET: &str = "VEJAS_LEASES"; /// This process's identity, stable for its lifetime. `VEJAS_INSTANCE` overrides @@ -75,22 +77,18 @@ pub fn lease_ttl() -> Duration { /// Open (or create) the shared lease bucket. Its own sibling root, like the DLQ /// and audit streams. Returns None if JetStream KV is unavailable — the caller /// then runs the driver unleased (single-instance behaviour, no worse than today). -pub fn open_lease_store(js: &nats::jetstream::JetStream) -> Option { - let cfg = nats::kv::Config { +pub fn open_lease_store(js: &bus::JetStream) -> Option { + js.key_value(&bus::KvConfig { bucket: LEASE_BUCKET.to_string(), max_age: lease_ttl(), history: 1, - ..Default::default() - }; - js.create_key_value(&cfg) - .or_else(|_| js.key_value(LEASE_BUCKET)) - .ok() + }) } /// Held while this instance owns a singleton's lease. Dropping it stops the /// renewal thread and releases the lease (instant handoff on shutdown). pub struct LeaseGuard { - store: nats::kv::Store, + store: bus::Kv, key: String, stop_renew: Arc, } @@ -112,7 +110,7 @@ impl Drop for LeaseGuard { /// and the supervisor re-enters standby. Returns None if `alive()` goes false /// before the lease is acquired (asked to stop while standing by). pub fn acquire_blocking( - store: &nats::kv::Store, + store: &bus::Kv, key: &str, running: Arc, alive: impl Fn() -> bool, diff --git a/core/src/connectors.rs b/core/src/connectors.rs index 2d2fde7..6f3bcb8 100644 --- a/core/src/connectors.rs +++ b/core/src/connectors.rs @@ -23,6 +23,8 @@ use std::time::{Duration, Instant}; use serde_json::{Map, Value}; +use crate::bus; + // ───────────────────────── config & context ───────────────────────── /// A connector instance's config: the UPPERCASE literals of its manifest. @@ -89,23 +91,14 @@ impl Ctx { fn alive(&self) -> bool { self.running.load(Ordering::SeqCst) } - fn jetstream(&self) -> Result { - Ok(self.jetstream_and_conn()?.0) - } - /// Like `jetstream()`, but also returns the raw `Connection` (a cheap clone — - /// `Connection` is `Arc` inside) so a driver can use `publish_confirmed`, which - /// needs the connection to flush directly past the 5ms flusher floor. - fn jetstream_and_conn( - &self, - ) -> Result<(nats::jetstream::JetStream, nats::Connection), String> { - let nc = nats::connect(&self.nats_url).map_err(|e| e.to_string())?; - let js = nats::jetstream::new(nc.clone()); - let _ = js.add_stream(&nats::jetstream::StreamConfig { + fn jetstream(&self) -> Result { + let js = bus::connect(&self.nats_url)?.jetstream(); + let _ = js.add_stream(&bus::StreamConfig { name: self.stream.clone(), subjects: vec![format!("{}.>", self.subj_root)], ..Default::default() }); - Ok((js, nc)) + Ok(js) } /// Full subject from a config value that may be a bare suffix or a vx.* path. fn subject(&self, raw: &str) -> String { @@ -197,10 +190,7 @@ impl Driver for HttpIn { let max_connections = ctx.config.u64_or("MAX_CONNECTIONS", 512).max(1) as usize; let open = Arc::new(AtomicUsize::new(0)); let mut refused = 0u64; - // nc for publish_confirmed (direct flush past the 5ms flusher floor — - // measured: /ingest p50 5.2ms→0.77ms, no throughput regression at conc=32); - // _js keeps the stream ensured. - let (_js, nc) = ctx.jetstream_and_conn()?; + let js = ctx.jetstream()?; // Optional ALLOW: the subjects this unauthenticated surface may publish to. // Absent = any suffix (compat); set = only these (defence in depth, ADR-0029 // R7 F2 — recommended when the port is reachable by anyone but the operator). @@ -237,7 +227,7 @@ impl Driver for HttpIn { refuse(sock); continue; }; - let nc = nc.clone(); + let js = js.clone(); let root = ctx.subj_root.clone(); let name = ctx.name.clone(); let allow = allow.clone(); @@ -248,7 +238,7 @@ impl Driver for HttpIn { let spawned = thread::Builder::new().name(format!("{name}:conn")).spawn( move || { let _slot = slot; - handle_http_in(&name, &mut sock, &nc, &root, &allow) + handle_http_in(&name, &mut sock, &js, &root, &allow) }, ); if let Err(e) = spawned { @@ -314,7 +304,7 @@ const REQUEST_READ_LIMIT: Duration = Duration::from_secs(30); fn handle_http_in( name: &str, sock: &mut std::net::TcpStream, - nc: &nats::Connection, + js: &bus::JetStream, subj_root: &str, allow: &[String], ) { @@ -394,7 +384,7 @@ fn handle_http_in( ("403 Forbidden", format!("{{\"error\":\"subject {suffix:?} not permitted by ALLOW\"}}")) } else { let subject = format!("{subj_root}.{suffix}"); - match publish_confirmed(nc, &subject, body, Duration::from_secs(5)) { + match js.publish(&subject, body) { Ok(_) => { trace_pub(name, &subject, body); ("202 Accepted", format!("{{\"published\":\"{subject}\"}}")) @@ -544,12 +534,6 @@ impl Driver for HttpPoll { // ───────────────────────── pull rounds ───────────────────────── -/// One bounded pull round: up to 10 messages, and the server resolves the -/// request within PULL_EXPIRES_NS (messages or a 408 timeout status), so the -/// calling loop gets control back and can re-check its stop flag. A plain -/// `fetch()` parks forever on an idle subject: a stopped flow's thread stayed -/// wedged in `recv()`, reported "running", and processed one more message -/// before dying (the zombie-consumer trap caught while rehearsing the demo). /// Consumer ack-wait (redelivery latency): a message not acked within this window /// redelivers (at-least-once, ADR-0002). `VEJAS_ACK_WAIT_SECS` tunes it; unset (or /// ≤0) keeps `Duration::ZERO`, which JetStream reads as its 30s default — the @@ -566,141 +550,11 @@ pub fn ack_wait() -> Duration { .unwrap_or_default() } -/// The idle poll window (`expires`). One outstanding pull covers this whole span, -/// so a message arriving any time inside it is delivered at once — there is NO gap -/// between rounds where a message would sit unpolled. It is therefore two things at -/// once: the idle pull cadence (one pull per window when quiet) AND the worst-case -/// stop-response latency (the loop re-checks its `alive` flag each time a pull -/// returns). 500ms keeps idle chatter to ~2 pulls/s per consumer while leaving stop -/// far inside the transport suite's 3s bound (T5). -const PULL_EXPIRES_MS: u64 = 500; - -pub fn fetch_round(sub: &nats::jetstream::PullSubscription) -> Result, String> { - Ok(fetch_iter_with(sub, None)?.collect()) -} - -/// [`fetch_round`] for a hot loop: the pull request leaves NOW, and the -/// messages come back one by one as the reader thread hands them over. -/// -/// Two things a hot loop needs that a one-off scan does not. -/// -/// **The request leaves now.** A pull request is a publish, and the `nats` -/// 0.25 client only *buffers* a publish: its flusher thread writes the buffer -/// out, and it waits at least 5 ms between two writes (`MIN_FLUSH_BETWEEN` in -/// its `client.rs`, a never-done `TODO(dlc)`). So every round of the consumer -/// loop paid up to 5 ms before the server even saw the request. Measured on -/// the flow loop under a backlog: 64 messages per round, ~7 ms per round, -/// ~9 000 evt/s — with the interpreter itself at 10 µs per event. Flushing -/// the request in the calling thread (`Connection::flush` writes a PING and -/// flushes the writer directly, the same bypass `publish_confirmed` uses for -/// pub-acks) turns that wait into one round-trip: 13 500 evt/s. -/// -/// **Streaming, not collecting.** Collecting the whole batch into a `Vec` -/// before processing meant the reader thread decoded 64 messages while this -/// thread waited, then this thread processed 64 while the reader idled — -/// two costs added instead of overlapped. Iterating as messages arrive lets -/// both run at once. The batch boundary is unchanged: the caller still does -/// its publish-before-ack barrier once the iterator ends. -/// -/// One-off scans (replay hydration, the audit list, the shadow watch) keep -/// [`fetch_round`]: they pull a handful of rounds and a few milliseconds each -/// do not matter there. -pub fn fetch_iter_flushed<'a>( - sub: &'a nats::jetstream::PullSubscription, - nc: &nats::Connection, -) -> Result, String> { - fetch_iter_with(sub, Some(nc)) -} - -/// One pull round: the messages of one request, ending when the request is -/// exhausted (batch filled, or the server's own timeout status). -pub struct FetchRound<'a>(Box> + 'a>); - -impl Iterator for FetchRound<'_> { - type Item = nats::Message; - fn next(&mut self) -> Option { - self.0.next()?.ok() - } -} - -fn fetch_iter_with<'a>( - sub: &'a nats::jetstream::PullSubscription, - flush_on: Option<&nats::Connection>, -) -> Result, String> { - // A continuous long-poll. `no_wait` returns the instant messages are available - // (a single message is delivered at once — no batch-fill latency, and the - // benched high-rate drain still fills 64-batches in one round). When the subject - // is idle the pull is HELD server-side for `expires` and returns empty — so the - // loop paces itself on the server with no client sleep, and crucially no - // coverage gap: previously a 150ms idle sleep sat between rounds with no pull - // outstanding, and a message landing in that window waited it out (measured tail - // ~150ms — the connector-latency bug the MQTT bench surfaced). No pull parks in - // max_waiting: one is in flight at a time and each resolves within `expires`, so - // the anti-zombie invariant holds and stop is re-checked every round. - let expires_ns = (PULL_EXPIRES_MS * 1_000_000) as usize; - let iter = sub - .timeout_fetch( - nats::jetstream::BatchOptions { - batch: 64, - expires: Some(expires_ns), - no_wait: true, - }, - // client-side backstop: comfortably past the server's `expires` so the - // server's own empty/timeout status ends the round, not this timeout. - Duration::from_millis(PULL_EXPIRES_MS + 250), - ) - .map_err(|e| e.to_string())?; - // `timeout_fetch` has buffered the pull request; send it before waiting on it. - if let Some(nc) = flush_on { - nc.flush().map_err(|e| e.to_string())?; - } - Ok(FetchRound(Box::new(iter))) -} - -/// Publish to a JetStream subject and wait for the pub-ack, flushing the write -/// DIRECTLY in this thread rather than via the shared flusher. -/// -/// `nats` 0.25's flusher thread imposes a hard-coded 5ms floor between flushes -/// (`client.rs` `MIN_FLUSH_BETWEEN`, a never-done `TODO(dlc)`). Since a JetStream -/// pub-ack is a request/reply, EVERY sequential `js.publish` is capped at ~200/s by -/// that floor — the ceiling the MQTT loopback bench surfaced (js.publish p50 -/// ≈5.1ms, dead on the tick). `Connection::flush()` does a direct writer flush + -/// PING/PONG in the calling thread, bypassing the floor, which lifts sequential -/// publish to ~1-2k/s. This reimplements `request_with_headers_or_timeout` (nats -/// `lib.rs`) verbatim — inbox, buffered publish-with-reply, the **no-responders -/// guard** — and only adds the direct flush, so the at-least-once contract is -/// unchanged: a returned Ok still means the message was persisted (a real pub-ack -/// with a `stream`/`seq`), never a mis-read of a 503 or an error ack. -pub fn publish_confirmed( - nc: &nats::Connection, - subject: &str, - payload: &[u8], - wait: Duration, -) -> Result<(), String> { - let reply = nc.new_inbox(); - let sub = nc.subscribe(&reply).map_err(|e| e.to_string())?; - nc.publish_request(subject, &reply, payload) - .map_err(|e| e.to_string())?; - nc.flush().map_err(|e| e.to_string())?; // direct flush — bypass the 5ms floor - let msg = sub - .next_timeout(wait) - .map_err(|_| "timed out waiting for JetStream pub-ack".to_string())?; - if msg.is_no_responders() { - return Err("no JetStream responder for the subject (stream missing?)".into()); - } - // ApiResponse: an "error" key = rejected; a "stream" key = persisted. - let ack: serde_json::Value = - serde_json::from_slice(&msg.data).map_err(|e| format!("bad pub-ack: {e}"))?; - if let Some(err) = ack.get("error") { - return Err(format!("JetStream rejected the publish: {err}")); - } - if ack.get("stream").is_none() { - return Err(format!( - "unexpected pub-ack (no stream): {}", - String::from_utf8_lossy(&msg.data) - )); - } - Ok(()) +/// One whole pull round, collected: for one-off scans (replay hydration, the +/// audit list, the shadow watch). A hot loop iterates `Puller::round` instead, +/// so that it works on messages as they arrive. +pub fn fetch_round(sub: &bus::Puller) -> Result, String> { + Ok(sub.round()?.collect()) } /// Hydrate up to `n` recent REAL events for `subject` from the persisted stream, @@ -714,16 +568,13 @@ pub fn publish_confirmed( /// sequence rather than from the beginning: at most `window` messages are /// scanned, of which the last `n` matching `subject` are kept. pub fn hydrate_recent( - js: &nats::jetstream::JetStream, + js: &bus::JetStream, stream: &str, subject: &str, n: usize, ) -> Result, String> { let n = n.clamp(1, 5000); - let last = js - .stream_info(stream) - .map(|si| si.state.last_seq) - .unwrap_or(0); + let last = js.stream_state(stream).map(|state| state.last_seq).unwrap_or(0); if last == 0 { return Ok(Vec::new()); // empty stream: nothing persisted yet } @@ -731,24 +582,16 @@ pub fn hydrate_recent( // but keep it bounded (an operator action, not the hot path). let window = (n as u64).saturating_mul(8).max(256).min(200_000); let start = last.saturating_sub(window).max(1); - let cfg = nats::jetstream::ConsumerConfig { - deliver_policy: nats::jetstream::DeliverPolicy::ByStartSeq, - opt_start_seq: Some(start), - // Explicit but never acked: the consumer is ephemeral and discarded, so - // un-acked messages simply expire with it — nothing redelivers to the - // flow, nothing is consumed off the stream. - ack_policy: nats::jetstream::AckPolicy::Explicit, - filter_subject: subject.to_string(), + // Never acked: the consumer is ephemeral and discarded, so un-acked messages + // simply expire with it — nothing redelivers to the flow, nothing is consumed + // off the stream. + let cfg = bus::EphemeralConfig { + filter: subject, + start_seq: Some(start), // auto-reap the ephemeral consumer shortly after we stop pulling inactive_threshold: Duration::from_secs(30), - ..Default::default() }; - let sub = js - .pull_subscribe_with_options( - subject, - &nats::jetstream::PullSubscribeOptions::new().consumer_config(cfg), - ) - .map_err(|e| e.to_string())?; + let sub = js.pull_ephemeral(stream, &cfg)?; let mut ring: std::collections::VecDeque<(u64, String, serde_json::Value)> = std::collections::VecDeque::with_capacity(n); let mut scanned = 0u64; @@ -762,7 +605,7 @@ pub fn hydrate_recent( if let Ok(v) = serde_json::from_slice::(&m.data) { // the stream sequence is the intrinsic join key for time-travel / // canary diffs (ADR-0021) — no per-flow key to invent. - let seq = m.jetstream_message_info().map(|i| i.stream_seq).unwrap_or(0); + let seq = m.delivery().map(|i| i.stream_seq).unwrap_or(0); ring.push_back((seq, m.subject.clone(), v)); while ring.len() > n { ring.pop_front(); @@ -1010,7 +853,7 @@ impl Driver for OAuthPoll { e })?; let publish_page = - |parsed: &Value, logical: &str, js: &nats::jetstream::JetStream| -> Result<(), String> { + |parsed: &Value, logical: &str, js: &bus::JetStream| -> Result<(), String> { let envelope = serde_json::json!({ "endpoint": logical, "fetched_at": iso8601_utc(crate::now_secs()), @@ -1181,70 +1024,6 @@ pub fn subject_matches(pattern: &str, subject: &str) -> bool { } } -/// Create (or verify) a durable pull consumer that delivers every one of -/// `filters` in stream order, from the beginning or, with `start_seq`, from -/// that stream sequence on. The sync `nats` client knows one filter subject -/// per consumer; the JetStream API knows several since 2.10, so the request is -/// sent raw. Idempotent: the same request on an existing consumer is accepted, -/// and 2.10 updates the filters of an existing one in place (its ack floor -/// stays). The reply's filters must be exactly the ones asked for — a server -/// that ignored `filter_subjects` would deliver the whole stream. -pub fn create_pull_consumer( - nc: &nats::Connection, - stream: &str, - durable: &str, - filters: &[String], - ack_wait: Duration, - start_seq: Option, -) -> Result<(), String> { - let mut config = serde_json::json!({ - "durable_name": durable, - "ack_policy": "explicit", - "ack_wait": ack_wait.as_nanos() as u64, - "deliver_policy": "all", - "replay_policy": "instant", - }); - // Resuming after a snapshot: start at the sequence after it. The server - // cannot change an existing consumer's deliver policy, so the caller - // deletes the durable first. - if let Some(seq) = start_seq { - config["deliver_policy"] = Value::from("by_start_sequence"); - config["opt_start_seq"] = Value::from(seq); - } - match filters { - [one] => config["filter_subject"] = Value::String(one.clone()), - many => config["filter_subjects"] = Value::from(many.to_vec()), - } - let body = serde_json::json!({ "stream_name": stream, "config": config }).to_string(); - let reply = nc - .request(&format!("$JS.API.CONSUMER.DURABLE.CREATE.{stream}.{durable}"), body) - .map_err(|e| format!("consumer {durable}: {e}"))?; - let v: Value = serde_json::from_slice(&reply.data) - .map_err(|e| format!("consumer {durable}: unreadable reply: {e}"))?; - if let Some(err) = v.get("error") { - let what = err.get("description").and_then(Value::as_str).unwrap_or("error"); - return Err(format!("consumer {durable}: {what}")); - } - let mut kept: Vec = match v["config"].get("filter_subjects").and_then(Value::as_array) { - Some(a) => a.iter().filter_map(|x| x.as_str().map(str::to_string)).collect(), - None => v["config"] - .get("filter_subject") - .and_then(Value::as_str) - .map(|x| vec![x.to_string()]) - .unwrap_or_default(), - }; - let mut asked = filters.to_vec(); - kept.sort(); - asked.sort(); - if kept != asked { - return Err(format!( - "consumer {durable}: the server kept the filters {kept:?}, not {asked:?} \ - (several `.from()` subjects need NATS 2.10 or newer)" - )); - } - Ok(()) -} - // ───────────────────────── detect snapshots ───────────────────────── pub const DETECT_STATE_BUCKET: &str = "VEJAS_DETECT_STATE"; @@ -1262,21 +1041,13 @@ pub struct Snapshot { /// The object store the detect units keep their snapshots in: one object per /// unit, the latest only. Create-or-open, like every bucket in this repo. -pub fn open_state_store( - js: &nats::jetstream::JetStream, -) -> Option { - let cfg = nats::object_store::Config { - bucket: DETECT_STATE_BUCKET.to_string(), - ..Default::default() - }; - js.create_object_store(&cfg) - .or_else(|_| js.object_store(DETECT_STATE_BUCKET)) - .ok() +pub fn open_state_store(js: &bus::JetStream) -> Option { + js.object_store(DETECT_STATE_BUCKET) } /// Store a snapshot: one JSON header line, then the engine's bytes as they are. pub fn snapshot_put( - store: &nats::object_store::ObjectStore, + store: &bus::Objects, unit: &str, snap: &Snapshot, ) -> Result<(), String> { @@ -1290,34 +1061,14 @@ pub fn snapshot_put( .into_bytes(); bytes.push(b'\n'); bytes.extend_from_slice(&snap.state); - store - .put(unit, &mut bytes.as_slice()) - .map(|_| ()) - .map_err(|e| format!("snapshot put: {e}")) + store.put(unit, &bytes).map_err(|e| format!("snapshot put: {e}")) } /// The unit's snapshot, if it has one. -pub fn snapshot_get( - store: &nats::object_store::ObjectStore, - unit: &str, -) -> Result, String> { - let mut object = match store.get(unit) { - Ok(o) => o, - Err(e) => { - let text = e.to_string(); - if e.kind() == std::io::ErrorKind::NotFound - || text.contains("not found") - || text.contains("no message") - { - return Ok(None); - } - return Err(format!("snapshot get: {e}")); - } +pub fn snapshot_get(store: &bus::Objects, unit: &str) -> Result, String> { + let Some(bytes) = store.get(unit).map_err(|e| format!("snapshot get: {e}"))? else { + return Ok(None); }; - let mut bytes = Vec::new(); - object - .read_to_end(&mut bytes) - .map_err(|e| format!("snapshot read: {e}"))?; let nl = bytes .iter() .position(|b| *b == b'\n') @@ -1374,12 +1125,12 @@ pub const DLQ_MAX_MSGS: i64 = 100_000; /// Ensure the dead-letter stream exists (idempotent). Bounded + discard-oldest: /// a full DLQ is itself an operator signal, never a silent unbounded growth. -pub fn ensure_dlq_stream(js: &nats::jetstream::JetStream) { - let _ = js.add_stream(&nats::jetstream::StreamConfig { +pub fn ensure_dlq_stream(js: &bus::JetStream) { + let _ = js.add_stream(&bus::StreamConfig { name: DLQ_STREAM.to_string(), subjects: vec![format!("{DLQ_ROOT}.>")], max_msgs: DLQ_MAX_MSGS, - discard: nats::jetstream::DiscardPolicy::Old, + discard: bus::Discard::Old, ..Default::default() }); } @@ -1402,7 +1153,7 @@ fn epoch_secs() -> u64 { /// JetStream has stored it — the caller must ack the original only on Ok. #[allow(clippy::too_many_arguments)] pub fn to_dlq( - js: &nats::jetstream::JetStream, + js: &bus::JetStream, unit: &str, subject: &str, attempts: i64, @@ -1443,18 +1194,15 @@ pub fn to_dlq( pub const OFFSET_BUCKET: &str = "VEJAS_OFFSETS"; -pub fn open_offset_store(js: &nats::jetstream::JetStream) -> Option { - let cfg = nats::kv::Config { +pub fn open_offset_store(js: &bus::JetStream) -> Option { + js.key_value(&bus::KvConfig { bucket: OFFSET_BUCKET.to_string(), history: 1, ..Default::default() - }; - js.create_key_value(&cfg) - .or_else(|_| js.key_value(OFFSET_BUCKET)) - .ok() + }) } -pub fn offset_get(store: &nats::kv::Store, key: &str) -> Option { +pub fn offset_get(store: &bus::Kv, key: &str) -> Option { store .get(key) .ok() @@ -1463,7 +1211,7 @@ pub fn offset_get(store: &nats::kv::Store, key: &str) -> Option { .filter(|s| !s.is_empty()) } -pub fn offset_put(store: &nats::kv::Store, key: &str, value: &str) { +pub fn offset_put(store: &bus::Kv, key: &str, value: &str) { let _ = store.put(key, value.as_bytes()); } @@ -1477,12 +1225,12 @@ pub const AUDIT_ROOT: &str = "vxaudit"; // `max_age` on the hot stream can touch it. pub const AUDIT_MAX_MSGS: i64 = 1_000_000; -pub fn ensure_audit_stream(js: &nats::jetstream::JetStream) { - let _ = js.add_stream(&nats::jetstream::StreamConfig { +pub fn ensure_audit_stream(js: &bus::JetStream) { + let _ = js.add_stream(&bus::StreamConfig { name: AUDIT_STREAM.to_string(), subjects: vec![format!("{AUDIT_ROOT}.>")], max_msgs: AUDIT_MAX_MSGS, - discard: nats::jetstream::DiscardPolicy::Old, + discard: bus::Discard::Old, ..Default::default() }); } @@ -1492,7 +1240,7 @@ pub fn ensure_audit_stream(js: &nats::jetstream::JetStream) { /// caller can trust the trail; the live-promote path logs but does not fail the /// promote if this errors (git remains the trail for committed changes). pub fn to_audit( - js: &nats::jetstream::JetStream, + js: &bus::JetStream, unit: &str, record: &serde_json::Value, ) -> Result<(), String> { @@ -1504,7 +1252,7 @@ pub fn to_audit( /// Recent audit records for a unit, oldest last (read-only, ephemeral consumer). pub fn audit_recent( - js: &nats::jetstream::JetStream, + js: &bus::JetStream, unit: &str, n: usize, ) -> Result, String> { @@ -1525,29 +1273,17 @@ fn run_sink( subject: &str, handler: impl Fn(&[u8]) -> Result, String>, ) -> Result<(), String> { - let (js, nc) = ctx.jetstream_and_conn()?; + let js = ctx.jetstream()?; let durable = ctx.name.replace([':', '.', '-'], "_"); - let _ = js.add_consumer( - &ctx.stream, - nats::jetstream::ConsumerConfig { - durable_name: Some(durable.clone()), - filter_subject: subject.to_string(), - ack_wait: ack_wait(), - ..Default::default() - }, - ); - let sub = js - .pull_subscribe_with_options( - subject, - &nats::jetstream::PullSubscribeOptions::new().durable_name(durable), - ) - .map_err(|e| e.to_string())?; + let filter = [subject.to_string()]; + let _ = js.create_pull_consumer(&ctx.stream, &durable, &filter, ack_wait(), None); + let sub = js.pull_durable(&ctx.stream, &durable, Some(subject))?; eprintln!("[{}] consuming {subject}", ctx.name); loop { if !ctx.alive() { return Ok(()); } - for msg in fetch_iter_flushed(&sub, &nc)? { + for msg in sub.round()? { if !ctx.alive() { // stopped mid-batch: leave the message un-acked, it redelivers return Ok(()); @@ -1575,7 +1311,7 @@ fn run_sink( }); // poison guard: give up after MAX_DELIVERIES attempts let delivered = - msg.jetstream_message_info().map(|i| i.delivered).unwrap_or(1); + msg.delivery().map(|i| i.delivered).unwrap_or(1); if delivered >= crate::MAX_DELIVERIES { // Poison: park it in the DLQ, then ack — publish before ack // so it is never lost (ADR-0015). On DLQ failure, nak. @@ -1597,7 +1333,7 @@ fn run_sink( Some(format!("{e} — DLQ publish failed: {de}")), vec![], preview, None, None, ); - let _ = msg.ack_kind(nats::jetstream::AckKind::Nak); + let _ = msg.nak(); } } } else { @@ -1605,7 +1341,7 @@ fn run_sink( crate::record_trace_full( &ctx.name, subject, false, Some(e), vec![], preview, None, None, ); - let _ = msg.ack_kind(nats::jetstream::AckKind::Nak); + let _ = msg.nak(); } } } @@ -1806,9 +1542,7 @@ impl Driver for ExecStreamSource { let subject = ctx.subject(&ctx.config.str("SUBJECT").ok_or("SUBJECT required")?); let env = ctx.config.env_vars(); let restart = ctx.config.u64_or("RESTART_SECS", 2).max(1); - // js for the offset KV store; nc for publish_confirmed (this streams stdout - // line-by-line sequentially — the exact ~200/s flusher-floor case). - let (js, nc) = ctx.jetstream_and_conn()?; + let js = ctx.jetstream()?; // Optional offset resume: when OFFSET_KV is set, hand the child the last // committed offset as $OFFSET at (re)start (e.g. `kcat -o $OFFSET`) and // commit each record's OFFSET_FIELD after publishing it. @@ -1817,8 +1551,8 @@ impl Driver for ExecStreamSource { let offset_start = ctx.config.str("OFFSET_START").unwrap_or_else(|| "end".into()); let offset_store = offset_kv.as_ref().and_then(|_| open_offset_store(&js)); // Offset commit CADENCE (keeps the KV put out of the per-record hot path). - // A KV put is itself a throttled request/reply, so committing per record - // re-caps the stream at ~200/s even with the publish path lifted. Commit + // A KV put is itself a request/reply, and committing per record would + // add a round-trip to every record. Commit // every OFFSET_COMMIT_MS (default 100ms) and once more when the child // drains/exits: publish-before-commit still holds (an offset is committed // only for a record already on the bus), the crash duplicate window just @@ -1890,7 +1624,7 @@ impl Driver for ExecStreamSource { continue; } if let Ok(rec) = serde_json::from_str::(line) { - match publish_confirmed(&nc, &subject, line.as_bytes(), Duration::from_secs(5)) { + match js.publish(&subject, line) { Ok(_) => { trace_pub(&ctx.name, &subject, line.as_bytes()); // publish-before-commit: advance the offset in @@ -2049,9 +1783,7 @@ impl Driver for MqttIn { let client_id = mqtt_client_id(ctx); let user = ctx.config.str("USERNAME"); let pass = ctx.config.str("PASSWORD"); - // nc for publish_confirmed (direct flush past the 5ms flusher floor — the - // MQTT-loopback ceiling); _js keeps the stream ensured. - let (_js, nc) = ctx.jetstream_and_conn()?; + let js = ctx.jetstream()?; while ctx.alive() { let attempt = (|| -> Result<(), String> { let mut client = mqtt_connect( @@ -2066,7 +1798,7 @@ impl Driver for MqttIn { // publish-before-ack: bus first (await JetStream pub-ack), // THEN PUBACK — a crash before the bus write re-delivers // from the broker (ADR-0025). - publish_confirmed(&nc, &subject, &payload, Duration::from_secs(5)) + js.publish(&subject, &payload) .map_err(|e| format!("bus publish: {e}"))?; trace_pub(&ctx.name, &subject, &payload); if pq > 0 { @@ -2175,7 +1907,7 @@ impl Driver for ExecRpc { )); } let env = ctx.config.env_vars(); - let nc = nats::connect(&ctx.nats_url).map_err(|e| e.to_string())?; + let nc = bus::connect(&ctx.nats_url)?; let mut child = Command::new("sh") .arg("-c") @@ -2197,9 +1929,7 @@ impl Driver for ExecRpc { // Queue group (not a plain subscribe) so N clustered instances share the // requests — exactly one answers each, load-balanced (ADR-0020). Each // instance keeps its own child process (e.g. its own SAP logon). - let sub = nc - .queue_subscribe(&subject, "vejas") - .map_err(|e| e.to_string())?; + let mut sub = nc.queue_subscribe(&subject, "vejas")?; loop { if !ctx.alive() { let _ = child.kill(); @@ -2208,7 +1938,7 @@ impl Driver for ExecRpc { match sub.next_timeout(Duration::from_millis(300)) { Ok(msg) => { // One request line in, one reply line out (serialized). - let mut req = msg.data.clone(); + let mut req = msg.data.to_vec(); req.push(b'\n'); if stdin.write_all(&req).is_err() { return Err("child stdin closed".into()); @@ -2486,51 +2216,40 @@ mod tests { assert!(Config::default().headers().is_empty()); } - /// Verifies `publish_confirmed` clears the nats flusher throttle AND keeps the - /// at-least-once contract: messages actually persist, and a subject with no - /// stream errors (never a false Ok). Needs a live NATS on 127.0.0.1:4222; - /// excluded from CI. Run: - /// `cargo test publish_confirmed_beats_throttle -- --ignored --nocapture` + /// Verifies a confirmed publish keeps the at-least-once contract: messages + /// actually persist, and a subject with no stream errors (never a false Ok). + /// Needs a live NATS on 127.0.0.1:4222; excluded from CI. Run: + /// `cargo test confirmed_publish -- --ignored --nocapture` #[test] #[ignore] - fn publish_confirmed_beats_throttle_and_persists() { + fn confirmed_publish_persists_or_errors() { let url = std::env::var("NATS_URL").unwrap_or_else(|_| "127.0.0.1:4222".into()); - let nc = nats::connect(&url).expect("connect"); - let js = nats::jetstream::new(nc.clone()); + let js = bus::connect(&url).expect("connect").jetstream(); let pid = std::process::id(); let stream = format!("PBFAST_{pid}"); let subject = format!("pbfast.{pid}.in"); let _ = js.delete_stream(&stream); - js.add_stream(nats::jetstream::StreamConfig { + js.add_stream(&bus::StreamConfig { name: stream.clone(), subjects: vec![format!("pbfast.{pid}.>")], ..Default::default() }) .expect("add_stream"); - const N: usize = 200; - // (A) baseline: throttled js.publish + const N: usize = 400; let t = Instant::now(); for _ in 0..N { js.publish(&subject, b"x").expect("js.publish"); } - let base = N as f64 / t.elapsed().as_secs_f64(); - // (B) direct-flush publish_confirmed - let t = Instant::now(); - for _ in 0..N { - publish_confirmed(&nc, &subject, b"x", Duration::from_secs(5)).expect("confirmed"); - } - let fast = N as f64 / t.elapsed().as_secs_f64(); - eprintln!("[pbfast] js.publish={base:.0}/s publish_confirmed={fast:.0}/s ({:.1}x)", fast / base); - assert!(fast > base * 2.0, "direct flush should clear the 5ms floor (got {fast:.0} vs {base:.0}/s)"); + eprintln!("[pbfast] js.publish={:.0}/s", N as f64 / t.elapsed().as_secs_f64()); - // (C) durability: all 2N messages actually landed on the stream - let msgs = js.stream_info(&stream).expect("info").state.messages; - assert!(msgs >= (2 * N) as u64, "every confirmed publish persisted (have {msgs})"); + // durability: all N messages actually landed on the stream + let msgs = js.stream_state(&stream).expect("info").messages; + assert!(msgs >= N as u64, "every confirmed publish persisted (have {msgs})"); - // (D) no-responders guard: a subject with no stream must ERROR, not false-Ok + // no-responders guard: a subject with no stream must ERROR, not false-Ok let orphan = format!("orphan.{pid}.no.stream"); - let r = publish_confirmed(&nc, &orphan, b"x", Duration::from_secs(1)); + let r = js.publish(&orphan, b"x"); assert!(r.is_err(), "publish to a stream-less subject must error, got {r:?}"); let _ = js.delete_stream(&stream); @@ -2544,33 +2263,22 @@ mod tests { #[ignore] fn fetch_round_latency_probe() { let url = std::env::var("NATS_URL").unwrap_or_else(|_| "127.0.0.1:4222".into()); - let nc = nats::connect(&url).expect("connect nats"); - let js = nats::jetstream::new(nc.clone()); + let nc = bus::connect(&url).expect("connect nats"); + let js = nc.jetstream(); let pid = std::process::id(); let stream = format!("PROBE_{pid}"); let subject = format!("probe.{pid}.in"); let _ = js.delete_stream(&stream); - js.add_stream(nats::jetstream::StreamConfig { + js.add_stream(&bus::StreamConfig { name: stream.clone(), subjects: vec![format!("probe.{pid}.>")], ..Default::default() }) .expect("add_stream"); let durable = format!("probe_dur_{pid}"); - let _ = js.add_consumer( - &stream, - nats::jetstream::ConsumerConfig { - durable_name: Some(durable.clone()), - filter_subject: subject.clone(), - ..Default::default() - }, - ); - let sub = js - .pull_subscribe_with_options( - &subject, - &nats::jetstream::PullSubscribeOptions::new().durable_name(durable), - ) - .expect("pull_subscribe"); + let filter = [subject.clone()]; + js.create_pull_consumer(&stream, &durable, &filter, Duration::ZERO, None).expect("consumer"); + let sub = js.pull_durable(&stream, &durable, Some(&subject)).expect("pull_subscribe"); // (A) idle: how long does an empty round block? let mut empties = Vec::new(); @@ -2583,9 +2291,8 @@ mod tests { eprintln!("[probe] empty-round ms: {empties:?}"); // (B) publish->receipt: a message lands at a SWEPT offset into an idle - // stretch (20..~300ms) so samples hit every phase of the poll cycle — most - // importantly the coverage GAP where no pull is outstanding (the current - // 150ms idle sleep). Detection latency = receipt - the publish offset. + // stretch (20..~300ms) so samples hit every phase of the poll cycle. + // Detection latency = receipt - the publish offset. let mut lat = Vec::new(); for i in 0..30u32 { let delay_ms = 20 + (i as u64 * 23) % 280; // sweep across the ~252ms cycle @@ -2593,8 +2300,7 @@ mod tests { let t0 = Instant::now(); let pubber = std::thread::spawn(move || { std::thread::sleep(Duration::from_millis(delay_ms)); - ncp.publish(&subj, payload.as_bytes()).expect("publish"); - ncp.flush().ok(); + ncp.publish(&subj, payload).expect("publish"); }); let detected = loop { let msgs = fetch_round(&sub).expect("fetch"); diff --git a/core/src/control.rs b/core/src/control.rs index 65f724e..321e7d9 100644 --- a/core/src/control.rs +++ b/core/src/control.rs @@ -53,8 +53,8 @@ fn supervise(registry: Registry, root: PathBuf, tenant: String) { return; } let attempt = (|| -> Result<(), String> { - let nc = nats::connect(&url).map_err(|e| e.to_string())?; - let sub = nc.subscribe(&cmd_subj).map_err(|e| e.to_string())?; + let nc = crate::bus::connect(&url)?; + let mut sub = nc.subscribe(&cmd_subj)?; eprintln!("[control] tenant {tenant}: serving {cmd_subj}, status every {}s", status_every.as_secs()); // first status leaves immediately: the console sees us on connect let mut last_status = Instant::now() - status_every; @@ -90,9 +90,7 @@ fn supervise(registry: Registry, root: PathBuf, tenant: String) { })) .unwrap_or_default(), ); - if msg.reply.is_some() { - let _ = msg.respond(serde_json::to_vec(&reply).unwrap_or_default()); - } + let _ = msg.respond(serde_json::to_vec(&reply).unwrap_or_default()); } Err(_) => {} // timeout: loop to re-check RUNNING / status schedule } diff --git a/core/src/main.rs b/core/src/main.rs index b056160..7049e58 100644 --- a/core/src/main.rs +++ b/core/src/main.rs @@ -24,6 +24,7 @@ // POST /flows/new agent CLI writes a VejasScript flow // POST /reload rescan; restart changed files (mtime) +mod bus; mod cluster; mod connectors; mod control; @@ -234,8 +235,7 @@ fn replay_literal( let hydrated = before.source.as_deref().and_then(|subject| { let url = env::var("NATS_URL").unwrap_or_else(|_| "nats://127.0.0.1:4222".into()); let stream = env::var("VEJAS_STREAM").unwrap_or_else(|_| "VEJAS".into()); - let nc = nats::connect(&url).ok()?; - let js = nats::jetstream::new(nc); + let js = bus::connect(&url).ok()?.jetstream(); connectors::hydrate_recent(&js, &stream, subject, n).ok() }); match hydrated { @@ -313,8 +313,7 @@ fn time_travel(root: &Path, file: &str, candidate_src: &str, n: usize) -> Result let pkg = pkg_of_path(&path); let url = env::var("NATS_URL").unwrap_or_else(|_| "nats://127.0.0.1:4222".into()); let stream = env::var("VEJAS_STREAM").unwrap_or_else(|_| "VEJAS".into()); - let nc = nats::connect(&url).map_err(|e| format!("NATS unreachable: {e}"))?; - let js = nats::jetstream::new(nc); + let js = bus::connect(&url).map_err(|e| format!("NATS unreachable: {e}"))?.jetstream(); // diff against the CURRENT EFFECTIVE version (a live overlay if one is valid) let store = versions::open_store(&js); let (current_src, _) = versions::resolve_source(store.as_ref(), &unit, &baseline, now_secs()); @@ -401,8 +400,7 @@ fn canary_start(root: &Path, file: &str, candidate_src: &str) -> Result Result s, Err(_) => return, }; @@ -476,7 +471,7 @@ fn canary_start(root: &Path, file: &str, candidate_src: &str) -> Result(&m.data) else { continue; }; - let seq = m.jetstream_message_info().map(|i| i.stream_seq).unwrap_or(0); + let seq = m.delivery().map(|i| i.stream_seq).unwrap_or(0); let b = shadow_emits(¤t, &event, &root, &pkg); let a = shadow_emits(&candidate, &event, &root, &pkg); let ch = b != a; @@ -551,8 +546,7 @@ fn promote_version( let baseline = fs::read_to_string(&path).map_err(|e| e.to_string())?; let unit = flow_proc_name(&path); let url = env::var("NATS_URL").unwrap_or_else(|_| "nats://127.0.0.1:4222".into()); - let nc = nats::connect(&url).map_err(|e| format!("NATS unreachable: {e}"))?; - let js = nats::jetstream::new(nc); + let js = bus::connect(&url).map_err(|e| format!("NATS unreachable: {e}"))?.jetstream(); let store = versions::open_store(&js).ok_or("version store unavailable")?; // current effective source: a prior overlay if valid, else the baseline let (current, _) = versions::resolve_source(Some(&store), &unit, &baseline, now_secs()); @@ -571,7 +565,7 @@ fn promote_version( /// The current effective-source hash of a flow file — the baseline a proposal is /// pinned to (the overlay-or-baseline the evidence saw, ADR-0021/0024). fn effective_source_hash( - js: &nats::jetstream::JetStream, + js: &bus::JetStream, root: &Path, file: &str, ) -> Result { @@ -585,7 +579,7 @@ fn effective_source_hash( /// Emit a proposal transition on the bus so a notification flow can route it /// (Slack/PagerDuty/email) — the platform dogfoods its own connectors (ADR-0024). -fn emit_proposal_event(js: &nats::jetstream::JetStream, transition: &str, proposal: &Value) { +fn emit_proposal_event(js: &bus::JetStream, transition: &str, proposal: &Value) { let subj_root = env::var("VEJAS_SUBJECT_ROOT").unwrap_or_else(|_| "vx".into()); let subject = format!("{subj_root}.proposals.events"); let ev = json!({"transition": transition, "ts": now_secs(), "proposal": proposal}); @@ -690,8 +684,7 @@ fn submit_proposal( } let file = payload["file"].as_str().ok_or("payload.file required")?.to_string(); let url = env::var("NATS_URL").unwrap_or_else(|_| "nats://127.0.0.1:4222".into()); - let nc = nats::connect(&url).map_err(|e| format!("NATS unreachable: {e}"))?; - let js = nats::jetstream::new(nc); + let js = bus::connect(&url).map_err(|e| format!("NATS unreachable: {e}"))?.jetstream(); let baseline = effective_source_hash(&js, root, &file)?; let store = proposals::open_store(&js).ok_or("proposal store unavailable")?; let id = proposals::new_id(now_secs()); @@ -705,8 +698,7 @@ fn submit_proposal( /// never shows an approvable change the evidence never saw (ADR-0024). fn reconcile_and_list(root: &Path) -> Result, String> { let url = env::var("NATS_URL").unwrap_or_else(|_| "nats://127.0.0.1:4222".into()); - let nc = nats::connect(&url).map_err(|e| format!("NATS unreachable: {e}"))?; - let js = nats::jetstream::new(nc); + let js = bus::connect(&url).map_err(|e| format!("NATS unreachable: {e}"))?.jetstream(); let store = proposals::open_store(&js).ok_or("proposal store unavailable")?; let mut out = Vec::new(); for p in proposals::list(&store) { @@ -769,8 +761,7 @@ fn apply_approved( let baseline = fs::read_to_string(&path).map_err(|e| e.to_string())?; let unit = flow_proc_name(&path); let url = env::var("NATS_URL").unwrap_or_else(|_| "nats://127.0.0.1:4222".into()); - let nc = nats::connect(&url).map_err(|e| e.to_string())?; - let js = nats::jetstream::new(nc); + let js = bus::connect(&url)?.jetstream(); if cluster::clustered() { let store = versions::open_store(&js).ok_or("version store unavailable")?; let parent = versions::hash_content(&baseline); @@ -799,8 +790,7 @@ fn decide_proposal( decided_by: &str, ) -> Result { let url = env::var("NATS_URL").unwrap_or_else(|_| "nats://127.0.0.1:4222".into()); - let nc = nats::connect(&url).map_err(|e| format!("NATS unreachable: {e}"))?; - let js = nats::jetstream::new(nc); + let js = bus::connect(&url).map_err(|e| format!("NATS unreachable: {e}"))?.jetstream(); let store = proposals::open_store(&js).ok_or("proposal store unavailable")?; let proposal = proposals::get(&store, id).ok_or("proposal not found")?; if proposal["status"] != "pending" { @@ -911,14 +901,14 @@ fn record_promote( let Some(path) = guard_path(root, file) else { return }; let unit = flow_proc_name(&path); let url = env::var("NATS_URL").unwrap_or_else(|_| "nats://127.0.0.1:4222".into()); - let Ok(nc) = nats::connect(&url) else { + let Ok(bus) = bus::connect(&url) else { eprintln!( "[vejas] audit: NATS unreachable — live promote of {name} not recorded \ (git remains the trail for committed changes)" ); return; }; - let js = nats::jetstream::new(nc); + let js = bus.jetstream(); let record = json!({ "ts": now_secs(), "actor": actor, "unit": unit, "file": file, "name": name, "key": key, @@ -944,8 +934,7 @@ fn rollback_literal( let path = guard_path(root, file).ok_or("path not allowed")?; let unit = flow_proc_name(&path); let url = env::var("NATS_URL").unwrap_or_else(|_| "nats://127.0.0.1:4222".into()); - let nc = nats::connect(&url).map_err(|e| format!("NATS unreachable: {e}"))?; - let js = nats::jetstream::new(nc); + let js = bus::connect(&url).map_err(|e| format!("NATS unreachable: {e}"))?.jetstream(); let records = connectors::audit_recent(&js, &unit, 500)?; // newest first: the most recent promote of exactly this literal let target = records @@ -1116,7 +1105,7 @@ fn supervise_vjs(handle: Arc, root: PathBuf) { // file iff it forked from exactly this baseline; a stale overlay // (baseline advanced / flow git-deleted) is evicted here — git wins. let src = { - let vconn = nats::connect(&url).ok().map(nats::jetstream::new); + let vconn = bus::connect(&url).ok().map(|bus| bus.jetstream()); let vstore = vconn.as_ref().and_then(versions::open_store); let (s, outcome) = versions::resolve_source( vstore.as_ref(), @@ -1168,41 +1157,23 @@ fn supervise_vjs(handle: Arc, root: PathBuf) { } return Err("no `source` declaration".into()); }; - let nc = nats::connect(&url).map_err(|e| e.to_string())?; - // Acks travel on their own connection. An ack is a publish to the - // message's reply subject, and any connection may send it. On the - // consuming connection the 64 acks of a round sat in front of the - // next round's PING, and every PONG waited for the server to work - // through them (~30 µs each, ~2 ms a round, measured). On a - // connection of their own they cost the server the same and this - // loop nothing. If that connection drops, un-acked messages simply - // redeliver (at-least-once, ADR-0002). - let ack_nc = nats::connect(&url).map_err(|e| e.to_string())?; - let js = nats::jetstream::new(nc.clone()); - let _ = js.add_stream(&nats::jetstream::StreamConfig { + let nc = bus::connect(&url)?; + let js = nc.jetstream(); + let _ = js.add_stream(&bus::StreamConfig { name: stream.clone(), subjects: vec![format!("{subj_root}.>")], ..Default::default() }); let durable = handle.spec.name.replace([':', '.'], "_"); // create the durable pull consumer (idempotent-ish), then bind - let _ = js.add_consumer( + let _ = js.create_pull_consumer( &stream, - nats::jetstream::ConsumerConfig { - durable_name: Some(durable.clone()), - deliver_subject: None, - filter_subject: source.clone(), - ack_wait: connectors::ack_wait(), - ..Default::default() - }, + &durable, + std::slice::from_ref(&source), + connectors::ack_wait(), + None, ); - let sub = js - .pull_subscribe_with_options( - &source, - &nats::jetstream::PullSubscribeOptions::new() - .durable_name(durable), - ) - .map_err(|e| e.to_string())?; + let sub = js.pull_durable(&stream, &durable, Some(&source))?; set_state(&handle, |st| { st.status = "running".into(); st.started_at = Some(now_secs()); @@ -1225,24 +1196,25 @@ fn supervise_vjs(handle: Arc, root: PathBuf) { // (`PULL_EXPIRES_MS`) and its round comes back empty. A stop // leaves at most one request outstanding, whose messages redeliver // un-acked after `ack_wait` — the same as stopping mid-batch. - let mut round = connectors::fetch_iter_flushed(&sub, &nc)?; + let mut round = sub.round()?; + let mut batch = nc.batch(); loop { if !RUNNING.load(Ordering::SeqCst) || handle.stop.load(Ordering::SeqCst) { return Ok(()); } engine.invalidate(); // pick up live service edits let round_t0 = Instant::now(); - let next = connectors::fetch_iter_flushed(&sub, &nc)?; - // Messages are processed as the reader thread hands them over, not - // once the whole batch has landed, so the two threads overlap - // instead of taking turns (see `fetch_iter_flushed`). + let next = sub.round()?; + // Messages are processed as they arrive, not once the whole batch + // has landed, so reading and working overlap instead of taking + // turns. let fetch = &mut round; let mut fetched = 0usize; let mut first_at: Option = None; - // Emits are buffered (core publish) and confirmed by ONE flush per + // Emits are core publishes, held and confirmed by ONE barrier per // batch before their source messages are acked — one round-trip a // batch instead of one per emit, at-least-once preserved (ADR-0002). - let mut to_ack: Vec = Vec::new(); + let mut to_ack: Vec = Vec::new(); for msg in fetch { fetched += 1; first_at.get_or_insert_with(Instant::now); @@ -1259,7 +1231,7 @@ fn supervise_vjs(handle: Arc, root: PathBuf) { let err = format!("bad json: {e}"); eprintln!("[vejas] {}: {err}", handle.spec.name); let delivered = - msg.jetstream_message_info().map(|i| i.delivered).unwrap_or(1); + msg.delivery().map(|i| i.delivered).unwrap_or(1); let prev: String = String::from_utf8_lossy(&msg.data).chars().take(160).collect(); metrics::observe(&handle.spec.name, false, 0, 0.0); @@ -1273,7 +1245,7 @@ fn supervise_vjs(handle: Arc, root: PathBuf) { Err(de) => { record_trace(&handle.spec.name, &msg.subject, false, Some(format!("{err} — DLQ publish failed: {de}")), vec![], prev, None); - let _ = msg.ack_kind(nats::jetstream::AckKind::Nak); + let _ = msg.nak(); } } continue; @@ -1286,8 +1258,8 @@ fn supervise_vjs(handle: Arc, root: PathBuf) { let mut ok = true; for (subj, payload) in &ctx.emits { let bytes = serde_json::to_vec(payload).unwrap_or_default(); - // buffered core publish; flushed once per batch below - if let Err(e) = nc.publish(subj, bytes) { + // held; sent and confirmed once per batch below + if let Err(e) = batch.publish(subj, bytes) { eprintln!( "[vejas] {}: publish {subj}: {e}", handle.spec.name @@ -1338,7 +1310,7 @@ fn supervise_vjs(handle: Arc, root: PathBuf) { // dead-lettered (ADR-0015) rather than dropped; publish // before ack, nak if the DLQ write fails. let delivered = - msg.jetstream_message_info().map(|i| i.delivered).unwrap_or(1); + msg.delivery().map(|i| i.delivered).unwrap_or(1); if delivered >= MAX_DELIVERIES { match connectors::to_dlq(&js, &handle.spec.name, &msg.subject, delivered, &e, &msg.data, &flow_version) { Ok(()) => { @@ -1352,7 +1324,7 @@ fn supervise_vjs(handle: Arc, root: PathBuf) { record_trace(&handle.spec.name, &msg.subject, false, Some(format!("{e} — DLQ publish failed: {de}")), vec![], String::new(), Some(event.clone())); - let _ = msg.ack_kind(nats::jetstream::AckKind::Nak); + let _ = msg.nak(); } } } else { @@ -1364,36 +1336,22 @@ fn supervise_vjs(handle: Arc, root: PathBuf) { } } } - // Publish-before-ack barrier for the batch: one flush confirms the - // server received every buffered emit, THEN ack the fully-published - // source messages. On flush failure, ack none — the batch - // redelivers (nothing is lost; at-least-once, ADR-0002). + // Publish-before-ack barrier for the batch: one round-trip confirms + // the server received every emit, THEN ack the fully-published + // source messages. On failure, ack none — the batch redelivers + // (nothing is lost; at-least-once, ADR-0002). metrics::observe_fetch(&handle.spec.name, fetched); let processed_at = Instant::now(); - if !to_ack.is_empty() { - match nc.flush() { - Ok(_) => { - for m in &to_ack { - match m.reply.as_deref() { - Some(reply) => { - if let Err(e) = ack_nc.publish(reply, b"") { - eprintln!( - "[vejas] {}: ack: {e} (redelivers)", - handle.spec.name - ); - } - } - None => { - let _ = m.ack(); - } - } - } + match batch.confirm() { + Ok(()) => { + if let Err(e) = nc.ack_all(&to_ack) { + eprintln!("[vejas] {}: ack: {e} (redelivers)", handle.spec.name); } - Err(e) => eprintln!( - "[vejas] {}: flush before ack failed: {e} (batch redelivers)", - handle.spec.name - ), } + Err(e) => eprintln!( + "[vejas] {}: flush before ack failed: {e} (batch redelivers)", + handle.spec.name + ), } round = next; let first = first_at.unwrap_or(processed_at); @@ -1503,10 +1461,9 @@ fn supervise_detect(handle: Arc, _root: PathBuf) { .next() .unwrap_or(&handle.spec.name) .to_string(); - let nc = nats::connect(&url).map_err(|e| e.to_string())?; - let ack_nc = nats::connect(&url).map_err(|e| e.to_string())?; - let js = nats::jetstream::new(nc.clone()); - let _ = js.add_stream(&nats::jetstream::StreamConfig { + let nc = bus::connect(&url)?; + let js = nc.jetstream(); + let _ = js.add_stream(&bus::StreamConfig { name: stream.clone(), subjects: vec![format!("{subj_root}.>")], ..Default::default() @@ -1521,7 +1478,7 @@ fn supervise_detect(handle: Arc, _root: PathBuf) { // this attempt, so a restart never leaves a stale reader behind. let attempt_stop = Arc::new(AtomicBool::new(false)); let _stop_reader = StopOnDrop(attempt_stop.clone()); - let (tx, rx) = std::sync::mpsc::channel::(); + let (tx, rx) = std::sync::mpsc::channel::(); let durable = handle.spec.name.replace([':', '.'], "_"); let filters: Vec = sources.iter().map(|(s, _)| s.clone()).collect(); // Snapshot and resume (ADR-0031 points 4-5), for a program with state @@ -1570,33 +1527,24 @@ fn supervise_detect(handle: Arc, _root: PathBuf) { Some(seq) => { // The server cannot move an existing consumer: re-create it after the snapshot. let _ = js.delete_consumer(&stream, &durable); - connectors::create_pull_consumer(&nc, &stream, &durable, &filters, connectors::ack_wait(), Some(seq))?; + js.create_pull_consumer(&stream, &durable, &filters, connectors::ack_wait(), Some(seq))?; metrics::observe_restore(&handle.spec.name, seq); } - None => connectors::create_pull_consumer(&nc, &stream, &durable, &filters, connectors::ack_wait(), None)?, + None => js.create_pull_consumer(&stream, &durable, &filters, connectors::ack_wait(), None)?, } // A first snapshot at the consumer's position, so a crash before the // first cadence still replays from here rather than resuming past // events the state never saw. if let (Some(store), None) = (&store, resume_seq) { - let floor = js - .consumer_info(&stream, &durable) - .map(|i| i.ack_floor.stream_seq) - .unwrap_or(0); + let floor = js.ack_floor(&stream, &durable).unwrap_or(0); if let Err(e) = snapshot_unit(store, &handle.spec.name, &version, &program, floor) { eprintln!("[vejas] {}: first snapshot: {e}", handle.spec.name); } } let sub = js - .pull_subscribe_with_options( - &filters[0], - &nats::jetstream::PullSubscribeOptions::new() - .durable_name(durable.clone()) - .bind_stream(stream.clone()), - ) + .pull_durable(&stream, &durable, None) .map_err(|e| format!("{durable}: {e}"))?; { - let reader_nc = nc.clone(); let stop = attempt_stop.clone(); let unit = handle.clone(); let name = handle.spec.name.clone(); @@ -1605,7 +1553,7 @@ fn supervise_detect(handle: Arc, _root: PathBuf) { && !unit.stop.load(Ordering::SeqCst) && !stop.load(Ordering::SeqCst) { - match connectors::fetch_iter_flushed(&sub, &reader_nc) { + match sub.round() { Ok(round) => { let mut n = 0usize; for msg in round { @@ -1642,6 +1590,7 @@ fn supervise_detect(handle: Arc, _root: PathBuf) { let mut retrying: std::collections::BTreeSet = std::collections::BTreeSet::new(); let mut acks_since_snapshot: u64 = 0; let mut last_snapshot = Instant::now(); + let mut outbox = nc.batch(); let result = (|| -> Result<(), String> { loop { if !RUNNING.load(Ordering::SeqCst) || handle.stop.load(Ordering::SeqCst) { @@ -1694,12 +1643,12 @@ fn supervise_detect(handle: Arc, _root: PathBuf) { Err(_) => break, } } - let mut to_ack: Vec<(nats::Message, u64)> = Vec::new(); + let mut to_ack: Vec<(bus::Message, u64)> = Vec::new(); for msg in batch { if !RUNNING.load(Ordering::SeqCst) || handle.stop.load(Ordering::SeqCst) { return Ok(()); // un-acked: redelivers to whoever holds the durable next } - let seq = msg.jetstream_message_info().map(|i| i.stream_seq).unwrap_or(0); + let seq = msg.delivery().map(|i| i.stream_seq).unwrap_or(0); // The consumer's filters are the `.from()` subjects, so one // matches; the first declared wins when two overlap. let Some(event_type) = sources @@ -1725,7 +1674,7 @@ fn supervise_detect(handle: Arc, _root: PathBuf) { let err = format!("bad payload: {e}"); eprintln!("[vejas] {}: {err}", handle.spec.name); let delivered = - msg.jetstream_message_info().map(|i| i.delivered).unwrap_or(1); + msg.delivery().map(|i| i.delivered).unwrap_or(1); metrics::observe(&handle.spec.name, false, 0, 0.0); match connectors::to_dlq(&js, &handle.spec.name, &msg.subject, delivered, &err, &msg.data, &version) { Ok(()) => { @@ -1734,7 +1683,7 @@ fn supervise_detect(handle: Arc, _root: PathBuf) { retrying.remove(&seq); } Err(_) => { - let _ = msg.ack_kind(nats::jetstream::AckKind::Nak); + let _ = msg.nak(); retrying.insert(seq); } } @@ -1744,7 +1693,7 @@ fn supervise_detect(handle: Arc, _root: PathBuf) { let mut subjects = Vec::with_capacity(emits.len()); for emit in &emits { let subject = emit_subject(emit, &subj_root, &stem); - if let Err(e) = nc.publish(&subject, emit.to_payload()) { + if let Err(e) = outbox.publish(&subject, emit.to_payload()) { eprintln!("[vejas] {}: publish {subject}: {e}", handle.spec.name); ok = false; } @@ -1769,7 +1718,7 @@ fn supervise_detect(handle: Arc, _root: PathBuf) { eprintln!("[vejas] {}: {e}", handle.spec.name); metrics::observe(&handle.spec.name, false, 0, t0.elapsed().as_secs_f64()); let delivered = - msg.jetstream_message_info().map(|i| i.delivered).unwrap_or(1); + msg.delivery().map(|i| i.delivered).unwrap_or(1); if delivered >= MAX_DELIVERIES { match connectors::to_dlq(&js, &handle.spec.name, &msg.subject, delivered, &e, &msg.data, &version) { Ok(()) => { @@ -1778,7 +1727,7 @@ fn supervise_detect(handle: Arc, _root: PathBuf) { retrying.remove(&seq); } Err(_) => { - let _ = msg.ack_kind(nats::jetstream::AckKind::Nak); + let _ = msg.nak(); retrying.insert(seq); } } @@ -1792,29 +1741,20 @@ fn supervise_detect(handle: Arc, _root: PathBuf) { } } let processed_at = Instant::now(); - // Publish-before-ack barrier, one flush per batch (ADR-0002). - if !to_ack.is_empty() { - match nc.flush() { - Ok(_) => { - for (m, seq) in &to_ack { - match m.reply.as_deref() { - Some(reply) => { - let _ = ack_nc.publish(reply, b""); - } - None => { - let _ = m.ack(); - } - } - retrying.remove(seq); - acked_max = acked_max.max(*seq); - } - acks_since_snapshot += to_ack.len() as u64; + // Publish-before-ack barrier, one round-trip per batch (ADR-0002). + match outbox.confirm() { + Ok(()) => { + let _ = nc.ack_all(to_ack.iter().map(|(m, _)| m)); + for (_, seq) in &to_ack { + retrying.remove(seq); + acked_max = acked_max.max(*seq); } - Err(e) => eprintln!( - "[vejas] {}: flush before ack failed: {e} (batch redelivers)", - handle.spec.name - ), + acks_since_snapshot += to_ack.len() as u64; } + Err(e) => eprintln!( + "[vejas] {}: flush before ack failed: {e} (batch redelivers)", + handle.spec.name + ), } metrics::observe_round( &handle.spec.name, @@ -1883,7 +1823,7 @@ fn program_is_stateful(program: &varpulis_engine::Program) -> bool { /// Take the unit's snapshot standing for stream sequence `seq` and store it. fn snapshot_unit( - store: &nats::object_store::ObjectStore, + store: &bus::Objects, unit: &str, version: &str, program: &varpulis_engine::Program, @@ -2062,8 +2002,7 @@ fn supervise_connector(handle: Arc, root: PathBuf) { // graceful stop. If the lease is lost mid-run, the renewal thread clears // `running`, the driver returns, and the supervisor re-enters standby. let _lease = if cluster::is_singleton(driver.kind()) { - let nc = nats::connect(&url).map_err(|e| e.to_string())?; - let js = nats::jetstream::new(nc); + let js = bus::connect(&url)?.jetstream(); match cluster::open_lease_store(&js) { Some(store) => { let key = connectors::dlq_unit_token(&handle.spec.name); @@ -2129,31 +2068,32 @@ fn supervise_connector(handle: Arc, root: PathBuf) { fn version_watch(registry: Registry, root: PathBuf) { let url = env::var("NATS_URL").unwrap_or_else(|_| "nats://127.0.0.1:4222".into()); while RUNNING.load(Ordering::SeqCst) { - let store = nats::connect(&url) + let store = bus::connect(&url) .ok() - .map(nats::jetstream::new) - .and_then(|js| versions::open_store(&js)); + .and_then(|bus| versions::open_store(&bus.jetstream())); let Some(store) = store else { thread::sleep(Duration::from_secs(2)); continue; }; - let watch = match store.watch_all() { + let mut watch = match store.watch_all() { Ok(w) => w, Err(_) => { thread::sleep(Duration::from_secs(2)); continue; } }; - for entry in watch { - if !RUNNING.load(Ordering::SeqCst) { - return; - } + while RUNNING.load(Ordering::SeqCst) { + let key = match watch.next_timeout(Duration::from_millis(500)) { + Ok(Some(key)) => key, + Ok(None) => continue, + Err(_) => break, + }; // map the KV key token back to a supervised flow unit and restart it // (it then re-resolves the overlay: apply on a promote, baseline on // an eviction/delete). let target = { let reg = registry.lock().unwrap(); - reg.keys().find(|n| versions::key_token(n) == entry.key).cloned() + reg.keys().find(|n| versions::key_token(n) == key).cloned() }; if let Some(name) = target { eprintln!("[vejas] version change for {name} → hot-reloading"); @@ -3056,19 +2996,18 @@ fn sap_rpc_request(root: &Path, op: &Value) -> Result { "no exec-rpc SAP connector found — add a manifest with driver \"exec-rpc\" and REQUEST_SUBJECT (see docs/examples/sap_rpc.vjs.example)", )?; let url = env::var("NATS_URL").unwrap_or_else(|_| "nats://127.0.0.1:4222".into()); - let nc = nats::connect(&url).map_err(|e| e.to_string())?; + let nc = bus::connect(&url)?; let payload = serde_json::to_vec(op).map_err(|e| e.to_string())?; let msg = nc - .request_timeout(&subject, payload, Duration::from_secs(30)) + .request(&subject, payload, Duration::from_secs(30)) .map_err(|e| format!("SAP connector did not answer on {subject}: {e}"))?; serde_json::from_slice::(&msg.data).map_err(|e| e.to_string()) } // ───────────────────────── dead-letter queue (ADR-0015) ───────────────────────── -fn dlq_js() -> Result { +fn dlq_js() -> Result { let url = env::var("NATS_URL").unwrap_or_else(|_| "nats://127.0.0.1:4222".into()); - let nc = nats::connect(&url).map_err(|e| e.to_string())?; - let js = nats::jetstream::new(nc); + let js = bus::connect(&url)?.jetstream(); connectors::ensure_dlq_stream(&js); Ok(js) } @@ -3076,10 +3015,10 @@ fn dlq_js() -> Result { /// List dead letters, newest first (each carries its stream `seq` for replay/purge). fn dlq_list(limit: usize) -> Result, String> { let js = dlq_js()?; - let info = js.stream_info(connectors::DLQ_STREAM).map_err(|e| e.to_string())?; - let (first, last) = (info.state.first_seq, info.state.last_seq); + let state = js.stream_state(connectors::DLQ_STREAM)?; + let (first, last) = (state.first_seq, state.last_seq); let mut out = Vec::new(); - if last == 0 || info.state.messages == 0 { + if last == 0 || state.messages == 0 { return Ok(out); } let mut seq = last; @@ -3103,7 +3042,7 @@ fn dlq_list(limit: usize) -> Result, String> { /// Record a replay or a purge: one summary per call, best-effort like the /// live-promote path. A purge in particular leaves no other trace. fn dlq_audit( - js: &nats::jetstream::JetStream, + js: &bus::JetStream, kind: &str, actor: &str, seq: Option, @@ -3134,7 +3073,7 @@ fn dlq_replay(seq: Option, unit: Option<&str>, actor: &str) -> Result, unit: Option<&str>, actor: &str) -> Result, unit: Option<&str>, actor: &str) -> Result { let js = dlq_js()?; if seq.is_none() && unit.is_none() { - let r = js.purge_stream(connectors::DLQ_STREAM).map_err(|e| e.to_string())?; - dlq_audit(&js, "dlq-purge", actor, None, None, r.purged); - return Ok(json!({"purged": r.purged})); + let purged = js.purge_stream(connectors::DLQ_STREAM)?; + dlq_audit(&js, "dlq-purge", actor, None, None, purged); + return Ok(json!({"purged": purged})); } let mut n = 0u64; if let Some(s) = seq { - js.delete_message(connectors::DLQ_STREAM, s).map_err(|e| e.to_string())?; + js.delete_message(connectors::DLQ_STREAM, s)?; n = 1; } else if let Some(u) = unit { for env in dlq_list(100_000)? { @@ -3416,11 +3355,12 @@ fn publish_emits(emits: &[(String, Value)]) { return; } let url = env::var("NATS_URL").unwrap_or_else(|_| "nats://127.0.0.1:4222".into()); - if let Ok(nc) = nats::connect(&url) { + if let Ok(nc) = bus::connect(&url) { + let mut batch = nc.batch(); for (subj, payload) in emits { - let _ = nc.publish(subj, serde_json::to_vec(payload).unwrap_or_default()); + let _ = batch.publish(subj, serde_json::to_vec(payload).unwrap_or_default()); } - let _ = nc.flush(); + let _ = batch.confirm(); } } diff --git a/core/src/proposals.rs b/core/src/proposals.rs index a516cc9..758a5aa 100644 --- a/core/src/proposals.rs +++ b/core/src/proposals.rs @@ -13,6 +13,8 @@ use std::sync::atomic::{AtomicU64, Ordering}; +use crate::bus; + const PROPOSALS_BUCKET: &str = "VEJAS_PROPOSALS"; const PROPOSALS_HISTORY: i64 = 64; // bounded live queue; audit is the durable record @@ -21,15 +23,12 @@ static SEQ: AtomicU64 = AtomicU64::new(1); /// Open (or create) the shared proposal bucket. None if JetStream KV is /// unavailable (proposals need the shared bus; a single dev instance without it /// simply has no queue). -pub fn open_store(js: &nats::jetstream::JetStream) -> Option { - let cfg = nats::kv::Config { +pub fn open_store(js: &bus::JetStream) -> Option { + js.key_value(&bus::KvConfig { bucket: PROPOSALS_BUCKET.to_string(), history: PROPOSALS_HISTORY, ..Default::default() - }; - js.create_key_value(&cfg) - .or_else(|_| js.key_value(PROPOSALS_BUCKET)) - .ok() + }) } /// A fresh proposal id — `ts` is the wall-clock second (stamped by the caller; @@ -41,7 +40,7 @@ pub fn new_id(ts: u64) -> String { /// Submit a proposal (status "pending"). Returns the stored record. #[allow(clippy::too_many_arguments)] pub fn submit( - store: &nats::kv::Store, + store: &bus::Kv, id: &str, kind: &str, payload: serde_json::Value, @@ -66,7 +65,7 @@ pub fn submit( Ok(record) } -pub fn get(store: &nats::kv::Store, id: &str) -> Option { +pub fn get(store: &bus::Kv, id: &str) -> Option { store .get(id) .ok() @@ -75,10 +74,10 @@ pub fn get(store: &nats::kv::Store, id: &str) -> Option { } /// Every proposal, newest first (by created_at). -pub fn list(store: &nats::kv::Store) -> Vec { +pub fn list(store: &bus::Kv) -> Vec { let mut all: Vec = store .keys() - .map(|keys| keys.filter_map(|k| get(store, &k)).collect()) + .map(|keys| keys.iter().filter_map(|k| get(store, k)).collect()) .unwrap_or_default(); all.sort_by_key(|p| std::cmp::Reverse(p["created_at"].as_u64().unwrap_or(0))); all @@ -87,7 +86,7 @@ pub fn list(store: &nats::kv::Store) -> Vec { /// Transition a proposal's status (and stamp who/when for a human decision). /// Returns the updated record, or an error if it is gone. pub fn set_status( - store: &nats::kv::Store, + store: &bus::Kv, id: &str, status: &str, decided_by: Option<&str>, diff --git a/core/src/versions.rs b/core/src/versions.rs index 6e4bc33..53da923 100644 --- a/core/src/versions.rs +++ b/core/src/versions.rs @@ -15,6 +15,8 @@ use std::sync::{Mutex, OnceLock}; +use crate::bus; + const VERSIONS_BUCKET: &str = "VEJAS_VERSIONS"; const VERSIONS_HISTORY: i64 = 64; // bounded rollback/audit history per flow (ADR-0021) @@ -44,19 +46,16 @@ fn key_of(flow: &str) -> String { /// Open (or create) the shared version bucket. None if JetStream KV is /// unavailable — the caller then uses the baseline file (single-instance, no /// overlay), no worse than before versioning existed. -pub fn open_store(js: &nats::jetstream::JetStream) -> Option { - let cfg = nats::kv::Config { +pub fn open_store(js: &bus::JetStream) -> Option { + js.key_value(&bus::KvConfig { bucket: VERSIONS_BUCKET.to_string(), history: VERSIONS_HISTORY, ..Default::default() - }; - js.create_key_value(&cfg) - .or_else(|_| js.key_value(VERSIONS_BUCKET)) - .ok() + }) } /// The current overlay for a flow, if any (the parsed snapshot value). -pub fn get_version(store: &nats::kv::Store, flow: &str) -> Option { +pub fn get_version(store: &bus::Kv, flow: &str) -> Option { store .get(&key_of(flow)) .ok() @@ -68,7 +67,7 @@ pub fn get_version(store: &nats::kv::Store, flow: &str) -> Option Vec { /// is evicted here — git wins, loudly (logged + noted for the panel; the caller /// also writes an ADR-0018 audit record on `Evicted`). Returns (source, outcome). pub fn resolve_source( - store: Option<&nats::kv::Store>, + store: Option<&bus::Kv>, flow: &str, baseline: &str, ts: u64, From 289b16c3be550a877beb902b0527ab0bfae011a1 Mon Sep 17 00:00:00 2001 From: Cyril Poder Date: Fri, 6 Nov 2026 19:47:50 +0100 Subject: [PATCH 3/9] connectors: mq and amqp compile the runtime's bus module Both standalone binaries held their own nats 0.25 client, and each carried a copy of publish_confirmed and fetch_round to get around the same flusher floor. They now compile core/src/bus.rs through #[path], which depends on nothing else in core, and those copies go. The source and sink loops keep their shape: publish before commit, put or confirm before ack. connectors/amqp moves from rustls 0.22 to 0.23 with the ring provider, with no source change to its TLS adapter. That was the last rustls 0.22 in the repo. Both crates pass their tests, the live lease and source tests included, and e2e/admission admits amqp_source and amqp_sink against a real RabbitMQ. "No tokio" no longer describes them, so SUBJECTS.md and COOKBOOK.md stop saying it. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_018wXgxT2UoFSZfqhAE1Tb5V --- connectors/amqp/Cargo.lock | 731 +++++++++++---------------------- connectors/amqp/Cargo.toml | 15 +- connectors/amqp/src/lease.rs | 24 +- connectors/amqp/src/main.rs | 96 +---- connectors/mq/Cargo.lock | 768 +++++++++++------------------------ connectors/mq/Cargo.toml | 8 +- connectors/mq/src/lease.rs | 24 +- connectors/mq/src/main.rs | 110 ++--- docs/COOKBOOK.md | 2 +- docs/SUBJECTS.md | 2 +- 10 files changed, 567 insertions(+), 1213 deletions(-) diff --git a/connectors/amqp/Cargo.lock b/connectors/amqp/Cargo.lock index e3dd235..624e4c2 100644 --- a/connectors/amqp/Cargo.lock +++ b/connectors/amqp/Cargo.lock @@ -25,7 +25,7 @@ dependencies = [ "indexmap", "input_buffer", "log", - "mio", + "mio 0.6.23", "mio-extras", "percent-encoding 2.3.2", "snafu", @@ -70,29 +70,38 @@ dependencies = [ ] [[package]] -name = "async-channel" -version = "2.5.0" +name = "async-nats" +version = "0.52.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "924ed96dd52d1b75e9c1a3e6275715fd320f5f9439fb5a4a11fa51f4221158d2" +checksum = "015f0a7a3b0b29701b78c76a4d4a122624c97c71fa30363381664eb729d619c7" dependencies = [ - "concurrent-queue", - "event-listener-strategy", - "futures-core", - "pin-project-lite", + "base64", + "bytes", + "futures-util", + "memchr", + "pin-project", + "portable-atomic", + "rand", + "regex", + "ring", + "rustls-native-certs", + "rustls-pki-types", + "rustls-webpki", + "serde", + "serde_json", + "serde_nanos", + "serde_repr", + "thiserror", + "time", + "tokio", + "tokio-rustls", + "tokio-stream", + "tokio-util", + "tracing", + "tryhard", + "url 2.5.8", ] -[[package]] -name = "async-task" -version = "4.7.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8b75356056920673b02621b35afd0f7dda9306d03c79a30f5c56c44cf256e3de" - -[[package]] -name = "atomic-waker" -version = "1.1.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1505bd5d3d116872e7271a6d4e16d81d0c8570876c8de68093a09ac269d8aac0" - [[package]] name = "autocfg" version = "1.5.1" @@ -101,24 +110,9 @@ checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53" [[package]] name = "base64" -version = "0.13.1" +version = "0.22.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9e1b586273c5702936fe7b7d6896644d8be71e6314cfe09d3167c95f712589e8" - -[[package]] -name = "base64-url" -version = "1.4.13" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "67a99c239d0c7e77c85dddfa9cebce48704b3c49550fcd3b84dd637e4484899f" -dependencies = [ - "base64", -] - -[[package]] -name = "base64ct" -version = "1.8.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06" +checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6" [[package]] name = "bitflags" @@ -132,28 +126,6 @@ version = "2.13.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da" -[[package]] -name = "block-buffer" -version = "0.10.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71" -dependencies = [ - "generic-array", -] - -[[package]] -name = "blocking" -version = "1.6.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e83f8d02be6967315521be875afa792a316e28d57b5a2d401897e2a7921b7f21" -dependencies = [ - "async-channel", - "async-task", - "futures-io", - "futures-lite", - "piper", -] - [[package]] name = "built" version = "0.5.2" @@ -168,6 +140,9 @@ name = "bytes" version = "1.12.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "fc652a48c352aef3ea3aed32080501cf3ef6ed5da78602a020c991775b0aff04" +dependencies = [ + "serde", +] [[package]] name = "cargo-lock" @@ -204,20 +179,16 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" [[package]] -name = "concurrent-queue" -version = "2.5.0" +name = "chacha20" +version = "0.10.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "4ca0197aee26d1ae37445ee532fefce43251d24cc7c166799f4d46817f1d3973" +checksum = "65c35e4b699c7e15ccbe7ee35c005e4fc0a278d22238a2857e6ce2dadeda1b06" dependencies = [ - "crossbeam-utils", + "cfg-if 1.0.4", + "cpufeatures", + "rand_core", ] -[[package]] -name = "const-oid" -version = "0.9.6" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c2459377285ad874054d797f3ccebf984978aa39129f6eafde5cdc8315b612f8" - [[package]] name = "cookie-factory" version = "0.2.4" @@ -226,9 +197,9 @@ checksum = "98a479f8099cc5ac64915a3dd76c87be27f929ba406ad705aacb13f19b791207" [[package]] name = "core-foundation" -version = "0.9.4" +version = "0.10.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "91e195e091a93c46f7102ec7818a2aa394e1e1771c3ab4825963fa03e45afb8f" +checksum = "b2a6cd9ae233e7f62ba4e9353e81a88df7fc8a5987b8d445b4d90c879bd156f6" dependencies = [ "core-foundation-sys", "libc", @@ -242,9 +213,9 @@ checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b" [[package]] name = "cpufeatures" -version = "0.2.17" +version = "0.3.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280" +checksum = "5ca28b0ae3115b884660db4118d803791fd6756b6e88f39c0f3f7859060d7566" dependencies = [ "libc", ] @@ -264,59 +235,6 @@ version = "0.8.22" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "61803da095bee82a81bb1a452ecc25d3b2f1416d1897eb86430c6159ef717c17" -[[package]] -name = "crypto-common" -version = "0.1.7" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a" -dependencies = [ - "generic-array", - "typenum", -] - -[[package]] -name = "curve25519-dalek" -version = "4.1.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "97fb8b7c4503de7d6ae7b42ab72a5a59857b4c937ec27a3d4539dba95b5ab2be" -dependencies = [ - "cfg-if 1.0.4", - "cpufeatures", - "curve25519-dalek-derive", - "digest", - "fiat-crypto", - "rustc_version", - "subtle", -] - -[[package]] -name = "curve25519-dalek-derive" -version = "0.1.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f46882e17999c6cc590af592290432be3bce0428cb0d5f8b6715e4dc7b383eb3" -dependencies = [ - "proc-macro2", - "quote", - "syn 2.0.119", -] - -[[package]] -name = "data-encoding" -version = "2.11.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "4583a4551df46e2792f82ceeac45e850d2e2d5debba0b91f102385cda5b11f06" - -[[package]] -name = "der" -version = "0.7.10" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e7c1832837b905bbfb5101e07cc24c8deddf52f93225eee6ead5f4d63d53ddcb" -dependencies = [ - "const-oid", - "pem-rfc7468", - "zeroize", -] - [[package]] name = "deranged" version = "0.5.8" @@ -326,16 +244,6 @@ dependencies = [ "serde_core", ] -[[package]] -name = "digest" -version = "0.10.7" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292" -dependencies = [ - "block-buffer", - "crypto-common", -] - [[package]] name = "displaydoc" version = "0.2.7" @@ -353,69 +261,6 @@ version = "0.3.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "780955b8b195a21ab8e4ac6b60dd1dbdcec1dc6c51c0617964b08c81785e12c9" -[[package]] -name = "ed25519" -version = "2.2.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "115531babc129696a58c64a4fef0a8bf9e9698629fb97e9e40767d235cfbcd53" -dependencies = [ - "signature", -] - -[[package]] -name = "ed25519-dalek" -version = "2.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "70e796c081cee67dc755e1a36a0a172b897fab85fc3f6bc48307991f64e4eca9" -dependencies = [ - "curve25519-dalek", - "ed25519", - "sha2", - "signature", - "subtle", -] - -[[package]] -name = "event-listener" -version = "5.4.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5a23add41df1562121a9393cb065eab5146a1242410f23a644851e90cfd669d2" -dependencies = [ - "parking", - "pin-project-lite", -] - -[[package]] -name = "event-listener-strategy" -version = "0.5.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8be9f3dfaaffdae2972880079a491a1a8bb7cbed0b8dd7a347f668b4150a3b93" -dependencies = [ - "event-listener", - "pin-project-lite", -] - -[[package]] -name = "fastrand" -version = "1.9.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e51093e27b0797c359783294ca4f0a911c270184cb10f85783b118614a1501be" -dependencies = [ - "instant", -] - -[[package]] -name = "fastrand" -version = "2.5.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "da7c62ceae207dd37ea5b845da6a0696c799f85e97da1ab5b7910be3c1c80223" - -[[package]] -name = "fiat-crypto" -version = "0.2.9" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "28dea519a9695b9977216879a3ebfddf92f1c08c05d984f8996aecd6ecdc811d" - [[package]] name = "find-msvc-tools" version = "0.1.11" @@ -454,40 +299,51 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "92d699e522242e69e3003b94ecc1f960f3a5e015aa7c5d7486e65ad01dd94f5e" [[package]] -name = "futures-io" +name = "futures-sink" version = "0.3.34" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "53c0fa8157de1303bfffdaa1cc2a673bfffb60102f76b0ef4441659124373fed" +checksum = "1944426bf7d03f1d14f708785e4b33efd750b36d48a157b836b3efc15ede8e1d" [[package]] -name = "futures-lite" -version = "2.6.1" +name = "futures-task" +version = "0.3.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cd417de3d1d015fc3bfd2b1ea46dfc7bab72ef86f1cc7cc9c78e728b34a6d1fd" + +[[package]] +name = "futures-util" +version = "0.3.34" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f78e10609fe0e0b3f4157ffab1876319b5b0db102a2c60dc4626306dc46b44ad" +checksum = "0d50a92467f8ba5dd6e3ee5d4bd04d73ab2e4e1c44474a0674821dfce14b79bc" dependencies = [ "futures-core", + "futures-sink", + "futures-task", "pin-project-lite", + "slab", ] [[package]] -name = "generic-array" -version = "0.14.7" +name = "getrandom" +version = "0.2.17" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a" +checksum = "ff2abc00be7fca6ebc474524697ae276ad847ad0a6b3faa4bcb027e9a4614ad0" dependencies = [ - "typenum", - "version_check 0.9.5", + "cfg-if 1.0.4", + "libc", + "wasi", ] [[package]] name = "getrandom" -version = "0.2.17" +version = "0.4.3" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ff2abc00be7fca6ebc474524697ae276ad847ad0a6b3faa4bcb027e9a4614ad0" +checksum = "300e883d756b2e4ec94e02791f39b04b522276138852cfc41d9fb7e904106099" dependencies = [ "cfg-if 1.0.4", "libc", - "wasi", + "r-efi", + "rand_core", ] [[package]] @@ -653,15 +509,6 @@ dependencies = [ "bytes", ] -[[package]] -name = "instant" -version = "0.1.13" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e0242819d153cba4b4b05a5a8f2a7e9bbf97b6055b2a002b395c96b5ff3c0222" -dependencies = [ - "cfg-if 1.0.4", -] - [[package]] name = "iovec" version = "0.1.4" @@ -677,12 +524,6 @@ version = "1.0.18" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682" -[[package]] -name = "json" -version = "0.12.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "078e285eafdfb6c4b434e0d31e8cfcb5115b651496faca5749b88fafd4f23bfd" - [[package]] name = "kernel32-sys" version = "0.2.2" @@ -717,15 +558,6 @@ version = "0.8.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "47d9d19d1d6efa0109d2f65ff4c85cddd50bd572e5a00127ab10987290bcefae" -[[package]] -name = "lock_api" -version = "0.4.14" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "224399e74b87b5f3557511d98dff8b14089b3dadafcab6bb93eab67d3aace965" -dependencies = [ - "scopeguard", -] - [[package]] name = "log" version = "0.4.34" @@ -763,6 +595,17 @@ dependencies = [ "winapi 0.2.8", ] +[[package]] +name = "mio" +version = "1.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1788edb87fdc09c7e26304471e2f5be8cdefb1b6930d6e3985fc02ff53bf86ee" +dependencies = [ + "libc", + "wasi", + "windows-sys 0.61.2", +] + [[package]] name = "mio-extras" version = "2.0.6" @@ -771,7 +614,7 @@ checksum = "52403fe290012ce777c4626790c8951324a2b9e3316b3143779c72b029742f19" dependencies = [ "lazycell", "log", - "mio", + "mio 0.6.23", "slab", ] @@ -787,43 +630,6 @@ dependencies = [ "ws2_32-sys", ] -[[package]] -name = "nats" -version = "0.25.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "19090dd06e27eb59adfd16f332b85818a3e7dbbdbc13c448676c9bb69f964208" -dependencies = [ - "base64", - "base64-url", - "blocking", - "crossbeam-channel", - "fastrand 1.9.0", - "itoa", - "json", - "lazy_static", - "libc", - "log", - "memchr", - "nkeys", - "nuid", - "once_cell", - "parking_lot", - "portable-atomic", - "regex", - "ring", - "rustls", - "rustls-native-certs", - "rustls-pemfile", - "rustls-webpki", - "serde", - "serde_json", - "serde_nanos", - "serde_repr", - "time", - "url 2.5.8", - "winapi 0.3.9", -] - [[package]] name = "net2" version = "0.2.39" @@ -835,21 +641,6 @@ dependencies = [ "winapi 0.3.9", ] -[[package]] -name = "nkeys" -version = "0.4.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "879011babc47a1c7fdf5a935ae3cfe94f34645ca0cac1c7f6424b36fc743d1bf" -dependencies = [ - "data-encoding", - "ed25519", - "ed25519-dalek", - "getrandom", - "log", - "rand", - "signatory", -] - [[package]] name = "nom" version = "4.2.3" @@ -857,17 +648,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "2ad2a91a8e869eeb30b9cb3119ae87773a8f4ae617f41b1eb9c154b2905f7bd6" dependencies = [ "memchr", - "version_check 0.1.5", -] - -[[package]] -name = "nuid" -version = "0.3.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "20c1bb65186718d348306bf1afdeb20d9ab45b2ab80fb793c0fdcf59ffbb4f38" -dependencies = [ - "lazy_static", - "rand", + "version_check", ] [[package]] @@ -884,47 +665,9 @@ checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" [[package]] name = "openssl-probe" -version = "0.1.6" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d05e27ee213611ffe7d6348b942e8f942b37114c00cc03cec254295a4a17852e" - -[[package]] -name = "parking" -version = "2.2.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f38d5652c16fde515bb1ecef450ab0f6a219d619a7274976324d5e377f7dceba" - -[[package]] -name = "parking_lot" -version = "0.12.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "93857453250e3077bd71ff98b6a65ea6621a19bb0f559a85248955ac12c45a1a" -dependencies = [ - "lock_api", - "parking_lot_core", -] - -[[package]] -name = "parking_lot_core" -version = "0.9.12" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2621685985a2ebf1c516881c026032ac7deafcda1a2c9b7850dc81e3dfcb64c1" -dependencies = [ - "cfg-if 1.0.4", - "libc", - "redox_syscall", - "smallvec", - "windows-link", -] - -[[package]] -name = "pem-rfc7468" -version = "0.7.0" +version = "0.2.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "88b39c9bfcfc231068454382784bb460aae594343fb030d46e9f50a645418412" -dependencies = [ - "base64ct", -] +checksum = "7e036b411754132fd6b74e94b74e9a13c5cf606fa93d1885c35860d3060820b1" [[package]] name = "percent-encoding" @@ -981,31 +724,30 @@ dependencies = [ ] [[package]] -name = "pin-project-lite" -version = "0.2.17" +name = "pin-project" +version = "1.1.13" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd" +checksum = "2466b2336ed02bcdca6b294417127b90ec92038d1d5c4fbeac971a922e0e0924" +dependencies = [ + "pin-project-internal", +] [[package]] -name = "piper" -version = "0.2.5" +name = "pin-project-internal" +version = "1.1.13" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c835479a4443ded371d6c535cbfd8d31ad92c5d23ae9770a61bc155e4992a3c1" +checksum = "c96395f0a926bc13b1c17622aaddda1ecb55d49c8f1bf9777e4d877800a43f8b" dependencies = [ - "atomic-waker", - "fastrand 2.5.0", - "futures-io", + "proc-macro2", + "quote", + "syn 2.0.119", ] [[package]] -name = "pkcs8" -version = "0.10.2" +name = "pin-project-lite" +version = "0.2.17" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f950b2377845cebe5cf8b5165cb3cc1a5e0fa5cfa3e1f7f55707d8fd82e0a7b7" -dependencies = [ - "der", - "spki", -] +checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd" [[package]] name = "portable-atomic" @@ -1028,15 +770,6 @@ version = "0.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "439ee305def115ba05938db6eb1644ff94165c5ab5e9420d1c1bcedbba909391" -[[package]] -name = "ppv-lite86" -version = "0.2.21" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "85eae3c4ed2f50dcfe72643da4befc30deadb458a9b590d720cde2f2b1e97da9" -dependencies = [ - "zerocopy", -] - [[package]] name = "proc-macro2" version = "1.0.107" @@ -1062,43 +795,27 @@ dependencies = [ ] [[package]] -name = "rand" -version = "0.8.7" +name = "r-efi" +version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "22f6172bdec972074665ed81ed53b71da00bfc44b65a753cfde883ec4c702a1a" -dependencies = [ - "libc", - "rand_chacha", - "rand_core", -] +checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf" [[package]] -name = "rand_chacha" -version = "0.3.1" +name = "rand" +version = "0.10.3" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e6c10a63a0fa32252be49d21e7709d4d4baf8d231c2dbce1eaa8141b9b127d88" +checksum = "65c9fb96cbc91e3478eaae79a69fcd3f1ae4ad052e471fe6732fff548984b4af" dependencies = [ - "ppv-lite86", + "chacha20", + "getrandom 0.4.3", "rand_core", ] [[package]] name = "rand_core" -version = "0.6.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ec0be4795e2f6a28069bec0b5ff3e2ac9bafc99e6a9a7dc3547996c5c816922c" -dependencies = [ - "getrandom", -] - -[[package]] -name = "redox_syscall" -version = "0.5.18" +version = "0.10.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d" -dependencies = [ - "bitflags 2.13.1", -] +checksum = "63b8176103e19a2643978565ca18b50549f6101881c443590420e4dc998a3c69" [[package]] name = "regex" @@ -1137,28 +854,20 @@ checksum = "a4689e6c2294d81e88dc6261c768b63bc4fcdb852be6d1352498b114f61383b7" dependencies = [ "cc", "cfg-if 1.0.4", - "getrandom", + "getrandom 0.2.17", "libc", "untrusted", "windows-sys 0.52.0", ] -[[package]] -name = "rustc_version" -version = "0.4.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "cfcb3a22ef46e85b45de6ee7e79d063319ebb6594faafcf1c225ea92ab6e9b92" -dependencies = [ - "semver", -] - [[package]] name = "rustls" -version = "0.22.4" +version = "0.23.46" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "bf4ef73721ac7bcd79b2b315da7779d8fc09718c6b3d2d1b2d94850eb8c18432" +checksum = "48e13bd8c0e9365c43cfa5c9e8f9ad49d3c8444926c9aac819e0e4dc503c8fdf" dependencies = [ "log", + "once_cell", "ring", "rustls-pki-types", "rustls-webpki", @@ -1168,12 +877,11 @@ dependencies = [ [[package]] name = "rustls-native-certs" -version = "0.7.3" +version = "0.8.5" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e5bfb394eeed242e909609f56089eecfe5fda225042e8b171791b9c95f5931e5" +checksum = "4a073f5dc7469f984c52ad2752b63b0807745133b6de880b7b64c1ac4c48aec4" dependencies = [ "openssl-probe", - "rustls-pemfile", "rustls-pki-types", "schannel", "security-framework", @@ -1199,9 +907,9 @@ dependencies = [ [[package]] name = "rustls-webpki" -version = "0.102.8" +version = "0.103.16" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "64ca1bc8749bd4cf37b5ce386cc146580777b4e8572c7b97baf22c83f444bee9" +checksum = "4443b1b6d39d7c663139942cc4558262562d2c84123bb6282ef177009ca8b281" dependencies = [ "ring", "rustls-pki-types", @@ -1226,17 +934,11 @@ dependencies = [ "windows-sys 0.61.2", ] -[[package]] -name = "scopeguard" -version = "1.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49" - [[package]] name = "security-framework" -version = "2.11.1" +version = "3.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "897b2245f0b511c87893af39b033e5ca9cce68824c4d7e7630b5a1d339658d02" +checksum = "b7f4bc775c73d9a02cde8bf7b2ec4c9d12743edf609006c7facc23998404cd1d" dependencies = [ "bitflags 2.13.1", "core-foundation", @@ -1328,45 +1030,12 @@ dependencies = [ "syn 3.0.3", ] -[[package]] -name = "sha2" -version = "0.10.9" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283" -dependencies = [ - "cfg-if 1.0.4", - "cpufeatures", - "digest", -] - [[package]] name = "shlex" version = "2.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba" -[[package]] -name = "signatory" -version = "0.27.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c1e303f8205714074f6068773f0e29527e0453937fe837c9717d066635b65f31" -dependencies = [ - "pkcs8", - "rand_core", - "signature", - "zeroize", -] - -[[package]] -name = "signature" -version = "2.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "77549399552de45a898a580c1b41d445bf730df867cc44e6c0233bbc4b8329de" -dependencies = [ - "digest", - "rand_core", -] - [[package]] name = "slab" version = "0.4.12" @@ -1402,13 +1071,13 @@ dependencies = [ ] [[package]] -name = "spki" -version = "0.7.3" +name = "socket2" +version = "0.6.5" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d91ed6c858b01f942cd56b37a94b3e0a1798290327d1236e4d9cf4eaca44d29d" +checksum = "c3d1e2c7f27f8d4cb10542a02c49005dbd6e93095799d6f3be745fae9f8fedd4" dependencies = [ - "base64ct", - "der", + "libc", + "windows-sys 0.61.2", ] [[package]] @@ -1467,6 +1136,26 @@ dependencies = [ "syn 2.0.119", ] +[[package]] +name = "thiserror" +version = "2.0.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "09e52cb86a36cede5cb101bf8908837b3e4c6e5e59fe7fd85c23fb56200d189e" +dependencies = [ + "thiserror-impl", +] + +[[package]] +name = "thiserror-impl" +version = "2.0.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fe5197923287db20a58125f0bc85c062f7f2c892de97b18c356f9efb14b28524" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.3", +] + [[package]] name = "time" version = "0.3.55" @@ -1522,6 +1211,66 @@ version = "0.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1f3ccbac311fea05f86f61904b462b55fb3df8837a366dfc601a0161d0532f20" +[[package]] +name = "tokio" +version = "1.54.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b52fd3255258d26fac7264ab5a76b53c13367136dc011d3d6ad4f8ed070396a3" +dependencies = [ + "bytes", + "libc", + "mio 1.2.4", + "pin-project-lite", + "socket2", + "tokio-macros", + "windows-sys 0.61.2", +] + +[[package]] +name = "tokio-macros" +version = "2.7.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "78773a2a397f451582ce068015985c33193cf6dea8b74d2a639fe457b2f07b0e" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.3", +] + +[[package]] +name = "tokio-rustls" +version = "0.26.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c9cc2678c2cdd569ef8215e2afd7954ada2ae20b4fdd2c5fe6139a3b02d105db" +dependencies = [ + "rustls", + "tokio", +] + +[[package]] +name = "tokio-stream" +version = "0.1.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a3d06f0b082ba57c26b79407372e57cf2a1e28124f78e9479fe80322cf53420b" +dependencies = [ + "futures-core", + "pin-project-lite", + "tokio", +] + +[[package]] +name = "tokio-util" +version = "0.7.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e464cf451ba96ebfc6f9b6542f17ee8b8956e33f1e40d9690624e59d7a7f8a4b" +dependencies = [ + "bytes", + "futures-core", + "futures-sink", + "pin-project-lite", + "tokio", +] + [[package]] name = "toml" version = "0.5.11" @@ -1532,10 +1281,45 @@ dependencies = [ ] [[package]] -name = "typenum" -version = "1.20.1" +name = "tracing" +version = "0.1.44" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100" +dependencies = [ + "pin-project-lite", + "tracing-attributes", + "tracing-core", +] + +[[package]] +name = "tracing-attributes" +version = "0.1.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7490cfa5ec963746568740651ac6781f701c9c5ea257c58e057f3ba8cf69e8da" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "tracing-core" +version = "0.1.36" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a" +dependencies = [ + "once_cell", +] + +[[package]] +name = "tryhard" +version = "0.5.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b6f5e870be6c3b371b77fe0ee0bafb859fa4964b4404c27de1d380043c4dda20" +checksum = "9fe58ebd5edd976e0fe0f8a14d2a04b7c81ef153ea9a54eebc42e67c2c23b4e5" +dependencies = [ + "pin-project-lite", + "tokio", +] [[package]] name = "ucd-trie" @@ -1604,12 +1388,15 @@ name = "vejas-amqp" version = "0.1.0" dependencies = [ "amiquip", + "async-nats", + "bytes", "crossbeam-channel", - "mio", - "nats", + "futures-util", + "mio 0.6.23", "rustls", "rustls-pemfile", "serde_json", + "tokio", "webpki-roots 0.26.11", ] @@ -1619,12 +1406,6 @@ version = "0.1.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "914b1a6776c4c929a602fafd8bc742e06365d4bcbe48c30f9cca5824f70dc9dd" -[[package]] -name = "version_check" -version = "0.9.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a" - [[package]] name = "walkdir" version = "2.5.0" @@ -1829,26 +1610,6 @@ dependencies = [ "synstructure", ] -[[package]] -name = "zerocopy" -version = "0.8.56" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "556764e583adb45a9f8d413c2a147fa7e8d821e48e12b14fd560b607998b75eb" -dependencies = [ - "zerocopy-derive", -] - -[[package]] -name = "zerocopy-derive" -version = "0.8.56" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f2ab42fc20575779bd240faa45f94a74256f755c0fa9e89f0ede20d91d0cdfc1" -dependencies = [ - "proc-macro2", - "quote", - "syn 2.0.119", -] - [[package]] name = "zerofrom" version = "0.1.8" diff --git a/connectors/amqp/Cargo.toml b/connectors/amqp/Cargo.toml index 4c85196..31569fc 100644 --- a/connectors/amqp/Cargo.toml +++ b/connectors/amqp/Cargo.toml @@ -8,12 +8,17 @@ description = "Vejas AMQP 0-9-1 (RabbitMQ) connector spike: sync pure-Rust via a [dependencies] amiquip = { version = "0.4", default-features = false } crossbeam-channel = "0.5" -nats = "0.25" +# The bus: this crate compiles the runtime's core/src/bus.rs (ADR-0035). +async-nats = { version = "0.52", default-features = false, features = ["jetstream", "kv", "object-store", "ring", "server_2_10"] } +tokio = { version = "1", default-features = false, features = ["rt-multi-thread", "time"] } +bytes = "1" +futures-util = { version = "0.3", default-features = false, features = ["std"] } serde_json = "1" -# Pure-Rust TLS for amqps:// — rustls (already in the tree via nats, ring provider; -# no openssl-sys) wrapped over amiquip's mio-0.6 stream via a small Evented adapter -# (ADR-0026 Q1). This is the whole point of NOT using amiquip's native-tls default. -rustls = "0.22" +# Pure-Rust TLS for amqps:// — rustls (already in the tree via async-nats, ring +# provider; no openssl-sys) wrapped over amiquip's mio-0.6 stream via a small Evented +# adapter (ADR-0026 Q1). This is the whole point of NOT using amiquip's native-tls +# default. +rustls = { version = "0.23", default-features = false, features = ["ring", "std", "tls12", "logging"] } rustls-pemfile = "2" webpki-roots = "0.26" mio = "0.6" diff --git a/connectors/amqp/src/lease.rs b/connectors/amqp/src/lease.rs index 95a425a..691256e 100644 --- a/connectors/amqp/src/lease.rs +++ b/connectors/amqp/src/lease.rs @@ -8,10 +8,10 @@ //! //! Same JetStream KV mechanism as the core runtime (create = acquire, update-CAS = //! renew/fence, delete = release, bucket max_age = crash failover) — reimplemented -//! here because the connector is a standalone binary with its own NATS client, not -//! linked to the core. The bucket is shared (VEJAS_LEASES), keys namespaced, so it +//! here because the connector is a standalone binary, not linked to the core. The bucket is shared (VEJAS_LEASES), keys namespaced, so it //! coordinates with the fleet when they run on the same bus. +use crate::bus; use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::Arc; use std::thread; @@ -56,23 +56,19 @@ pub fn source_key(queue: &str) -> String { /// Open (or create) the shared lease bucket. None if JetStream KV is unavailable — /// the caller then runs unleased (single-instance behaviour, no worse than today). -pub fn open_store(js: &nats::jetstream::JetStream) -> Option { - let cfg = nats::kv::Config { +pub fn open_store(js: &bus::JetStream) -> Option { + js.key_value(&bus::KvConfig { bucket: LEASE_BUCKET.to_string(), max_age: lease_ttl(), history: 1, - ..Default::default() - }; - js.create_key_value(&cfg) - .or_else(|_| js.key_value(LEASE_BUCKET)) - .ok() + }) } /// Held while this instance owns the lease. Dropping it stops renewal and releases /// the lease (instant handoff on shutdown). `lost()` goes true if the renewal /// thread finds the lease fenced or aged out — the caller stops getting. pub struct Lease { - store: nats::kv::Store, + store: bus::Kv, key: String, stop_renew: Arc, lost: Arc, @@ -100,7 +96,7 @@ impl Drop for Lease { /// thread. Returns None if `alive()` goes false before acquisition (asked to stop /// while standing by). pub fn acquire_blocking( - store: &nats::kv::Store, + store: &bus::Kv, key: &str, alive: impl Fn() -> bool, ) -> Option { @@ -167,8 +163,7 @@ mod tests { #[ignore] fn lease_is_mutually_exclusive() { let url = std::env::var("NATS_URL").unwrap_or_else(|_| "127.0.0.1:4222".into()); - let nc = nats::connect(&url).expect("connect"); - let js = nats::jetstream::new(nc); + let js = bus::connect(&url).expect("connect").jetstream(); let store = open_store(&js).expect("lease store"); let key = format!("amqp_test_{}", std::process::id()); let _ = store.delete(&key); @@ -192,8 +187,7 @@ mod tests { #[ignore] fn stale_revision_is_fenced() { let url = std::env::var("NATS_URL").unwrap_or_else(|_| "127.0.0.1:4222".into()); - let nc = nats::connect(&url).expect("connect"); - let js = nats::jetstream::new(nc); + let js = bus::connect(&url).expect("connect").jetstream(); let store = open_store(&js).expect("lease store"); let key = format!("amqp_fence_{}", std::process::id()); let _ = store.delete(&key); diff --git a/connectors/amqp/src/main.rs b/connectors/amqp/src/main.rs index e11f17e..cc6d5c2 100644 --- a/connectors/amqp/src/main.rs +++ b/connectors/amqp/src/main.rs @@ -15,6 +15,11 @@ //! path. The transactional ordering is exercised against a fake below with fault //! injection; real-RabbitMQ certification is the declared CI exception (ADR-0017). +// The runtime's own bus module, compiled in as it is: one sync API over `async-nats` +// on one small tokio runtime, and no NATS type anywhere else (ADR-0035). +#[allow(dead_code)] +#[path = "../../../core/src/bus.rs"] +mod bus; mod lease; mod tls; @@ -77,48 +82,9 @@ pub trait AmqpSink { } // ───────────────────────── loops ───────────────────────── -/// Publish to a JetStream subject and wait for the pub-ack, flushing DIRECTLY past -/// nats 0.25's 5ms flusher floor (client.rs MIN_FLUSH_BETWEEN) — that floor caps -/// every sequential pub-ack at ~200/s, and the AMQP source is a sequential -/// consume→publish→ack loop (the spike measured 186/s, dead on the floor). Faithful -/// re-impl of the crate's request path incl. the no-responders guard + a direct -/// flush; Ok still means a real pub-ack (persisted), so the ack-after-bus-publish -/// ordering is unchanged. (Named bus_* to avoid the AmqpSink::publish_confirmed -/// method, which is the RabbitMQ *broker* confirm.) -fn bus_publish_confirmed( - nc: &nats::Connection, - subject: &str, - payload: &[u8], - wait: Duration, -) -> Result<(), String> { - let reply = nc.new_inbox(); - let sub = nc.subscribe(&reply).map_err(|e| e.to_string())?; - nc.publish_request(subject, &reply, payload) - .map_err(|e| e.to_string())?; - nc.flush().map_err(|e| e.to_string())?; // direct flush — bypass the 5ms floor - let msg = sub - .next_timeout(wait) - .map_err(|_| "timed out waiting for JetStream pub-ack".to_string())?; - if msg.is_no_responders() { - return Err("no JetStream responder for the subject (stream missing?)".into()); - } - let ack: serde_json::Value = - serde_json::from_slice(&msg.data).map_err(|e| format!("bad pub-ack: {e}"))?; - if let Some(err) = ack.get("error") { - return Err(format!("JetStream rejected the publish: {err}")); - } - if ack.get("stream").is_none() { - return Err(format!( - "unexpected pub-ack (no stream): {}", - String::from_utf8_lossy(&msg.data) - )); - } - Ok(()) -} - fn run_source( src: &mut dyn AmqpSource, - nc: &nats::Connection, + js: &bus::JetStream, subject: &str, wait: Duration, alive: impl Fn() -> bool, @@ -126,7 +92,7 @@ fn run_source( eprintln!("[vejas-amqp] source: AMQP → {subject} (consume→publish→ack)"); while alive() { let Some(msg) = src.recv(wait)? else { continue }; - match bus_publish_confirmed(nc, subject, &msg.body, Duration::from_secs(5)) { + match js.publish(subject, &msg.body) { Ok(_) => src.ack(msg.tag)?, // bus durable → ack the broker Err(e) => { eprintln!("[vejas-amqp] publish failed ({e}); nack+requeue"); @@ -139,13 +105,13 @@ fn run_source( fn run_sink( sink: &mut dyn AmqpSink, - sub: &nats::jetstream::PullSubscription, + sub: &bus::Puller, routing_key: &str, alive: impl Fn() -> bool, ) -> Result<(), String> { eprintln!("[vejas-amqp] sink: bus → AMQP (publish→confirm→ack)"); while alive() { - for m in fetch_round(sub)? { + for m in sub.round()? { if !alive() { return Ok(()); } @@ -161,16 +127,6 @@ fn run_sink( Ok(()) } -fn fetch_round(sub: &nats::jetstream::PullSubscription) -> Result, String> { - let iter = sub - .timeout_fetch( - nats::jetstream::BatchOptions { batch: 64, expires: Some(500_000_000), no_wait: true }, - Duration::from_millis(750), - ) - .map_err(|e| e.to_string())?; - Ok(iter.map_while(|m| m.ok()).collect()) -} - // ───────────────────────── amiquip-backed impls ───────────────────────── /// Real source: an amiquip Consumer plus a tag→Delivery map (amiquip acks a /// Delivery by value, so the loop-facing tag API holds the deliveries until acked). @@ -249,14 +205,13 @@ fn main() { let wait = Duration::from_secs(env("VEJAS_AMQP_WAIT_SECS").and_then(|s| s.parse().ok()).unwrap_or(3)); let competing = matches!(env("VEJAS_AMQP_COMPETING").as_deref(), Some("1") | Some("true")); - let nc = match nats::connect(&nats_url) { - Ok(nc) => nc, + let js = match bus::connect(&nats_url) { + Ok(bus) => bus.jetstream(), Err(e) => { eprintln!("[vejas-amqp] cannot connect to NATS at {nats_url}: {e}"); std::process::exit(1); } }; - let js = nats::jetstream::new(nc.clone()); // amqps:// → pure-Rust rustls over amiquip's mio stream (ADR-0026 Q1); amqp:// // → plaintext. Both avoid amiquip's native-tls (openssl-sys) default. @@ -301,7 +256,7 @@ fn main() { None }; let source_alive = || alive() && held.as_ref().map_or(true, |l| !l.lost()); - run_source_amqp(&channel, &nc, &queue, &subject, wait, source_alive) + run_source_amqp(&channel, &js, &queue, &subject, wait, source_alive) } else { run_sink_amqp(&channel, &js, &stream, &subject, &durable, &routing_key) }; @@ -317,7 +272,7 @@ fn main() { fn run_source_amqp( channel: &Channel, - nc: &nats::Connection, + js: &bus::JetStream, queue: &str, subject: &str, wait: Duration, @@ -331,12 +286,12 @@ fn run_source_amqp( .basic_consume(queue, ConsumerOptions::default()) .map_err(|e| e.to_string())?; let mut src = AmqpConsumer { consumer, pending: HashMap::new() }; - run_source(&mut src, nc, subject, wait, alive) + run_source(&mut src, js, subject, wait, alive) } fn run_sink_amqp( channel: &Channel, - js: &nats::jetstream::JetStream, + js: &bus::JetStream, stream: &str, subject: &str, durable: &str, @@ -346,22 +301,11 @@ fn run_sink_amqp( let confirms = channel.listen_for_publisher_confirms().map_err(|e| e.to_string())?; let exchange = Exchange::direct(channel); // Bind (creating if absent) the durable pull consumer on the stream — the - // runtime owns the stream, but pull_subscribe_with_options only BINDS an - // existing durable, so a standalone connector must declare it first (idempotent). - let _ = js.add_consumer( - stream, - nats::jetstream::ConsumerConfig { - durable_name: Some(durable.to_string()), - filter_subject: subject.to_string(), - ..Default::default() - }, - ); - let sub = js - .pull_subscribe_with_options( - subject, - &nats::jetstream::PullSubscribeOptions::new().durable_name(durable.to_string()), - ) - .map_err(|e| e.to_string())?; + // runtime owns the stream, but pull_durable only BINDS an existing durable, + // so a standalone connector must declare it first (idempotent). + let filter = [subject.to_string()]; + let _ = js.create_pull_consumer(stream, durable, &filter, Duration::ZERO, None); + let sub = js.pull_durable(stream, durable, Some(subject))?; let mut sink = AmqpPublisher { exchange, confirms }; run_sink(&mut sink, &sub, routing_key, alive) } diff --git a/connectors/mq/Cargo.lock b/connectors/mq/Cargo.lock index 3ea092c..37001fb 100644 --- a/connectors/mq/Cargo.lock +++ b/connectors/mq/Cargo.lock @@ -12,49 +12,43 @@ dependencies = [ ] [[package]] -name = "async-channel" -version = "2.5.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "924ed96dd52d1b75e9c1a3e6275715fd320f5f9439fb5a4a11fa51f4221158d2" -dependencies = [ - "concurrent-queue", - "event-listener-strategy", - "futures-core", - "pin-project-lite", -] - -[[package]] -name = "async-task" -version = "4.7.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8b75356056920673b02621b35afd0f7dda9306d03c79a30f5c56c44cf256e3de" - -[[package]] -name = "atomic-waker" -version = "1.1.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1505bd5d3d116872e7271a6d4e16d81d0c8570876c8de68093a09ac269d8aac0" - -[[package]] -name = "base64" -version = "0.13.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9e1b586273c5702936fe7b7d6896644d8be71e6314cfe09d3167c95f712589e8" - -[[package]] -name = "base64-url" -version = "1.4.13" +name = "async-nats" +version = "0.52.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "67a99c239d0c7e77c85dddfa9cebce48704b3c49550fcd3b84dd637e4484899f" +checksum = "015f0a7a3b0b29701b78c76a4d4a122624c97c71fa30363381664eb729d619c7" dependencies = [ "base64", + "bytes", + "futures-util", + "memchr", + "pin-project", + "portable-atomic", + "rand", + "regex", + "ring", + "rustls-native-certs", + "rustls-pki-types", + "rustls-webpki", + "serde", + "serde_json", + "serde_nanos", + "serde_repr", + "thiserror", + "time", + "tokio", + "tokio-rustls", + "tokio-stream", + "tokio-util", + "tracing", + "tryhard", + "url", ] [[package]] -name = "base64ct" -version = "1.8.3" +name = "base64" +version = "0.22.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06" +checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6" [[package]] name = "bitflags" @@ -63,25 +57,12 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da" [[package]] -name = "block-buffer" -version = "0.10.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71" -dependencies = [ - "generic-array", -] - -[[package]] -name = "blocking" -version = "1.6.2" +name = "bytes" +version = "1.12.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e83f8d02be6967315521be875afa792a316e28d57b5a2d401897e2a7921b7f21" +checksum = "fc652a48c352aef3ea3aed32080501cf3ef6ed5da78602a020c991775b0aff04" dependencies = [ - "async-channel", - "async-task", - "futures-io", - "futures-lite", - "piper", + "serde", ] [[package]] @@ -101,25 +82,21 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" [[package]] -name = "concurrent-queue" -version = "2.5.0" +name = "chacha20" +version = "0.10.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "4ca0197aee26d1ae37445ee532fefce43251d24cc7c166799f4d46817f1d3973" +checksum = "65c35e4b699c7e15ccbe7ee35c005e4fc0a278d22238a2857e6ce2dadeda1b06" dependencies = [ - "crossbeam-utils", + "cfg-if", + "cpufeatures", + "rand_core", ] -[[package]] -name = "const-oid" -version = "0.9.6" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c2459377285ad874054d797f3ccebf984978aa39129f6eafde5cdc8315b612f8" - [[package]] name = "core-foundation" -version = "0.9.4" +version = "0.10.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "91e195e091a93c46f7102ec7818a2aa394e1e1771c3ab4825963fa03e45afb8f" +checksum = "b2a6cd9ae233e7f62ba4e9353e81a88df7fc8a5987b8d445b4d90c879bd156f6" dependencies = [ "core-foundation-sys", "libc", @@ -133,81 +110,13 @@ checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b" [[package]] name = "cpufeatures" -version = "0.2.17" +version = "0.3.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280" +checksum = "5ca28b0ae3115b884660db4118d803791fd6756b6e88f39c0f3f7859060d7566" dependencies = [ "libc", ] -[[package]] -name = "crossbeam-channel" -version = "0.5.16" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d85363c37faeca707aef026efa9f3b34d077bce547e48f770770625c6013679e" -dependencies = [ - "crossbeam-utils", -] - -[[package]] -name = "crossbeam-utils" -version = "0.8.22" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "61803da095bee82a81bb1a452ecc25d3b2f1416d1897eb86430c6159ef717c17" - -[[package]] -name = "crypto-common" -version = "0.1.7" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a" -dependencies = [ - "generic-array", - "typenum", -] - -[[package]] -name = "curve25519-dalek" -version = "4.1.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "97fb8b7c4503de7d6ae7b42ab72a5a59857b4c937ec27a3d4539dba95b5ab2be" -dependencies = [ - "cfg-if", - "cpufeatures", - "curve25519-dalek-derive", - "digest", - "fiat-crypto", - "rustc_version", - "subtle", -] - -[[package]] -name = "curve25519-dalek-derive" -version = "0.1.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f46882e17999c6cc590af592290432be3bce0428cb0d5f8b6715e4dc7b383eb3" -dependencies = [ - "proc-macro2", - "quote", - "syn 2.0.119", -] - -[[package]] -name = "data-encoding" -version = "2.11.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "4583a4551df46e2792f82ceeac45e850d2e2d5debba0b91f102385cda5b11f06" - -[[package]] -name = "der" -version = "0.7.10" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e7c1832837b905bbfb5101e07cc24c8deddf52f93225eee6ead5f4d63d53ddcb" -dependencies = [ - "const-oid", - "pem-rfc7468", - "zeroize", -] - [[package]] name = "deranged" version = "0.5.8" @@ -217,16 +126,6 @@ dependencies = [ "serde_core", ] -[[package]] -name = "digest" -version = "0.10.7" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292" -dependencies = [ - "block-buffer", - "crypto-common", -] - [[package]] name = "displaydoc" version = "0.2.7" @@ -238,69 +137,6 @@ dependencies = [ "syn 3.0.3", ] -[[package]] -name = "ed25519" -version = "2.2.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "115531babc129696a58c64a4fef0a8bf9e9698629fb97e9e40767d235cfbcd53" -dependencies = [ - "signature", -] - -[[package]] -name = "ed25519-dalek" -version = "2.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "70e796c081cee67dc755e1a36a0a172b897fab85fc3f6bc48307991f64e4eca9" -dependencies = [ - "curve25519-dalek", - "ed25519", - "sha2", - "signature", - "subtle", -] - -[[package]] -name = "event-listener" -version = "5.4.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5a23add41df1562121a9393cb065eab5146a1242410f23a644851e90cfd669d2" -dependencies = [ - "parking", - "pin-project-lite", -] - -[[package]] -name = "event-listener-strategy" -version = "0.5.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8be9f3dfaaffdae2972880079a491a1a8bb7cbed0b8dd7a347f668b4150a3b93" -dependencies = [ - "event-listener", - "pin-project-lite", -] - -[[package]] -name = "fastrand" -version = "1.9.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e51093e27b0797c359783294ca4f0a911c270184cb10f85783b118614a1501be" -dependencies = [ - "instant", -] - -[[package]] -name = "fastrand" -version = "2.5.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "da7c62ceae207dd37ea5b845da6a0696c799f85e97da1ab5b7910be3c1c80223" - -[[package]] -name = "fiat-crypto" -version = "0.2.9" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "28dea519a9695b9977216879a3ebfddf92f1c08c05d984f8996aecd6ecdc811d" - [[package]] name = "find-msvc-tools" version = "0.1.11" @@ -323,40 +159,51 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "92d699e522242e69e3003b94ecc1f960f3a5e015aa7c5d7486e65ad01dd94f5e" [[package]] -name = "futures-io" +name = "futures-sink" version = "0.3.34" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "53c0fa8157de1303bfffdaa1cc2a673bfffb60102f76b0ef4441659124373fed" +checksum = "1944426bf7d03f1d14f708785e4b33efd750b36d48a157b836b3efc15ede8e1d" [[package]] -name = "futures-lite" -version = "2.6.1" +name = "futures-task" +version = "0.3.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cd417de3d1d015fc3bfd2b1ea46dfc7bab72ef86f1cc7cc9c78e728b34a6d1fd" + +[[package]] +name = "futures-util" +version = "0.3.34" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f78e10609fe0e0b3f4157ffab1876319b5b0db102a2c60dc4626306dc46b44ad" +checksum = "0d50a92467f8ba5dd6e3ee5d4bd04d73ab2e4e1c44474a0674821dfce14b79bc" dependencies = [ "futures-core", + "futures-sink", + "futures-task", "pin-project-lite", + "slab", ] [[package]] -name = "generic-array" -version = "0.14.7" +name = "getrandom" +version = "0.2.17" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a" +checksum = "ff2abc00be7fca6ebc474524697ae276ad847ad0a6b3faa4bcb027e9a4614ad0" dependencies = [ - "typenum", - "version_check", + "cfg-if", + "libc", + "wasi", ] [[package]] name = "getrandom" -version = "0.2.17" +version = "0.4.3" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ff2abc00be7fca6ebc474524697ae276ad847ad0a6b3faa4bcb027e9a4614ad0" +checksum = "300e883d756b2e4ec94e02791f39b04b522276138852cfc41d9fb7e904106099" dependencies = [ "cfg-if", "libc", - "wasi", + "r-efi", + "rand_core", ] [[package]] @@ -463,33 +310,12 @@ dependencies = [ "icu_properties", ] -[[package]] -name = "instant" -version = "0.1.13" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e0242819d153cba4b4b05a5a8f2a7e9bbf97b6055b2a002b395c96b5ff3c0222" -dependencies = [ - "cfg-if", -] - [[package]] name = "itoa" version = "1.0.18" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682" -[[package]] -name = "json" -version = "0.12.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "078e285eafdfb6c4b434e0d31e8cfcb5115b651496faca5749b88fafd4f23bfd" - -[[package]] -name = "lazy_static" -version = "1.5.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe" - [[package]] name = "libc" version = "0.2.189" @@ -502,21 +328,6 @@ version = "0.8.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "47d9d19d1d6efa0109d2f65ff4c85cddd50bd572e5a00127ab10987290bcefae" -[[package]] -name = "lock_api" -version = "0.4.14" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "224399e74b87b5f3557511d98dff8b14089b3dadafcab6bb93eab67d3aace965" -dependencies = [ - "scopeguard", -] - -[[package]] -name = "log" -version = "0.4.34" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f9f8bd3e56ce4dfc153cf470fffbfa98c7620958b312ca5c3a4b8d5181fd13c6" - [[package]] name = "memchr" version = "2.8.3" @@ -524,65 +335,14 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98" [[package]] -name = "nats" -version = "0.25.0" +name = "mio" +version = "1.2.4" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "19090dd06e27eb59adfd16f332b85818a3e7dbbdbc13c448676c9bb69f964208" +checksum = "1788edb87fdc09c7e26304471e2f5be8cdefb1b6930d6e3985fc02ff53bf86ee" dependencies = [ - "base64", - "base64-url", - "blocking", - "crossbeam-channel", - "fastrand 1.9.0", - "itoa", - "json", - "lazy_static", "libc", - "log", - "memchr", - "nkeys", - "nuid", - "once_cell", - "parking_lot", - "portable-atomic", - "regex", - "ring", - "rustls", - "rustls-native-certs", - "rustls-pemfile", - "rustls-webpki", - "serde", - "serde_json", - "serde_nanos", - "serde_repr", - "time", - "url", - "winapi", -] - -[[package]] -name = "nkeys" -version = "0.4.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "879011babc47a1c7fdf5a935ae3cfe94f34645ca0cac1c7f6424b36fc743d1bf" -dependencies = [ - "data-encoding", - "ed25519", - "ed25519-dalek", - "getrandom", - "log", - "rand", - "signatory", -] - -[[package]] -name = "nuid" -version = "0.3.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "20c1bb65186718d348306bf1afdeb20d9ab45b2ab80fb793c0fdcf59ffbb4f38" -dependencies = [ - "lazy_static", - "rand", + "wasi", + "windows-sys 0.61.2", ] [[package]] @@ -599,81 +359,42 @@ checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" [[package]] name = "openssl-probe" -version = "0.1.6" +version = "0.2.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d05e27ee213611ffe7d6348b942e8f942b37114c00cc03cec254295a4a17852e" +checksum = "7e036b411754132fd6b74e94b74e9a13c5cf606fa93d1885c35860d3060820b1" [[package]] -name = "parking" -version = "2.2.1" +name = "percent-encoding" +version = "2.3.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f38d5652c16fde515bb1ecef450ab0f6a219d619a7274976324d5e377f7dceba" +checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220" [[package]] -name = "parking_lot" -version = "0.12.5" +name = "pin-project" +version = "1.1.13" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "93857453250e3077bd71ff98b6a65ea6621a19bb0f559a85248955ac12c45a1a" +checksum = "2466b2336ed02bcdca6b294417127b90ec92038d1d5c4fbeac971a922e0e0924" dependencies = [ - "lock_api", - "parking_lot_core", + "pin-project-internal", ] [[package]] -name = "parking_lot_core" -version = "0.9.12" +name = "pin-project-internal" +version = "1.1.13" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2621685985a2ebf1c516881c026032ac7deafcda1a2c9b7850dc81e3dfcb64c1" +checksum = "c96395f0a926bc13b1c17622aaddda1ecb55d49c8f1bf9777e4d877800a43f8b" dependencies = [ - "cfg-if", - "libc", - "redox_syscall", - "smallvec", - "windows-link", -] - -[[package]] -name = "pem-rfc7468" -version = "0.7.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "88b39c9bfcfc231068454382784bb460aae594343fb030d46e9f50a645418412" -dependencies = [ - "base64ct", + "proc-macro2", + "quote", + "syn 2.0.119", ] -[[package]] -name = "percent-encoding" -version = "2.3.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220" - [[package]] name = "pin-project-lite" version = "0.2.17" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd" -[[package]] -name = "piper" -version = "0.2.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c835479a4443ded371d6c535cbfd8d31ad92c5d23ae9770a61bc155e4992a3c1" -dependencies = [ - "atomic-waker", - "fastrand 2.5.0", - "futures-io", -] - -[[package]] -name = "pkcs8" -version = "0.10.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f950b2377845cebe5cf8b5165cb3cc1a5e0fa5cfa3e1f7f55707d8fd82e0a7b7" -dependencies = [ - "der", - "spki", -] - [[package]] name = "portable-atomic" version = "1.15.0" @@ -695,15 +416,6 @@ version = "0.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "439ee305def115ba05938db6eb1644ff94165c5ab5e9420d1c1bcedbba909391" -[[package]] -name = "ppv-lite86" -version = "0.2.21" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "85eae3c4ed2f50dcfe72643da4befc30deadb458a9b590d720cde2f2b1e97da9" -dependencies = [ - "zerocopy", -] - [[package]] name = "proc-macro2" version = "1.0.107" @@ -723,43 +435,27 @@ dependencies = [ ] [[package]] -name = "rand" -version = "0.8.7" +name = "r-efi" +version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "22f6172bdec972074665ed81ed53b71da00bfc44b65a753cfde883ec4c702a1a" -dependencies = [ - "libc", - "rand_chacha", - "rand_core", -] +checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf" [[package]] -name = "rand_chacha" -version = "0.3.1" +name = "rand" +version = "0.10.3" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e6c10a63a0fa32252be49d21e7709d4d4baf8d231c2dbce1eaa8141b9b127d88" +checksum = "65c9fb96cbc91e3478eaae79a69fcd3f1ae4ad052e471fe6732fff548984b4af" dependencies = [ - "ppv-lite86", + "chacha20", + "getrandom 0.4.3", "rand_core", ] [[package]] name = "rand_core" -version = "0.6.4" +version = "0.10.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ec0be4795e2f6a28069bec0b5ff3e2ac9bafc99e6a9a7dc3547996c5c816922c" -dependencies = [ - "getrandom", -] - -[[package]] -name = "redox_syscall" -version = "0.5.18" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d" -dependencies = [ - "bitflags", -] +checksum = "63b8176103e19a2643978565ca18b50549f6101881c443590420e4dc998a3c69" [[package]] name = "regex" @@ -798,28 +494,19 @@ checksum = "a4689e6c2294d81e88dc6261c768b63bc4fcdb852be6d1352498b114f61383b7" dependencies = [ "cc", "cfg-if", - "getrandom", + "getrandom 0.2.17", "libc", "untrusted", "windows-sys 0.52.0", ] -[[package]] -name = "rustc_version" -version = "0.4.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "cfcb3a22ef46e85b45de6ee7e79d063319ebb6594faafcf1c225ea92ab6e9b92" -dependencies = [ - "semver", -] - [[package]] name = "rustls" -version = "0.22.4" +version = "0.23.46" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "bf4ef73721ac7bcd79b2b315da7779d8fc09718c6b3d2d1b2d94850eb8c18432" +checksum = "48e13bd8c0e9365c43cfa5c9e8f9ad49d3c8444926c9aac819e0e4dc503c8fdf" dependencies = [ - "log", + "once_cell", "ring", "rustls-pki-types", "rustls-webpki", @@ -829,26 +516,16 @@ dependencies = [ [[package]] name = "rustls-native-certs" -version = "0.7.3" +version = "0.8.5" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e5bfb394eeed242e909609f56089eecfe5fda225042e8b171791b9c95f5931e5" +checksum = "4a073f5dc7469f984c52ad2752b63b0807745133b6de880b7b64c1ac4c48aec4" dependencies = [ "openssl-probe", - "rustls-pemfile", "rustls-pki-types", "schannel", "security-framework", ] -[[package]] -name = "rustls-pemfile" -version = "2.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "dce314e5fee3f39953d46bb63bb8a46d40c2f8fb7cc5a3b6cab2bde9721d6e50" -dependencies = [ - "rustls-pki-types", -] - [[package]] name = "rustls-pki-types" version = "1.15.1" @@ -860,9 +537,9 @@ dependencies = [ [[package]] name = "rustls-webpki" -version = "0.102.8" +version = "0.103.16" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "64ca1bc8749bd4cf37b5ce386cc146580777b4e8572c7b97baf22c83f444bee9" +checksum = "4443b1b6d39d7c663139942cc4558262562d2c84123bb6282ef177009ca8b281" dependencies = [ "ring", "rustls-pki-types", @@ -878,17 +555,11 @@ dependencies = [ "windows-sys 0.61.2", ] -[[package]] -name = "scopeguard" -version = "1.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49" - [[package]] name = "security-framework" -version = "2.11.1" +version = "3.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "897b2245f0b511c87893af39b033e5ca9cce68824c4d7e7630b5a1d339658d02" +checksum = "b7f4bc775c73d9a02cde8bf7b2ec4c9d12743edf609006c7facc23998404cd1d" dependencies = [ "bitflags", "core-foundation", @@ -907,12 +578,6 @@ dependencies = [ "libc", ] -[[package]] -name = "semver" -version = "1.0.28" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8a7852d02fc848982e0c167ef163aaff9cd91dc640ba85e263cb1ce46fae51cd" - [[package]] name = "serde" version = "1.0.229" @@ -976,17 +641,6 @@ dependencies = [ "syn 3.0.3", ] -[[package]] -name = "sha2" -version = "0.10.9" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283" -dependencies = [ - "cfg-if", - "cpufeatures", - "digest", -] - [[package]] name = "shlex" version = "2.0.1" @@ -994,26 +648,10 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba" [[package]] -name = "signatory" -version = "0.27.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c1e303f8205714074f6068773f0e29527e0453937fe837c9717d066635b65f31" -dependencies = [ - "pkcs8", - "rand_core", - "signature", - "zeroize", -] - -[[package]] -name = "signature" -version = "2.2.0" +name = "slab" +version = "0.4.12" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "77549399552de45a898a580c1b41d445bf730df867cc44e6c0233bbc4b8329de" -dependencies = [ - "digest", - "rand_core", -] +checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5" [[package]] name = "smallvec" @@ -1022,13 +660,13 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8ed6a63f02c8539c91a8685a86f4099661ba3da017932f6ebbea6de3f0fa7c90" [[package]] -name = "spki" -version = "0.7.3" +name = "socket2" +version = "0.6.5" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d91ed6c858b01f942cd56b37a94b3e0a1798290327d1236e4d9cf4eaca44d29d" +checksum = "c3d1e2c7f27f8d4cb10542a02c49005dbd6e93095799d6f3be745fae9f8fedd4" dependencies = [ - "base64ct", - "der", + "libc", + "windows-sys 0.61.2", ] [[package]] @@ -1076,6 +714,26 @@ dependencies = [ "syn 2.0.119", ] +[[package]] +name = "thiserror" +version = "2.0.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "09e52cb86a36cede5cb101bf8908837b3e4c6e5e59fe7fd85c23fb56200d189e" +dependencies = [ + "thiserror-impl", +] + +[[package]] +name = "thiserror-impl" +version = "2.0.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fe5197923287db20a58125f0bc85c062f7f2c892de97b18c356f9efb14b28524" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.3", +] + [[package]] name = "time" version = "0.3.55" @@ -1117,10 +775,105 @@ dependencies = [ ] [[package]] -name = "typenum" -version = "1.20.1" +name = "tokio" +version = "1.54.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b52fd3255258d26fac7264ab5a76b53c13367136dc011d3d6ad4f8ed070396a3" +dependencies = [ + "bytes", + "libc", + "mio", + "pin-project-lite", + "socket2", + "tokio-macros", + "windows-sys 0.61.2", +] + +[[package]] +name = "tokio-macros" +version = "2.7.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "78773a2a397f451582ce068015985c33193cf6dea8b74d2a639fe457b2f07b0e" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.3", +] + +[[package]] +name = "tokio-rustls" +version = "0.26.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c9cc2678c2cdd569ef8215e2afd7954ada2ae20b4fdd2c5fe6139a3b02d105db" +dependencies = [ + "rustls", + "tokio", +] + +[[package]] +name = "tokio-stream" +version = "0.1.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a3d06f0b082ba57c26b79407372e57cf2a1e28124f78e9479fe80322cf53420b" +dependencies = [ + "futures-core", + "pin-project-lite", + "tokio", +] + +[[package]] +name = "tokio-util" +version = "0.7.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e464cf451ba96ebfc6f9b6542f17ee8b8956e33f1e40d9690624e59d7a7f8a4b" +dependencies = [ + "bytes", + "futures-core", + "futures-sink", + "pin-project-lite", + "tokio", +] + +[[package]] +name = "tracing" +version = "0.1.44" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100" +dependencies = [ + "pin-project-lite", + "tracing-attributes", + "tracing-core", +] + +[[package]] +name = "tracing-attributes" +version = "0.1.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7490cfa5ec963746568740651ac6781f701c9c5ea257c58e057f3ba8cf69e8da" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "tracing-core" +version = "0.1.36" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b6f5e870be6c3b371b77fe0ee0bafb859fa4964b4404c27de1d380043c4dda20" +checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a" +dependencies = [ + "once_cell", +] + +[[package]] +name = "tryhard" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9fe58ebd5edd976e0fe0f8a14d2a04b7c81ef153ea9a54eebc42e67c2c23b4e5" +dependencies = [ + "pin-project-lite", + "tokio", +] [[package]] name = "unicode-ident" @@ -1156,44 +909,19 @@ checksum = "b6c140620e7ffbb22c2dee59cafe6084a59b5ffc27a8859a5f0d494b5d52b6be" name = "vejas-mq" version = "0.1.0" dependencies = [ - "nats", + "async-nats", + "bytes", + "futures-util", "serde_json", + "tokio", ] -[[package]] -name = "version_check" -version = "0.9.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a" - [[package]] name = "wasi" version = "0.11.1+wasi-snapshot-preview1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b" -[[package]] -name = "winapi" -version = "0.3.9" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5c839a674fcd7a98952e593242ea400abe93992746761e38641405d28b00f419" -dependencies = [ - "winapi-i686-pc-windows-gnu", - "winapi-x86_64-pc-windows-gnu", -] - -[[package]] -name = "winapi-i686-pc-windows-gnu" -version = "0.4.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6" - -[[package]] -name = "winapi-x86_64-pc-windows-gnu" -version = "0.4.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f" - [[package]] name = "windows-link" version = "0.2.1" @@ -1311,26 +1039,6 @@ dependencies = [ "synstructure", ] -[[package]] -name = "zerocopy" -version = "0.8.56" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "556764e583adb45a9f8d413c2a147fa7e8d821e48e12b14fd560b607998b75eb" -dependencies = [ - "zerocopy-derive", -] - -[[package]] -name = "zerocopy-derive" -version = "0.8.56" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f2ab42fc20575779bd240faa45f94a74256f755c0fa9e89f0ede20d91d0cdfc1" -dependencies = [ - "proc-macro2", - "quote", - "syn 2.0.119", -] - [[package]] name = "zerofrom" version = "0.1.8" diff --git a/connectors/mq/Cargo.toml b/connectors/mq/Cargo.toml index 6067b08..c5eed03 100644 --- a/connectors/mq/Cargo.toml +++ b/connectors/mq/Cargo.toml @@ -8,7 +8,8 @@ description = "Vejas IBM MQ connector: a first-class bus citizen (ADR-0023). Own # NOT a member of the runtime workspace, and — unlike the SAP/Salesforce exec # children — this one holds its OWN NATS client (ADR-0023): a destructive MQGET # under syncpoint may only MQCMIT *after* the bus publish is confirmed, and a -# stdout-only child cannot know the bus confirmed. So it needs `nats` directly. +# stdout-only child cannot know the bus confirmed. So it needs the bus directly: it +# compiles the runtime's own core/src/bus.rs (ADR-0035), hence the NATS client below. # # The IBM MQ C client (libmqic_r.so) is dlopen'd at RUNTIME (like libsapnwrfc, # ADR-0014) — there is NO build-time MQ dependency, so this crate builds anywhere. @@ -17,7 +18,10 @@ description = "Vejas IBM MQ connector: a first-class bus citizen (ADR-0023). Own # via the standard MQSERVER / CCDT environment (no MQCD struct needed). [dependencies] -nats = "0.25" +async-nats = { version = "0.52", default-features = false, features = ["jetstream", "kv", "object-store", "ring", "server_2_10"] } +tokio = { version = "1", default-features = false, features = ["rt-multi-thread", "time"] } +bytes = "1" +futures-util = { version = "0.3", default-features = false, features = ["std"] } serde_json = "1" [[bin]] diff --git a/connectors/mq/src/lease.rs b/connectors/mq/src/lease.rs index dde76d3..d6bcba4 100644 --- a/connectors/mq/src/lease.rs +++ b/connectors/mq/src/lease.rs @@ -8,10 +8,10 @@ //! //! Same JetStream KV mechanism as the core runtime (create = acquire, update-CAS = //! renew/fence, delete = release, bucket max_age = crash failover) — reimplemented -//! here because the connector is a standalone binary with its own NATS client, not -//! linked to the core. The bucket is shared (VEJAS_LEASES), keys namespaced, so it +//! here because the connector is a standalone binary, not linked to the core. The bucket is shared (VEJAS_LEASES), keys namespaced, so it //! coordinates with the fleet when they run on the same bus. +use crate::bus; use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::Arc; use std::thread; @@ -56,23 +56,19 @@ pub fn source_key(queue: &str) -> String { /// Open (or create) the shared lease bucket. None if JetStream KV is unavailable — /// the caller then runs unleased (single-instance behaviour, no worse than today). -pub fn open_store(js: &nats::jetstream::JetStream) -> Option { - let cfg = nats::kv::Config { +pub fn open_store(js: &bus::JetStream) -> Option { + js.key_value(&bus::KvConfig { bucket: LEASE_BUCKET.to_string(), max_age: lease_ttl(), history: 1, - ..Default::default() - }; - js.create_key_value(&cfg) - .or_else(|_| js.key_value(LEASE_BUCKET)) - .ok() + }) } /// Held while this instance owns the lease. Dropping it stops renewal and releases /// the lease (instant handoff on shutdown). `lost()` goes true if the renewal /// thread finds the lease fenced or aged out — the caller stops getting. pub struct Lease { - store: nats::kv::Store, + store: bus::Kv, key: String, stop_renew: Arc, lost: Arc, @@ -100,7 +96,7 @@ impl Drop for Lease { /// thread. Returns None if `alive()` goes false before acquisition (asked to stop /// while standing by). pub fn acquire_blocking( - store: &nats::kv::Store, + store: &bus::Kv, key: &str, alive: impl Fn() -> bool, ) -> Option { @@ -167,8 +163,7 @@ mod tests { #[ignore] fn lease_is_mutually_exclusive() { let url = std::env::var("NATS_URL").unwrap_or_else(|_| "127.0.0.1:4222".into()); - let nc = nats::connect(&url).expect("connect"); - let js = nats::jetstream::new(nc); + let js = bus::connect(&url).expect("connect").jetstream(); let store = open_store(&js).expect("lease store"); let key = format!("mq_test_{}", std::process::id()); let _ = store.delete(&key); @@ -192,8 +187,7 @@ mod tests { #[ignore] fn stale_revision_is_fenced() { let url = std::env::var("NATS_URL").unwrap_or_else(|_| "127.0.0.1:4222".into()); - let nc = nats::connect(&url).expect("connect"); - let js = nats::jetstream::new(nc); + let js = bus::connect(&url).expect("connect").jetstream(); let store = open_store(&js).expect("lease store"); let key = format!("mq_fence_{}", std::process::id()); let _ = store.delete(&key); diff --git a/connectors/mq/src/main.rs b/connectors/mq/src/main.rs index 5942b51..74b534d 100644 --- a/connectors/mq/src/main.rs +++ b/connectors/mq/src/main.rs @@ -17,6 +17,11 @@ //! broker with fault injection; real-queue-manager certification is the declared //! CI exception (ADR-0023, like SAP/ADR-0017). +// The runtime's own bus module, compiled in as it is: one sync API over `async-nats` +// on one small tokio runtime, and no NATS type anywhere else (ADR-0035). +#[allow(dead_code)] +#[path = "../../../core/src/bus.rs"] +mod bus; mod lease; mod mqi; @@ -86,50 +91,13 @@ fn load_config() -> Result { } // ───────────────────────── source: MQ → bus ───────────────────────── -/// Publish and wait for the JetStream pub-ack, flushing DIRECTLY past nats 0.25's -/// 5ms flusher floor (client.rs MIN_FLUSH_BETWEEN) — that floor caps every -/// sequential pub-ack at ~200/s, and this get→publish→commit loop is exactly that. -/// Reimplements the crate's request path (inbox, buffered publish-with-reply, -/// no-responders guard) + a direct flush; Ok still means a real pub-ack (persisted), -/// so the at-least-once ordering (publish-before-MQCMIT) is unchanged. -fn publish_confirmed( - nc: &nats::Connection, - subject: &str, - payload: &[u8], - wait: Duration, -) -> Result<(), String> { - let reply = nc.new_inbox(); - let sub = nc.subscribe(&reply).map_err(|e| e.to_string())?; - nc.publish_request(subject, &reply, payload) - .map_err(|e| e.to_string())?; - nc.flush().map_err(|e| e.to_string())?; // direct flush — bypass the 5ms floor - let msg = sub - .next_timeout(wait) - .map_err(|_| "timed out waiting for JetStream pub-ack".to_string())?; - if msg.is_no_responders() { - return Err("no JetStream responder for the subject (stream missing?)".into()); - } - let ack: serde_json::Value = - serde_json::from_slice(&msg.data).map_err(|e| format!("bad pub-ack: {e}"))?; - if let Some(err) = ack.get("error") { - return Err(format!("JetStream rejected the publish: {err}")); - } - if ack.get("stream").is_none() { - return Err(format!( - "unexpected pub-ack (no stream): {}", - String::from_utf8_lossy(&msg.data) - )); - } - Ok(()) -} - /// Get under syncpoint, publish to the bus awaiting the JetStream pub-ack, then — /// and only then — MQCMIT. Any failure before the commit is an MQBACK, so the /// message stays on the queue and is re-got: at-least-once, no loss. The broker is /// the durable cursor (no offset KV — contrast Kafka/ADR-0022). fn run_source( broker: &mut dyn Broker, - nc: &nats::Connection, + js: &bus::JetStream, subject: &str, wait: Duration, alive: impl Fn() -> bool, @@ -141,7 +109,7 @@ fn run_source( .get_syncpoint(wait) .map_err(|e| format!("get: {e}"))?; let Some(msg) = got else { continue }; // idle: no message within the wait - match publish_confirmed(nc, subject, &msg.body, Duration::from_secs(5)) { + match js.publish(subject, &msg.body) { Ok(_) => { // JetStream pub-ack received (message is durable on the bus) → // safe to remove it from MQ. @@ -176,13 +144,16 @@ fn run_source( /// CorrelId (VEJAS_MQ_DEDUP_FIELD) lets a downstream consumer dedup. fn run_sink( broker: &mut dyn Broker, - sub: &nats::jetstream::PullSubscription, + sub: &bus::Puller, dedup_field: Option<&str>, alive: impl Fn() -> bool, ) -> Result<(), String> { eprintln!("[vejas-mq] sink: bus → MQ (put→commit→ack)"); while alive() { - for m in fetch_round(sub)? { + // One bounded round as a continuous long-poll: messages the instant they + // arrive, else the pull is held server-side for the window and comes back + // empty (no client sleep, no coverage gap, the shape the runtime uses). + for m in sub.round()? { if !alive() { return Ok(()); // stopped mid-batch: leave un-acked, it redelivers } @@ -190,7 +161,7 @@ fn run_sink( .and_then(|f| dedup_key(&m.data, f)) .map(|k| mqi::correlid_from_key(&k)) .unwrap_or_default(); - let out = MqMessage { body: m.data.clone(), correlid }; + let out = MqMessage { body: m.data.to_vec(), correlid }; match broker.put_syncpoint(&out) { Ok(()) => match broker.commit() { Ok(()) => { @@ -211,21 +182,6 @@ fn run_sink( Ok(()) } -/// Pull one bounded batch as a continuous long-poll: messages the instant they -/// arrive, else the pull is held server-side for the window and returns empty (no -/// client sleep, no coverage gap — the same shape the core runtime uses). -fn fetch_round( - sub: &nats::jetstream::PullSubscription, -) -> Result, String> { - let iter = sub - .timeout_fetch( - nats::jetstream::BatchOptions { batch: 64, expires: Some(500_000_000), no_wait: true }, - Duration::from_millis(750), - ) - .map_err(|e| e.to_string())?; - Ok(iter.map_while(|m| m.ok()).collect()) -} - /// Pull a stringified idempotency key out of the JSON body at a top-level field. fn dedup_key(body: &[u8], field: &str) -> Option { let v: serde_json::Value = serde_json::from_slice(body).ok()?; @@ -251,14 +207,13 @@ fn main() { ); } - let nc = match nats::connect(&cfg.nats_url) { - Ok(nc) => nc, + let js = match bus::connect(&cfg.nats_url) { + Ok(bus) => bus.jetstream(), Err(e) => { eprintln!("[vejas-mq] cannot connect to NATS at {}: {e}", cfg.nats_url); std::process::exit(1); } }; - let js = nats::jetstream::new(nc.clone()); let result = if cfg.mode == "source" { // Singleton by default (ADR-0023, "because order"): take the lease so one @@ -282,27 +237,16 @@ fn main() { // stop getting if the lease is lost (fenced / aged out) as well as on signal let source_alive = || alive() && held.as_ref().map_or(true, |l| !l.lost()); MqiQueue::open(&cfg.qmgr, &cfg.queue, false) - .and_then(|mut q| run_source(&mut q, &nc, &cfg.subject, cfg.wait, source_alive)) + .and_then(|mut q| run_source(&mut q, &js, &cfg.subject, cfg.wait, source_alive)) } else { // durable pull consumer, competing-safe by its own durable (SUBJECTS.md). - // pull_subscribe_with_options only BINDS an existing durable — the runtime - // owns the stream, but a standalone connector must declare the consumer - // first (idempotent), or binding 404s. VEJAS_STREAM names the stream. + // pull_durable only BINDS an existing durable — the runtime owns the + // stream, but a standalone connector must declare the consumer first + // (idempotent), or binding 404s. VEJAS_STREAM names the stream. let stream = env_or("VEJAS_STREAM", "VEJAS"); - let _ = js.add_consumer( - &stream, - nats::jetstream::ConsumerConfig { - durable_name: Some(cfg.durable.clone()), - filter_subject: cfg.subject.clone(), - ..Default::default() - }, - ); - let sub = js - .pull_subscribe_with_options( - &cfg.subject, - &nats::jetstream::PullSubscribeOptions::new().durable_name(cfg.durable.clone()), - ) - .map_err(|e| e.to_string()); + let filter = [cfg.subject.clone()]; + let _ = js.create_pull_consumer(&stream, &cfg.durable, &filter, Duration::ZERO, None); + let sub = js.pull_durable(&stream, &cfg.durable, Some(&cfg.subject)); match (MqiQueue::open(&cfg.qmgr, &cfg.queue, true), sub) { (Ok(mut q), Ok(sub)) => run_sink(&mut q, &sub, cfg.dedup_field.as_deref(), alive), (Err(e), _) | (_, Err(e)) => Err(e), @@ -436,20 +380,20 @@ mod tests { #[ignore] fn source_publishes_to_bus_e2e() { let url = std::env::var("NATS_URL").unwrap_or_else(|_| "127.0.0.1:4222".into()); - let nc = nats::connect(&url).expect("connect"); - let js = nats::jetstream::new(nc.clone()); + let nc = bus::connect(&url).expect("connect"); + let js = nc.jetstream(); let pid = std::process::id(); let stream = format!("MQPROBE_{pid}"); let subject = format!("mqprobe.{pid}.in"); let _ = js.delete_stream(&stream); - js.add_stream(nats::jetstream::StreamConfig { + js.add_stream(&bus::StreamConfig { name: stream.clone(), subjects: vec![format!("mqprobe.{pid}.>")], ..Default::default() }) .expect("add_stream"); // a plain subscriber to observe what lands on the bus - let sub = nc.subscribe(&subject).expect("sub"); + let mut sub = nc.subscribe(&subject).expect("sub"); let mut broker = FakeBroker::with(&["a1", "b2", "c3"]); // stop once the queue has drained (get returns None) — bounded, no globals. @@ -468,7 +412,7 @@ mod tests { fn backout(&mut self) -> Result<(), mqi::MqError> { self.inner.backout() } } let mut draining = Draining { inner: std::mem::take(&mut broker), flag: &seen_empty }; - run_source(&mut draining, &nc, &subject, Duration::from_millis(50), alive).expect("source"); + run_source(&mut draining, &js, &subject, Duration::from_millis(50), alive).expect("source"); let mut got = Vec::new(); while let Ok(m) = sub.next_timeout(Duration::from_millis(300)) { diff --git a/docs/COOKBOOK.md b/docs/COOKBOOK.md index 99786b9..6f6c08c 100644 --- a/docs/COOKBOOK.md +++ b/docs/COOKBOOK.md @@ -175,7 +175,7 @@ destructive, so competing instances never duplicate. > vault) and put every message on the bus as `vx.amqp.orders.in`. Like IBM MQ, a **first-class standalone binary** (`connectors/amqp`) — pure -Rust, synchronous, no tokio, and TLS through rustls (no OpenSSL anywhere). +Rust, a synchronous broker client, and TLS through rustls (no OpenSSL anywhere). The at-least-once contract is the platform's, spoken in AMQP: the source acks RabbitMQ only after the bus pub-ack; the sink acks the bus only after the publisher confirm. The agent fills the env recipe diff --git a/docs/SUBJECTS.md b/docs/SUBJECTS.md index 4bec851..25bc51c 100644 --- a/docs/SUBJECTS.md +++ b/docs/SUBJECTS.md @@ -37,7 +37,7 @@ hot-addable). No subprocess, no Python. Drivers today: MQI FFI, `dlopen` of the MQ client at runtime — builds with no MQ present. Configured by env (`VEJAS_MQ_*`, see the recipe), not by a manifest. - **amqp source/sink** (standalone binary, `connectors/amqp`) — RabbitMQ / - AMQP 0-9-1 as a first-class citizen (ADR-0026): pure Rust, sync, no tokio, + AMQP 0-9-1 as a first-class citizen (ADR-0026): pure Rust, a sync broker client, TLS via rustls over amiquip's mio loop (no OpenSSL). Source acks AMQP only after the bus pub-ack; sink acks the bus only after the publisher confirm. Configured by env (`VEJAS_AMQP_*`, see the recipe). Certified against a From 0a1ad4b0b3e2999fc7a0933cf9b1fc2831886139 Mon Sep 17 00:00:00 2001 From: Cyril Poder Date: Fri, 6 Nov 2026 19:47:50 +0100 Subject: [PATCH 4/9] deps: the seven advisories of the sync nats client leave audit.toml RUSTSEC-2026-0049, -0098, -0099, -0104, -0346, -0349 and -0382 were ignored only because of rustls 0.22.4 and rustls-webpki 0.102.8, held by nats 0.25. No lockfile has them now: all five are on rustls 0.23.46 and rustls-webpki 0.103.16, and async-nats pulls that same 0.103 line, so the audit watches the real versions again. scripts/audit.sh is green with the entries gone. RUSTSEC-2024-0421 stays: idna 0.1.5 comes through amiquip, not through nats. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_018wXgxT2UoFSZfqhAE1Tb5V --- .cargo/audit.toml | 32 -------------------------------- 1 file changed, 32 deletions(-) diff --git a/.cargo/audit.toml b/.cargo/audit.toml index 5132935..0d6806a 100644 --- a/.cargo/audit.toml +++ b/.cargo/audit.toml @@ -4,38 +4,6 @@ [advisories] ignore = [ - # rustls 0.22.4 and rustls-webpki 0.102.8 come with the sync `nats` 0.25 - # client (core, connectors/amqp, connectors/mq); connectors/amqp also uses - # rustls 0.22 directly for amqps://. nats 0.25 is the last release of the - # sync client, so this line gets no patched version; moving to async-nats - # removes all seven. Ignores go by advisory ID, so these would also hide - # 0346/0349 on rustls 0.23: keep core on >= 0.23.46. - # None is reachable the way the crates are used here: - - # Debug output of client session values leaks secrets only through a - # custom ClientSessionStore; neither nats nor vejas-amqp installs one. - "RUSTSEC-2026-0346", - # An unauthenticated ServerHello can drop later connections from hybrid - # post-quantum to classical key exchange. rustls 0.22 on ring never offers - # a post-quantum group, so there is nothing to downgrade. - "RUSTSEC-2026-0349", - # CRL distribution-point matching and CRL parsing: no CRL is ever loaded. - "RUSTSEC-2026-0049", - "RUSTSEC-2026-0104", - # URI and wildcard name constraints: reachable only after signature - # verification, with a certificate misissued by a CA the client already - # trusts, and only on a TLS connection (a nats:// URL to a server that does - # not require TLS stays plaintext). - "RUSTSEC-2026-0098", - "RUSTSEC-2026-0099", - # Certificate paths through the distrusted emSign intermediates: rustls-webpki - # 0.102 has no distrust list. Reachable only on a tls:// NATS_URL, from a - # network position that can impersonate the NATS server, with a chain the - # machine's OS trust store accepts (nats 0.25 loads the native roots). A NATS - # server on a plain or private-CA connection is outside it. This ID also - # covers rustls-webpki 0.103: keep core on >= 0.103.16, webpki-roots >= 1.0.10. - "RUSTSEC-2026-0382", - # idna 0.1.5 accepts Punycode labels that decode to plain ASCII. It comes # through url 1.7 <- amq-protocol 1.4 <- amiquip 0.4.2 (the last amiquip # release), in connectors/amqp only, where it parses the broker URL the From 9ab344c55422dbd0c085d79f5418e647cdff42df Mon Sep 17 00:00:00 2001 From: Cyril Poder Date: Fri, 6 Nov 2026 19:48:21 +0100 Subject: [PATCH 5/9] docs: the changelog, the threads section and the numbers that moved ARCHITECTURE gains the bus, the one thread that speaks NATS. The changelog records the move, the upgrade note on `_INBOX.>`, and what it fixed on the way: protocol injection through an emit subject, and the zombie left by a deleted consumer. Published numbers that changed, each against a run of the v0.3.6 binary in the same session: RSS under load 6-8 to 10 MB (v0.3.6 measures 9.2 today) and 49 to 38 MB at fifty flows, binary 6.2 to 11.2 MB (9.9 at v0.3.6), MQTT loopback 2 285 to 2 070 rt/s, and the multi-flow table. They change in README, bench/README, the book and site/index.html, which is deployed by hand. Left as published: cold start and the flow hop, which did not move; the paced p50 of 14 ms, which both binaries now beat, so this change did not earn it; the promote figure, not rerun on the old binary; the image size, not rebuilt; and bench/compare, a dated snapshot. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_018wXgxT2UoFSZfqhAE1Tb5V --- CHANGELOG.md | 29 +++++++++++++++++ README.md | 6 ++-- bench/README.md | 43 ++++++++++++++++++++------ docs/ARCHITECTURE.md | 15 ++++++--- docs/adr/0035-async-nats-bus-island.md | 5 ++- docs/book/src/introduction.md | 2 +- docs/book/src/reference/benchmarks.md | 6 ++-- site/index.html | 2 +- 8 files changed, 83 insertions(+), 25 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index be117b3..8da8813 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,12 +7,41 @@ is `0`, minor versions may carry breaking changes — they are called out here. ## [Unreleased] +### Security +- **Seven advisories leave `.cargo/audit.toml`**: RUSTSEC-2026-0049, -0098, -0099, + -0104, -0346, -0349 and -0382, all held by the sync `nats` client's rustls 0.22 + and rustls-webpki 0.102. Every lockfile is now on rustls 0.23.46 and + rustls-webpki 0.103.16, `connectors/amqp` included. + + ### Fixed - **A burst of connections during boot could end the version watch for good.** Its first connection panicked when the OS had no thread to give: nothing showed it, and a promoted version no longer reached that instance until a restart. It needs a thread limit and a burst at that moment. The watch and the control loop now carry a name and start again after a panic. +- **A consumer deleted under a running flow no longer leaves a zombie.** The flow + stayed `running` for good, consuming nothing, and two empty dead letters from an + `_INBOX` subject landed in the queue. It now restarts and recreates the consumer. + +### Changed +- **The NATS client is now `async-nats`, behind one module (ADR-0035).** `nats` 0.25 + is deprecated. `core/src/bus.rs` runs `async-nats` on one tokio worker and gives + the rest of the runtime a blocking API: no other module sees tokio or a NATS type, + and flows, connectors and supervisors stay threads. A process now holds one + connection, where it held two per flow. At 200 flows that is 210 threads against + 1 011, 1 connection against 401, and 26 MB against 140. A flow is running 11 ms + after spawn, against 1.7 s. The flow hop holds its rate, ten flows run about 10 % + faster, and the 5 ms flusher floor is gone, with `publish_confirmed`, + `fetch_iter_flushed` and the second connection for acks. The costs: the binary + grows from 9.9 to 11.2 MB, RSS at one flow from 9.2 to 10.2 MB, and the MQTT + loopback falls from about 2 400 to 2 070 round-trips/s. Existing streams, + consumers, buckets and snapshots work as they are, and v0.3.6 reads back what this + version writes. **To upgrade:** a NATS user with a publish allow-list needs + `_INBOX.>` on it. Without it nothing is ever acked and everything redelivers; the + log names the permissions violation. +- **`connectors/mq` and `connectors/amqp` compile the same module**, and + `connectors/amqp` moves to rustls 0.23. ## [0.4.0] — 2026-11-06 diff --git a/README.md b/README.md index 9050dad..f5fff31 100644 --- a/README.md +++ b/README.md @@ -33,10 +33,10 @@ end-to-end test ([the script](e2e/bridge-film.mjs)) — the video cannot lie. ![The expert corrects the meaning; the pipes never move](docs/demo/bridge-teaser.gif) **Measured, not claimed** — reproducible on an 8-core dev machine -([methodology and comparisons](bench/)): cold start **11–13 ms**, **6–8 MB** -RSS under load (49 MB with fifty live flows), **6.2 MB** binary / **201 MB** +([methodology and comparisons](bench/)): cold start **11–13 ms**, **10 MB** +RSS under load (38 MB with fifty live flows), **11.2 MB** binary / **201 MB** image, end-to-end p50 **2 ms** uncongested / p50 14 ms at a paced 1 900/s -sustained with **every event hop persisted** in JetStream, **2 285/s** through a +sustained with **every event hop persisted** in JetStream, **2 070/s** through a real MQTT broker and back (QoS 1 both ways, PUBACK-gated). Same order of magnitude as engines that persist nothing, in ~25× less memory. diff --git a/bench/README.md b/bench/README.md index 02e03a9..b06776e 100644 --- a/bench/README.md +++ b/bench/README.md @@ -31,12 +31,12 @@ All five ceilings fell (#4+#3 `5cffb7b`, #2 `b34d9f3`, #1 `75fad83`, | Metric | v0 | all five fixes in | |---|---|---| | Cold start | 13–15 ms | **11–13 ms** | -| Runtime RSS under load | 6–8 MB | **6–8 MB** | -| Binary size | 3.9 MB | 6.2 MB (rustls) | +| Runtime RSS under load | 6–8 MB | **10 MB** | +| Binary size | 3.9 MB | 11.2 MB (rustls, async-nats) | | e2e sustained, paced ~1 900/s (`bench/paced.sh 2000 15`) | — | **p50 14 ms, p99 36 ms**, 20 000/20 000 | | e2e saturated (32 conns) | 65/s delivered | ~4 900/s ingest, **~2 650/s delivered** (sink-bound, queue absorbs the rest) | | e2e latency, uncongested (`bench/paced.sh 20 15`) | p50 859 ms | **p50 2 ms, p99 3 ms** | -| MQTT broker loopback, QoS 1 both ways (`bench/broker-mqtt.sh 5000`) | — | **2 285 rt/s**, 5 000/5 000 | +| MQTT broker loopback, QoS 1 both ways (`bench/broker-mqtt.sh 5000`) | — | **2 070 rt/s**, 5 000/5 000 | | Isolated flow hop | 171/s | **8 110/s** | The old "sustained (12 conns) 1 701/s, p50 18 ms" row died with ceiling #5: @@ -47,6 +47,31 @@ any concurrency, so steady-state latency is now measured **paced** Every hop persisted in JetStream throughout — the guarantee never moved. +### Since ADR-0035 + +The sync `nats` client left for `async-nats` behind `core/src/bus.rs`, so findings +#5 and #6 below describe a client that is gone, along with `publish_confirmed`, +`fetch_iter_flushed` and the second connection of #7. Against the v0.3.6 binary, +alternating in one session on this box: + +| | v0.3.6 | now | +|---|---:|---:| +| Threads at 1 / 10 / 50 / 200 idle flows | 16 / 61 / 261 / 1 011 | 11 / 20 / 60 / 210 | +| NATS connections, same | 3 / 21 / 101 / 401 | 1 | +| RSS, same, MB | 8.9 / 16.7 / 46.9 / 139.8 | 9.6 / 10.5 / 14.2 / 26.4 | +| Flow hop (`PUBS=16`, 64 000 events) | 13 100 – 13 700/s | 13 200 – 13 500/s | +| Flow hop, `bench/pub.py` as publisher | 20 100 – 20 500/s | 19 400 – 21 400/s | +| One detect unit | 30 800 – 35 400/s, 11 MB | 34 700 – 41 100/s, 12.7 MB | +| MQTT loopback | 2 370 – 2 430 rt/s | 2 050 – 2 090 rt/s | +| Spawn to a flow running | 1 714 ms | 10.9 ms | +| Paced 2 000/s, p50 / p99 | 3–4 / 14–17 ms | 3 / 14–15 ms | +| Binary | 9.9 MB | 11.2 MB | + +The paced row is why the p50 14 ms above stands as published: both binaries are far +below it today, so this change did not move it. The detect tables further down +predate this: add about 1.7 MB to their RSS. The method and the MQTT cost are in +ADR-0035. + The footprint numbers are the thesis, measured. The throughput and latency numbers are **known ceilings of the v0 I/O paths, not of the interpreter** — each one matches its cause exactly: @@ -134,13 +159,13 @@ they publish). Scaling the number of flows (`bench/multi-flow.sh`): | Flows | Aggregate rate | Runtime RSS | |---|---|---| -| 1 | 7–8 k/s | 6 MB | -| 10 | **13 746/s** | 16.6 MB | -| 50 | 8 410/s | 49.3 MB | +| 1 | 7–8 k/s | 10 MB | +| 10 | **13 700 – 14 700/s** | 14.5 MB | +| 50 | 11 100 – 12 100/s | 38 MB | Throughput *rises* with flow count (consumers parallelize; the bus, not the -interpreter, is the bound) and memory stays ~1 MB per running flow — fifty -live, persisted flows in under 50 MB. +interpreter, is the bound) and memory stays around half a megabyte per running +flow under load — fifty live, persisted flows in under 40 MB. ## Detect units (`bench/detect.sh`) @@ -252,7 +277,7 @@ one unit handles that at around 25 000 events/s in 18 MB. - **One instance saturates near 44 000 events/s** whatever the unit count. Beyond it, add instances: they share the durable consumers and the work (see *Clustering*). -- **Artefacts:** the binary is 8.2 MiB, the image 37.6 MiB, and a unit that is +- **Artefacts:** the binary is 10.7 MiB, the image 37.6 MiB, and a unit that is idle costs nothing measurable. Measured on the dev box in this file's header — 8 cores, WSL2 — two runs of diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 65c65e5..c556b9f 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -31,11 +31,11 @@ docker-compose.yml two containers: nats + vejas The runtime is a single OS process. Inside it: -- **one supervisor thread per flow** — connects to NATS, creates a durable pull - consumer on the flow's `source`, and loops: fetch a bounded batch (the pull - carries a server-side expiry so a stopped flow actually stops), run each - event through the interpreter, publish the emits **before** acking - (at-least-once; a crash means redelivery, never a lost emit), and record the +- **one supervisor thread per flow** — creates a durable pull consumer on the + flow's `source`, and loops: fetch a bounded batch (the pull carries a + server-side expiry so a stopped flow actually stops), run each event through + the interpreter, publish the emits **before** acking (at-least-once; a crash + means redelivery, never a lost emit), and record the event in the in-memory trace ring (last 50 per flow, `/events`; the full event is kept internally as shadow-replay fuel). A message that still fails after 5 deliveries is dropped (acked) and the drop is traced — redelivery is @@ -48,6 +48,11 @@ The runtime is a single OS process. Inside it: endpoints, the editing endpoints, and the MCP endpoint. One request = one thread (so a slow `/flows/new` never blocks monitoring), at most `VEJAS_HTTP_MAX_INFLIGHT` at once; past it the request gets a 503. +- **the bus** (`core/src/bus.rs`, ADR-0035) — the only thread that speaks NATS. + One tokio worker runs the `async-nats` client on one connection, shared by the + whole process. Everything above stays synchronous: a thread that needs the bus + calls a blocking function and parks until it returns. No other module sees + tokio or a NATS type, and the interpreter has no access to the bus. A panic in a unit's supervisor is caught around each attempt and takes the path of any other failure: `restarting`, the error in `last_error`, backoff, another diff --git a/docs/adr/0035-async-nats-bus-island.md b/docs/adr/0035-async-nats-bus-island.md index 7f1aff8..79405bd 100644 --- a/docs/adr/0035-async-nats-bus-island.md +++ b/docs/adr/0035-async-nats-bus-island.md @@ -29,9 +29,8 @@ type. Flows, the interpreter, connectors and supervisors do not change shape. `Engine::live` and `Engine::shadow` are untouched and the interpreter has no bus access, so `Program::writes_bus()` means what it meant. -This is not an async rewrite. The rewrite was never on the table: threads are -what make a flow simple to read, and the cost above was the client's, not the -threads'. +There is no async rewrite. Threads are what make a flow simple to read, and the +cost above was the client's. The eight points that make the module. diff --git a/docs/book/src/introduction.md b/docs/book/src/introduction.md index 6df5a40..f7b1123 100644 --- a/docs/book/src/introduction.md +++ b/docs/book/src/introduction.md @@ -18,7 +18,7 @@ Three design commitments shape everything: approves (ADR-0006, ADR-0024). 3. **Measured, not claimed.** Every number in these docs comes from a reproducible benchmark in [`bench/`](reference/benchmarks.md): cold start - 11 ms, 6–8 MB RSS under load, end-to-end p50 2 ms uncongested, every hop + 11 ms, 10 MB RSS under load, end-to-end p50 2 ms uncongested, every hop persisted, cluster promote in 60 ms — lossless. ## Where to go diff --git a/docs/book/src/reference/benchmarks.md b/docs/book/src/reference/benchmarks.md index 6541d37..c00c1f4 100644 --- a/docs/book/src/reference/benchmarks.md +++ b/docs/book/src/reference/benchmarks.md @@ -10,13 +10,13 @@ yours. | Metric | Value | Reproduce with | |---|---|---| | Cold start (spawn → healthz) | 11–13 ms | `bench/run.sh` | -| Runtime RSS under load | 6–8 MB (49 MB with 50 live flows) | `bench/run.sh`, `bench/multi-flow.sh` | -| Binary / image | 6.2 MB / 201 MB | — | +| Runtime RSS under load | 10 MB (38 MB with 50 live flows) | `bench/flow-only.sh`, `bench/multi-flow.sh` | +| Binary / image | 11.2 MB / 201 MB | — | | e2e latency, uncongested (webhook→flow→sink) | **p50 2 ms, p99 3 ms** | `bench/paced.sh 20 15` | | e2e paced sustained ~1 900/s | p50 14 ms, p99 36 ms, 20 000/20 000 | `bench/paced.sh 2000 15` | | e2e saturated (32 conns) | ~4 900/s ingest, ~2 650/s delivered (sink-bound) | `bench/run.sh 15 32` | | Isolated flow hop | 13 652/s (13 746/s aggregate over 10 flows) | `bench/flow-only.sh`, `bench/multi-flow.sh` | -| MQTT loopback, QoS 1 both ways, real mosquitto | 2 285 rt/s, 5 000/5 000 | `bench/broker-mqtt.sh 5000` | +| MQTT loopback, QoS 1 both ways, real mosquitto | 2 070 rt/s, 5 000/5 000 | `bench/broker-mqtt.sh 5000` | | Cluster: instance kill -9 under load | 20 000/20 000, zero loss | `bench/cluster.sh` | | Cluster: singleton failover | ~2.6 s graceful / ~5.9 s crash (TTL-bound) | `bench/cluster-gaps.sh` | | Cluster-wide version promote | 60 ms convergence, lossless mid-burst | `bench/cluster-promote.sh` | diff --git a/site/index.html b/site/index.html index b7fab6d..55e07f4 100644 --- a/site/index.html +++ b/site/index.html @@ -287,7 +287,7 @@

Business surface

1 binary  ·  1 dependency (NATS)  ·  0 builder UI  ·  nothing proprietary to export

-

measured: cold start 11–13 ms  ·  6–8 MB RSS  ·  p50 2 ms, every event hop persisted  ·  cluster promote 60 ms, lossless  ·  the benchmarks

+

measured: cold start 11–13 ms  ·  10 MB RSS  ·  p50 2 ms, every event hop persisted  ·  cluster promote 60 ms, lossless  ·  the benchmarks

From 49100057a6ae81c659f5774ee9021c3926c55d1e Mon Sep 17 00:00:00 2001 From: Cyril Poder Date: Fri, 6 Nov 2026 20:33:37 +0100 Subject: [PATCH 6/9] flows: a publish refused before it leaves is a failed run batch.publish refuses what can never leave, a payload over the server's limit above all. The consumer loop left the source message un-acked and went on, with no check of delivered, so that message redelivered for ever and never reached the DLQ, against the cap of ADR-0015. It now fails the run: same cap, same dead letter, the reason in last_error. Emits held before the refused one in that run still go out, so a redelivery can repeat them, within at-least-once. Detect units keep the old path: their subject comes from the program and their engine has already moved on, so a redelivery there is a different question. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_018wXgxT2UoFSZfqhAE1Tb5V --- CHANGELOG.md | 6 +++++- core/src/bus.rs | 4 ++-- core/src/main.rs | 37 +++++++++++++++++-------------------- e2e/emit-subject.sh | 12 ++++++++++++ 4 files changed, 36 insertions(+), 23 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8da8813..6f48062 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,7 +13,6 @@ is `0`, minor versions may carry breaking changes — they are called out here. and rustls-webpki 0.102. Every lockfile is now on rustls 0.23.46 and rustls-webpki 0.103.16, `connectors/amqp` included. - ### Fixed - **A burst of connections during boot could end the version watch for good.** Its first connection panicked when the OS had no thread to give: nothing showed it, @@ -23,6 +22,11 @@ is `0`, minor versions may carry breaking changes — they are called out here. - **A consumer deleted under a running flow no longer leaves a zombie.** The flow stayed `running` for good, consuming nothing, and two empty dead letters from an `_INBOX` subject landed in the queue. It now restarts and recreates the consumer. +- **An emit the server cannot take no longer redelivers forever.** A publish + refused before it leaves, for a payload over the server's limit, left its source + message un-acked with no cap on deliveries. It is now a failed run, dead-lettered + after `MAX_DELIVERIES` like any other. Detect units keep the old behaviour here: + their emits are small and their engine state makes a redelivery a different case. ### Changed - **The NATS client is now `async-nats`, behind one module (ADR-0035).** `nats` 0.25 diff --git a/core/src/bus.rs b/core/src/bus.rs index 1da151f..6e75c7f 100644 --- a/core/src/bus.rs +++ b/core/src/bus.rs @@ -365,8 +365,8 @@ pub struct Batch { impl Batch { /// Hold a publish. An error means this one can never leave (a subject the /// protocol cannot carry, a payload over the server's limit), and only this - /// one: the caller leaves its source message un-acked, and the rest of the - /// batch goes on. + /// one: the caller treats its source message as a failed run, and the rest of + /// the batch goes on. pub fn publish(&mut self, subject: &str, payload: impl Into) -> Result<(), String> { let payload = payload.into(); if subject.is_empty() || subject.contains([' ', '\t', '\r', '\n']) { diff --git a/core/src/main.rs b/core/src/main.rs index 7049e58..0f1e020 100644 --- a/core/src/main.rs +++ b/core/src/main.rs @@ -1253,26 +1253,25 @@ fn supervise_vjs(handle: Arc, root: PathBuf) { }; let t0 = Instant::now(); let start_nanos = metrics::now_nanos(); - match vjs::run(&prog, &event, &mut engine) { + // Holding an emit is local, so a refusal is final for this message: + // it can never leave, however often it redelivers. That makes it + // a failed run, capped by MAX_DELIVERIES and dead-lettered. + let outcome = vjs::run(&prog, &event, &mut engine).and_then(|ctx| { + for (subj, payload) in &ctx.emits { + let bytes = serde_json::to_vec(payload).unwrap_or_default(); + // held; sent and confirmed once per batch below + batch.publish(subj, bytes).map_err(|e| format!("publish {subj}: {e}"))?; + } + Ok(ctx) + }); + match outcome { Ok(ctx) => { - let mut ok = true; - for (subj, payload) in &ctx.emits { - let bytes = serde_json::to_vec(payload).unwrap_or_default(); - // held; sent and confirmed once per batch below - if let Err(e) = batch.publish(subj, bytes) { - eprintln!( - "[vejas] {}: publish {subj}: {e}", - handle.spec.name - ); - ok = false; - } - } let emits = ctx.emits.len() as u64; - metrics::observe(&handle.spec.name, ok, emits, t0.elapsed().as_secs_f64()); + metrics::observe(&handle.spec.name, true, emits, t0.elapsed().as_secs_f64()); metrics::span(metrics::Span { unit: handle.spec.name.clone(), subject: msg.subject.clone(), - ok, + ok: true, error: None, start_nanos, end_nanos: metrics::now_nanos(), @@ -1281,8 +1280,8 @@ fn supervise_vjs(handle: Arc, root: PathBuf) { record_trace( &handle.spec.name, &msg.subject, - ok, - (!ok).then(|| "publish failed (will redeliver)".to_string()), + true, + None, ctx.emits.iter().map(|(s, _)| s.clone()).collect(), // The preview is derived from the stored event when // /events is read; serialising it here cost every @@ -1290,9 +1289,7 @@ fn supervise_vjs(handle: Arc, root: PathBuf) { String::new(), Some(event), ); - if ok { - to_ack.push(msg); // acked after the batch flush - } + to_ack.push(msg); // acked after the batch flush } Err(e) => { eprintln!("[vejas] {}: {e}", handle.spec.name); diff --git a/e2e/emit-subject.sh b/e2e/emit-subject.sh index 25f9032..9a7497b 100755 --- a/e2e/emit-subject.sh +++ b/e2e/emit-subject.sh @@ -19,6 +19,9 @@ URL=nats://127.0.0.1:4281; H="http://127.0.0.1:8753"; IN=18794 mkdir -p "$R/flows" "$R/connectors" printf '# flow: dyn\nsource "vx.dyn.in"\nemit f"vx.out.{tenant}", {id: id}\n' > "$R/flows/dyn.vjs" printf '# connector: http_in\ndriver "http-in"\nPORT = %s\n' $IN > "$R/connectors/http_in.vjs" +# an emit over the server's 1 MB limit: 16 bytes doubled seventeen times +printf '# flow: big\nsource "vx.big.in"\ns = "xxxxxxxxxxxxxxxx"\nfor i in items:\n s = s + s\nend\nemit "vx.big.out", {s: s}\n' \ + > "$R/flows/big.vjs" nats-server -js -sd "$S/n" -a 127.0.0.1 -p 4281 > /dev/null 2>&1 & NP=$! sleep 0.5 @@ -57,6 +60,15 @@ print(len([d for d in json.load(sys.stdin)["dead_letters"] if "not a valid subje for _ in $(seq 1 150); do [ "$(letters)" = "2" ] && break; sleep 0.1; done [ "$(letters)" = "2" ] && ok "both messages reached the DLQ with the reason" || ko "$(letters) of 2 reached the DLQ" +echo "── an emit the server would refuse takes it too" +curl -s -m 5 -o /dev/null -X POST -H 'Content-Type: application/json' \ + -d '{"items":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17]}' "http://127.0.0.1:$IN/ingest/big.in" +big_letters(){ curl -s -m 5 "$H/dlq" | python3 -c ' +import json, sys +print(len([d for d in json.load(sys.stdin)["dead_letters"] if d.get("unit") == "flow:big"]))'; } +for _ in $(seq 1 150); do [ "$(big_letters)" = "1" ] && break; sleep 0.1; done +[ "$(big_letters)" = "1" ] && ok "an oversized emit reached the DLQ" || ko "the oversized emit never reached the DLQ" + echo "── a shadow run shows the same error" curl -s -m 10 -H 'Content-Type: application/json' -d \ '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"vejas_run_flow","arguments":{"file":"flows/dyn.vjs","input":{"id":1,"tenant":"a b"}}}}' \ From 0847fb888cba0f9902deba846f91e92b6b476233 Mon Sep 17 00:00:00 2001 From: Cyril Poder Date: Fri, 6 Nov 2026 20:33:37 +0100 Subject: [PATCH 7/9] docs(adr-0035): accepted Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_018wXgxT2UoFSZfqhAE1Tb5V --- docs/adr/0035-async-nats-bus-island.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adr/0035-async-nats-bus-island.md b/docs/adr/0035-async-nats-bus-island.md index 79405bd..48bad51 100644 --- a/docs/adr/0035-async-nats-bus-island.md +++ b/docs/adr/0035-async-nats-bus-island.md @@ -1,6 +1,6 @@ # 0035 — The NATS client moves to async-nats, behind one synchronous module -- Status: Proposed +- Status: Accepted - Date: 2026-11-06 ## Context From 90e93fd876baa8609a92aa3d1828c943a7ff4ced Mon Sep 17 00:00:00 2001 From: Cyril Poder Date: Fri, 6 Nov 2026 17:55:31 +0100 Subject: [PATCH 8/9] dlq: a dead letter announces itself on vx.dlq.events Nothing told anyone that a message had died: the self-healing loop waited for someone to open a session. to_dlq now publishes on .dlq.events, an ordinary subject, so waking a person or an agent is a flow or a sink, as ADR-0024 wants of notifications. The event holds what the runtime wrote itself and never the payload or the error text: a dead letter is untrusted content and this is what wakes things up. A burst is one event per unit per VEJAS_DLQ_EVENT_SECS (default 60), with count for the letters it stands for, so that it wakes a reader once and loses nothing; a small thread sends what the window held back if no letter follows. A letter whose subject is the event subject is not announced, so a failing reader cannot feed itself. The documented route out is http-out. A child of an exec-* connector inherits the runtime's environment, VEJAS_APPROVAL_TOKEN included, so an agent must not be started that way. This is the runtime half of the trigger: the agent that answers it is still to be built. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_018wXgxT2UoFSZfqhAE1Tb5V --- .github/workflows/ci.yml | 2 + CHANGELOG.md | 8 ++ core/src/connectors.rs | 158 +++++++++++++++++++++++++- docs/SUBJECTS.md | 14 ++- docs/adr/0015-dead-letter-queue.md | 13 +++ docs/book/src/guides/dlq-replay.md | 30 ++++- docs/book/src/reference/env-vars.md | 1 + docs/examples/dlq_webhook.vjs.example | 11 ++ e2e/dlq-events.sh | 84 ++++++++++++++ 9 files changed, 317 insertions(+), 4 deletions(-) create mode 100644 docs/examples/dlq_webhook.vjs.example create mode 100755 e2e/dlq-events.sh diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 58a8671..84895f0 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -96,6 +96,8 @@ jobs: run: e2e/emit-subject.sh - name: http-in survives a burst and a thread limit run: e2e/http-in-bound.sh + - name: Dead letters announce themselves, without feeding themselves + run: e2e/dlq-events.sh - name: Docs book builds (broken includes fail here, not on deploy) run: | curl -sL https://github.com/rust-lang/mdBook/releases/download/v0.5.4/mdbook-v0.5.4-x86_64-unknown-linux-gnu.tar.gz | tar xz diff --git a/CHANGELOG.md b/CHANGELOG.md index 6f48062..3d32461 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,14 @@ is `0`, minor versions may carry breaking changes — they are called out here. ## [Unreleased] +### Added +- **Dead letters announce themselves** on `vx.dlq.events`, an ordinary subject, + so that a flow or a sink can wake a person or an agent (ADR-0015 Amendment A). + The event carries metadata only, never the payload or the error text, and + `VEJAS_DLQ_EVENT_SECS` (default 60) groups a burst into one event per unit, + with `count` for the letters it stands for. `docs/examples/dlq_webhook.vjs.example` + posts it to a URL. + ### Security - **Seven advisories leave `.cargo/audit.toml`**: RUSTSEC-2026-0049, -0098, -0099, -0104, -0346, -0349 and -0382, all held by the sync `nats` client's rustls 0.22 diff --git a/core/src/connectors.rs b/core/src/connectors.rs index 6f3bcb8..42c6551 100644 --- a/core/src/connectors.rs +++ b/core/src/connectors.rs @@ -16,8 +16,9 @@ use std::io::{Read, Write}; use std::net::{TcpListener, TcpStream}; +use std::collections::HashMap; use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering}; -use std::sync::Arc; +use std::sync::{Arc, Mutex, OnceLock}; use std::thread; use std::time::{Duration, Instant}; @@ -1180,7 +1181,125 @@ pub fn to_dlq( })) .map_err(|e| e.to_string())?; let dlq_subject = format!("{DLQ_ROOT}.{}", dlq_unit_token(unit)); - js.publish(&dlq_subject, body).map(|_| ()).map_err(|e| e.to_string()) + js.publish(&dlq_subject, body).map_err(|e| e.to_string())?; + announce_dead_letter(js, unit, subject, attempts, version); + Ok(()) +} + +// ───────────────────────── dead-letter events (ADR-0015, Amendment A) ───────── +// +// Nothing used to tell anyone that a message died: the self-healing loop waited +// for someone to open a session. Each dead letter is now announced on +// `.dlq.events`, an ordinary subject, so waking a person or an agent is a +// flow or a sink, like every other notification here (ADR-0024). The event holds +// only what the runtime wrote itself, never the payload or the error text: a dead +// letter is untrusted content, and this is what wakes things up. A reader that +// needs the letter fetches it through `vejas_dlq`. + +/// Dead letters of one unit not announced yet, and the last one's coordinates. +#[derive(Default)] +struct DlqAnnouncement { + last_sent: u64, + count: u64, + subject: String, + attempts: i64, + version: String, +} + +static DLQ_ANNOUNCEMENTS: OnceLock>> = OnceLock::new(); +static DLQ_FLUSHER: AtomicBool = AtomicBool::new(false); + +fn dlq_announcements() -> &'static Mutex> { + DLQ_ANNOUNCEMENTS.get_or_init(|| Mutex::new(HashMap::new())) +} + +fn dlq_events_subject() -> String { + let root = std::env::var("VEJAS_SUBJECT_ROOT").unwrap_or_else(|_| "vx".into()); + format!("{root}.dlq.events") +} + +/// At most one event per unit per this many seconds, so that a burst of poison +/// wakes a reader once and not ten thousand times. 0 announces every letter. +fn dlq_event_gap() -> u64 { + std::env::var("VEJAS_DLQ_EVENT_SECS").ok().and_then(|v| v.parse().ok()).unwrap_or(60) +} + +/// The events due now. Each carries `count`, the dead letters it stands for, so +/// the window delays news without losing any. +fn take_due_dlq_events( + pending: &mut HashMap, + now: u64, + gap: u64, +) -> Vec { + pending + .iter_mut() + .filter(|(_, a)| a.count > 0 && now.saturating_sub(a.last_sent) >= gap) + .map(|(unit, a)| { + let event = serde_json::json!({ + "transition": "dead-lettered", "ts": now, "unit": unit, + "subject": a.subject, "attempts": a.attempts, "version": a.version, + "count": a.count, + }); + a.last_sent = now; + a.count = 0; + event + }) + .collect() +} + +/// Best-effort, like the proposal events: the letter is stored and the caller +/// goes on to ack, whatever happens to its announcement. +fn publish_dlq_event(js: &bus::JetStream, event: &Value) { + let body = serde_json::to_vec(event).unwrap_or_default(); + if let Err(e) = js.publish(&dlq_events_subject(), body) { + eprintln!("[vejas] dead-letter event not published: {e}"); + } +} + +fn announce_dead_letter( + js: &bus::JetStream, + unit: &str, + subject: &str, + attempts: i64, + version: &str, +) { + // A reader of these events that keeps failing dead-letters them in turn. If + // that were announced too, the reader would feed itself forever. + if subject == dlq_events_subject() { + return; + } + let gap = dlq_event_gap(); + let due = { + let mut pending = dlq_announcements().lock().unwrap(); + let a = pending.entry(unit.to_string()).or_default(); + a.count += 1; + a.subject = subject.to_string(); + a.attempts = attempts; + a.version = version.to_string(); + take_due_dlq_events(&mut pending, epoch_secs(), gap) + }; + for event in &due { + publish_dlq_event(js, event); + } + // What the window held back must still go out if no letter follows it. + if gap > 0 && !DLQ_FLUSHER.swap(true, Ordering::SeqCst) { + let js = js.clone(); + let flusher = thread::Builder::new().name("dlq-events".into()).spawn(move || { + while crate::RUNNING.load(Ordering::SeqCst) { + thread::sleep(Duration::from_secs(1)); + let due = { + let mut pending = dlq_announcements().lock().unwrap(); + take_due_dlq_events(&mut pending, epoch_secs(), gap) + }; + for event in &due { + publish_dlq_event(&js, event); + } + } + }); + if flusher.is_err() { + DLQ_FLUSHER.store(false, Ordering::SeqCst); + } + } } // ───────────────────────── exec-source offset resume (ADR-0011/0021) ───────────────────────── @@ -2121,6 +2240,41 @@ fn next_url(base: &str, body: &serde_json::Value, field: &str) -> Option #[cfg(test)] mod tests { + fn letter(pending: &mut HashMap, unit: &str) { + let a = pending.entry(unit.to_string()).or_default(); + a.count += 1; + a.subject = "vx.in".into(); + } + + #[test] + fn a_burst_of_dead_letters_is_one_event_that_counts_them() { + let mut pending = HashMap::new(); + letter(&mut pending, "flow:a"); + let first = take_due_dlq_events(&mut pending, 1000, 60); + assert_eq!(first.len(), 1, "the first letter is news at once"); + assert_eq!(first[0]["count"], 1); + for _ in 0..3 { + letter(&mut pending, "flow:a"); + } + assert!(take_due_dlq_events(&mut pending, 1030, 60).is_empty(), "inside the window"); + let held = take_due_dlq_events(&mut pending, 1060, 60); + assert_eq!(held[0]["count"], 3, "the window delays news, it loses none"); + assert!(take_due_dlq_events(&mut pending, 2000, 60).is_empty(), "nothing pending"); + } + + #[test] + fn units_are_windowed_apart_and_zero_means_every_letter() { + let mut pending = HashMap::new(); + letter(&mut pending, "flow:a"); + take_due_dlq_events(&mut pending, 1000, 60); + letter(&mut pending, "flow:a"); + letter(&mut pending, "flow:b"); + let due = take_due_dlq_events(&mut pending, 1001, 60); + assert_eq!(due.len(), 1); + assert_eq!(due[0]["unit"], "flow:b"); + assert_eq!(take_due_dlq_events(&mut pending, 1001, 0).len(), 1, "gap 0 lets flow:a through"); + } + use super::*; use serde_json::json; diff --git a/docs/SUBJECTS.md b/docs/SUBJECTS.md index 25bc51c..68f90ae 100644 --- a/docs/SUBJECTS.md +++ b/docs/SUBJECTS.md @@ -7,7 +7,7 @@ Bundled connectors are **native Rust drivers** run from a declarative manifest (`connectors/.vjs`: `driver "..."` + literal config, editable in the panel, hot-addable). No subprocess, no Python. Drivers today: -- **http-in** (source:webhook) — `POST /ingest/` → `vx.`. Config: PORT. +- **http-in** (source:webhook) — `POST /ingest/` → `vx.`. Config: PORT, MAX_CONNECTIONS. - **timer** (source:interval) — emits PAYLOAD on SUBJECT every INTERVAL_SECS (an object payload gains a `ts` field, ISO 8601 UTC, when absent). - **http-poll** (source:poll) — GETs URL every INTERVAL_SECS → SUBJECT. Optional HEADERS; optional `ENVELOPE = true` publishes `{endpoint, fetched_at, body}` (like oauth-poll) so a stateless flow gets a `collected_at`. @@ -65,3 +65,15 @@ for content changes). It lives OUTSIDE `vx.>` on purpose: control traffic is transient and must never be captured by a stream (a stream on the command subject would hijack request/reply with its pub-ack). Specification: [CONTROL.md](CONTROL.md), decision: ADR-0013. + +## Events the platform publishes + +Ordinary subjects under the same root, so a flow or a sink reacts to them like to +anything else. Both carry metadata the runtime wrote itself. + +- `vx.dlq.events`: a message reached the DLQ (ADR-0015 Amendment A). `{transition, + ts, unit, subject, attempts, version, count}`, at most one per unit per + `VEJAS_DLQ_EVENT_SECS`, with `count` for the letters it stands for. Never the + payload or the error text. +- `vx.proposals.events`: a proposal changed state (ADR-0024). `{transition, ts, + proposal}`. diff --git a/docs/adr/0015-dead-letter-queue.md b/docs/adr/0015-dead-letter-queue.md index 0db18f2..197fdf7 100644 --- a/docs/adr/0015-dead-letter-queue.md +++ b/docs/adr/0015-dead-letter-queue.md @@ -83,6 +83,19 @@ The operator loop is: a dead letter shows the failure → correct the literal permanently unpublishable loops without a delivery cap. Out of scope here (it is a bus/config fault, not a poison payload), noted for a later pass. +**Amendment A (2026-11-06, normative):** a dead letter announces itself. Nothing +told anyone a message had died, so the self-healing loop of ADR-0024 waited for +someone to open a session. `to_dlq` now publishes on `.dlq.events`, an +ordinary subject: waking a person or an agent is a flow or a sink, following +ADR-0024's "notifications are flows". The event carries metadata the runtime +wrote itself, never the payload or the error text, because a dead letter is +untrusted content and this event is what wakes things up. One event per unit per +`VEJAS_DLQ_EVENT_SECS` (default 60), with `count` for the letters it stands for, +so that a burst wakes a reader once and loses nothing. A letter whose subject is +the event subject is not announced, so a failing reader cannot feed itself. An +agent started from an `exec-*` connector would inherit the runtime's environment, +so the documented route out is `http-out`. + ## Alternatives considered - **Keep dropping after the cap** — rejected: the failed work is lost and diff --git a/docs/book/src/guides/dlq-replay.md b/docs/book/src/guides/dlq-replay.md index f78068d..2c88f7b 100644 --- a/docs/book/src/guides/dlq-replay.md +++ b/docs/book/src/guides/dlq-replay.md @@ -23,9 +23,37 @@ post-mortem knows *which* logic failed, even after a promote. and `vejas_dlq_purge` are refused, and the HTTP routes want the `X-Approval-Token` header, which the panel sends. +## Being told + +Each dead letter is announced on `vx.dlq.events`, an ordinary subject, so what +happens next is a flow or a sink like any other: + +```json +{"transition": "dead-lettered", "ts": 1793980000, "unit": "flow:orders", + "subject": "vx.orders.in", "attempts": 5, "version": "d5768a2de6e8cbb0", + "count": 1} +``` + +- **Metadata only.** The payload and the error text stay in the DLQ: a dead + letter is untrusted content, and this event is what wakes a person or an + agent. Read the letter with `vejas_dlq`. +- **One event per unit per window** (`VEJAS_DLQ_EVENT_SECS`, 60 s). The first + letter is news at once; `count` says how many letters the next event stands + for, so a burst wakes a reader once and loses nothing. Each instance of a + cluster keeps its own window. +- **It cannot feed itself.** A reader of these events that keeps failing + dead-letters them, and those letters are not announced. + +[`dlq_webhook.vjs.example`](https://github.com/cpoder/vejas/blob/master/docs/examples/dlq_webhook.vjs.example) +posts the event to a URL: a chat channel, or a runner that starts your agent +with the agent's token alone. Do not start an agent from an `exec-*` connector: +a child inherits the runtime's environment, `VEJAS_APPROVAL_TOKEN` and +`VEJAS_SECRET_*` included. + ## The loop that makes it self-healing -1. A message dies; the envelope carries version `v3`. +1. A message dies; the envelope carries version `v3`, and `vx.dlq.events` says + so. 2. An agent reads `vejas_dlq`, drafts a candidate flow, proves it: replay yesterday's real traffic through it ([time-travel](change-safely.md)), watch it shadow live traffic diff --git a/docs/book/src/reference/env-vars.md b/docs/book/src/reference/env-vars.md index 1deb4a0..25ad3cb 100644 --- a/docs/book/src/reference/env-vars.md +++ b/docs/book/src/reference/env-vars.md @@ -19,6 +19,7 @@ directory is a complete system. | `VEJAS_TENANT` | *(unset)* | Tenant label for provisioned packages. | | `VEJAS_BUNDLE` | *(unset)* | Bundle path for provisioning (`vejas_provision`). | | `VEJAS_AGENT_CMD` | *(unset)* | Command the panel uses for its agent-assist box. | +| `VEJAS_DLQ_EVENT_SECS` | `60` | At most one dead-letter event per unit per this many seconds on `vx.dlq.events`. `0` announces every letter. | | `VEJAS_HTTP_MAX_INFLIGHT` | `256` | Requests handled at once on the panel/MCP port, one thread each. Past it: 503. | ## Secrets (ADR-0008) diff --git a/docs/examples/dlq_webhook.vjs.example b/docs/examples/dlq_webhook.vjs.example new file mode 100644 index 0000000..0dde399 --- /dev/null +++ b/docs/examples/dlq_webhook.vjs.example @@ -0,0 +1,11 @@ +# connector: dlq_webhook — tell someone when a message dies (ADR-0015). +# Each dead letter is announced on vx.dlq.events: unit, subject, attempts, version, +# and count, the letters the event stands for. Metadata only: the payload stays in +# the DLQ, so what this wakes reads it with vejas_dlq, as untrusted content. +# +# The URL is a chat webhook, or a runner of your own that starts your agent with +# the agent's VEJAS_TOKEN alone. Keep the agent out of the runtime: a child of an +# exec-* connector inherits its environment, VEJAS_APPROVAL_TOKEN included. +driver "http-out" +SUBJECT = "vx.dlq.events" +URL = secret("ops/dlq_webhook_url") diff --git a/e2e/dlq-events.sh b/e2e/dlq-events.sh new file mode 100755 index 0000000..17f19a6 --- /dev/null +++ b/e2e/dlq-events.sh @@ -0,0 +1,84 @@ +#!/bin/bash +# A dead letter announces itself on vx.dlq.events (ADR-0015 Amendment A): metadata +# only, one event per unit per window with `count` for the letters it stands for, +# through an ordinary sink, and a failing reader of the events cannot feed itself. +# e2e/dlq-events.sh (needs the nats CLI) +set -uo pipefail +cd "$(dirname "$0")/.." +BIN="$PWD/core/target/release/vejas-runtime" +[ -x "$BIN" ] || { echo "build first" >&2; exit 1; } +S=$(mktemp -d); R=$(mktemp -d) +RT=""; NP=""; SUB=""; HOOK="" +trap 'kill $RT $NP $SUB $HOOK 2>/dev/null; wait $RT $NP $SUB $HOOK 2>/dev/null; rm -rf "$S" "$R"' EXIT +fail=0 +ok(){ echo " ✓ $1"; } +ko(){ echo " ✗ $1"; fail=1; } + +URL=nats://127.0.0.1:4280; H="http://127.0.0.1:8752"; HOOK_PORT=18791 +mkdir -p "$R/flows" "$R/connectors" +printf '# flow: demo\nsource "vx.demo.in"\nemit "vx.demo.out", {id: id}\n' > "$R/flows/demo.vjs" +# a reader of the events that fails on every one of them +printf '# flow: watcher\nsource "vx.dlq.events"\nx = secret("missing/x")\nemit "vx.demo.out", {x: x}\n' > "$R/flows/watcher.vjs" +printf '# connector: hook\ndriver "http-out"\nSUBJECT = "vx.dlq.events"\nURL = "http://127.0.0.1:%s/hook"\n' $HOOK_PORT \ + > "$R/connectors/hook.vjs" + +python3 -c ' +import http.server, sys +class H(http.server.BaseHTTPRequestHandler): + def do_POST(self): + body = self.rfile.read(int(self.headers.get("content-length", 0))) + open(sys.argv[2], "ab").write(body + b"\n") + self.send_response(200); self.send_header("Content-Length", "0"); self.end_headers() + def log_message(self, *a): pass +http.server.HTTPServer(("127.0.0.1", int(sys.argv[1])), H).serve_forever() +' $HOOK_PORT "$S/hook.out" & HOOK=$! + +nats-server -js -sd "$S/n" -a 127.0.0.1 -p 4280 > /dev/null 2>&1 & NP=$! +sleep 0.5 +NATS_URL=$URL VEJAS_ROOT="$R" VEJAS_HTTP_ADDR=127.0.0.1:8752 VEJAS_ACK_WAIT_SECS=1 VEJAS_DLQ_EVENT_SECS=2 \ + "$BIN" > "$S/rt.log" 2>&1 & RT=$! +until curl -sf -o /dev/null "$H/healthz"; do sleep 0.1; done +for _ in $(seq 1 100); do + curl -s -m 5 -H 'Content-Type: application/json' \ + -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"vejas_topology","arguments":{}}}' "$H/mcp" \ + | grep -q 'flow:demo.*running\|running.*flow:demo' && break + sleep 0.1 +done +sleep 1 +nats -s $URL sub vx.dlq.events --raw > "$S/events" 2>/dev/null & SUB=$! +sleep 0.5 +counted(){ python3 -c ' +import json, sys +print(sum(json.loads(l).get("count", 0) for l in open(sys.argv[1]) if l.startswith("{") and json.loads(l).get("unit") == sys.argv[2])) +' "$S/events" "$1"; } + +echo "── one poison message: announced at once, metadata only" +nats -s $URL pub vx.demo.in 'CANARY-not-json' > /dev/null 2>&1 +for _ in $(seq 1 50); do grep -q 'dead-lettered' "$S/events" && break; sleep 0.1; done +grep -q '"unit":"flow:demo"' "$S/events" && ok "the event names the unit" || ko "no event: $(cut -c1-120 "$S/events")" +grep -q '"count":1' "$S/events" && ok "it counts one letter" || ko "wrong count" +grep -q 'CANARY' "$S/events" && ko "the payload reached the event" || ok "the payload stays in the DLQ" +grep -q 'bad json' "$S/events" && ko "the error text reached the event" || ok "the error text stays in the DLQ" + +echo "── a burst: one event, nothing lost" +for i in $(seq 1 20); do nats -s $URL pub vx.demo.in "poison-$i" > /dev/null 2>&1; done +LINES_DURING=$(grep -c 'flow:demo' "$S/events") +[ "$LINES_DURING" -le 3 ] && ok "$LINES_DURING event(s) during 20 letters" || ko "$LINES_DURING events for 20 letters" +for _ in $(seq 1 60); do [ "$(counted flow:demo)" = "21" ] && break; sleep 0.1; done +[ "$(counted flow:demo)" = "21" ] && ok "the counts add up to all 21 letters" || ko "the counts add up to $(counted flow:demo)" + +echo "── an ordinary sink carries it" +for _ in $(seq 1 50); do grep -q 'dead-lettered' "$S/hook.out" 2>/dev/null && break; sleep 0.1; done +grep -q 'dead-lettered' "$S/hook.out" 2>/dev/null && ok "http-out posted the event" || ko "the sink got nothing" + +echo "── a failing reader does not feed itself" +watcher_letters(){ curl -s -m 5 "$H/dlq" | python3 -c ' +import json, sys +print(len([d for d in json.load(sys.stdin)["dead_letters"] if d.get("unit") == "flow:watcher"]))'; } +for _ in $(seq 1 200); do [ "$(watcher_letters)" -ge 1 ] && break; sleep 0.1; done +[ "$(watcher_letters)" -ge 1 ] && ok "the watcher dead-lettered the events it failed on" || ko "the watcher never dead-lettered" +sleep 4 +grep -q '"unit":"flow:watcher"' "$S/events" && ko "a failing reader announced its own letters" || ok "its letters are not announced" + +[ $fail -eq 0 ] && echo "DLQ EVENTS HOLD" || echo "DLQ EVENTS REGRESSION — do not ship" +exit $fail From 449b329536949858fc8e3a077a4478e543f42132 Mon Sep 17 00:00:00 2001 From: Cyril Poder Date: Sat, 7 Nov 2026 00:16:48 +0100 Subject: [PATCH 9/9] e2e: http-in-bound bursts during boot again The wait came from the sync client, which spawned threads on every connection and panicked in version_watch when the burst landed on its first one. The bus holds one connection from the start, so the test can be as strict as it was: ten bursts at boot gave no panic, and eight runs of this script pass. background() keeps its place as the net for any other panic, and its comment no longer names the client. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_018wXgxT2UoFSZfqhAE1Tb5V --- core/src/main.rs | 8 ++++---- e2e/http-in-bound.sh | 3 --- 2 files changed, 4 insertions(+), 7 deletions(-) diff --git a/core/src/main.rs b/core/src/main.rs index 0f1e020..96770bd 100644 --- a/core/src/main.rs +++ b/core/src/main.rs @@ -1867,10 +1867,10 @@ fn contained(attempt: impl FnOnce() -> Result<(), String>) -> Result<(), String> } /// Start a background loop of the runtime itself, which no supervisor watches. -/// `nats::connect` spawns threads and panics when the OS has none, and that used -/// to end the loop for good with nothing to show for it: a burst of connections -/// during boot took `version_watch`, and a promoted version no longer reached this -/// instance. The thread carries a name, and a panic starts the loop again. +/// A panic used to end such a loop for good with nothing to show for it: in v0.4.0 +/// a burst of connections during boot took `version_watch`, and a promoted version +/// no longer reached that instance. The thread carries a name, and a panic starts +/// the loop again. fn background(name: &'static str, body: impl Fn() + Send + 'static) { let spawned = thread::Builder::new().name(name.into()).spawn(move || { while RUNNING.load(Ordering::SeqCst) { diff --git a/e2e/http-in-bound.sh b/e2e/http-in-bound.sh index 501712f..e683967 100755 --- a/e2e/http-in-bound.sh +++ b/e2e/http-in-bound.sh @@ -98,9 +98,6 @@ else debian:trixie-slim vejas-runtime > /dev/null for _ in $(seq 1 100); do curl -sf -o /dev/null "$H2/healthz" && break; sleep 0.1; done wait_ingest $IN2 || ko "http-in never came up in the container" - # Past the boot: a burst that lands on the first NATS connections is another - # case, covered by background(), and it made this check flaky. - sleep 5 hold $IN2 300 3; HOLD=$! sleep 1 wait $HOLD 2>/dev/null