From 0012296ffa124577366d1ded8824e57eef719860 Mon Sep 17 00:00:00 2001 From: jrd Date: Fri, 17 Jul 2026 19:31:38 +0000 Subject: [PATCH 1/3] Add AGENTS.md and DEPLOY.md; expand COMPILING.md and CONTRIBUTING.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit AGENTS.md (based on the draft in #3785) is the entry point for coding agents: scope rules, the architecture laws whose violation causes audible failures (real-time path, wire compatibility, adversarial input), the do-not-edit list, a concrete manual verification recipe, and links to the existing docs. DEPLOY.md covers the step after COMPILING.md: moving a self-built server binary onto production hosts — architecture/glibc matching, ldd verification, systemd, UDP buffer tuning, firewalling, and a post-deploy checklist. Every rule corresponds to a real-world failure. COMPILING.md gains a GCC 13 moc workaround and a link to DEPLOY.md; CONTRIBUTING.md gains a pointer to AGENTS.md stating that its rules apply to agent-assisted work without exception. Co-Authored-By: Claude Fable 5 --- AGENTS.md | 72 +++++++++++++++++++++++++++++++++++++++++++++++++ COMPILING.md | 14 +++++++++- CONTRIBUTING.md | 4 ++- DEPLOY.md | 49 +++++++++++++++++++++++++++++++++ 4 files changed, 137 insertions(+), 2 deletions(-) create mode 100644 AGENTS.md create mode 100644 DEPLOY.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000000..16fbdb67de --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,72 @@ +# Jamulus — Agent Instructions + +Jamulus is a real-time networked music jamming application (client and server). Qt/C++, built with qmake. Entry point: `src/main.cpp`; build variants via `CONFIG` flags in `Jamulus.pro`. + +This file is the starting point for coding agents. It links to the detailed docs — read the relevant one before starting: + +- [CONTRIBUTING.md](CONTRIBUTING.md) — process, code style, licensing, ownership. Its rules apply to agent-assisted work without exception. +- [COMPILING.md](COMPILING.md) — building on every supported platform. +- [DEPLOY.md](DEPLOY.md) — moving a self-built server binary onto production hosts and verifying it. +- [docs/JAMULUS_PROTOCOL.md](docs/JAMULUS_PROTOCOL.md) — the wire protocol. +- [SECURITY.md](SECURITY.md) — report vulnerabilities to team@jamulus.io, never in a public issue. + +## Scope + +- One logical change per PR. No drive-by refactors, no reformatting of code you didn't otherwise touch, no bundled extras. +- Features must be agreed in a GitHub issue or Discussion **before** coding — see CONTRIBUTING.md. If no agreement exists, propose; don't implement. +- Stability outranks everything: Jamulus runs live performances. Prefer not adding a feature over adding risk (KISS principles in CONTRIBUTING.md). + +## Architecture laws + +Violating these causes audible failures for every connected musician, so they are non-negotiable: + +1. **Never block the real-time path.** The audio capture/playback callbacks (`src/sound/`) and the server's UDP receive → mix → send path (`socket.cpp`, `channel.cpp`, `server.cpp`) must not perform blocking I/O (DNS, HTTP, disk, database), wait on locks contended by non-real-time threads, or do unbounded allocation per packet. Slow work must run asynchronously on another thread; when its result isn't ready, the real-time path fails open and continues. A single synchronous lookup here freezes the audio of everyone on the server. +2. **The wire protocol is a compatibility contract.** `protocol.cpp` must interoperate with every older client and server in the wild. Never renumber `PROTMESSID_*` values, never change the layout of an existing message — extend only by adding new message IDs. Read the header comment in `src/protocol.cpp` and [docs/JAMULUS_PROTOCOL.md](docs/JAMULUS_PROTOCOL.md) before touching it. +3. **All network input is adversarial.** UDP packets arrive from arbitrary internet hosts. Bounds-check every length and count field before use. A malformed packet must be dropped silently — it must never crash, block, or corrupt the server. + +## DO NOT edit (generated or third-party) + +- `moc_*.cpp`, `ui_*.h`, `qrc_*.cpp` are generated at build time (moc/uic/rcc) — never hand-edit. +- `*.qm` files are compiled translations — regenerate from the `.ts` sources in `src/translation/`, never hand-edit. +- `libs/` is third-party (opus, oboe, NSIS). `libs/oboe` is a git submodule — run `git submodule update --init` if it is empty. +- Never run clang-format on any of the above. + +## Build / verify + +- Linux desktop: `qmake && make` (`qmake-qt5` on Fedora). +- Headless server: `qmake "CONFIG+=headless serveronly" && make`. +- macOS: `qmake QMAKE_APPLE_DEVICE_ARCHS=arm64 QT_ARCH=arm64 -spec macx-xcode Jamulus.pro` (use `x86_64` on Intel Macs; `macx-clang` to build with `make`), then `xcodebuild build` and `macdeployqt ./Release/Jamulus.app`. Full platform details: [COMPILING.md](COMPILING.md). +- There is no automated test suite. To verify a change: build a headless server and run `./Jamulus --server --nogui`, connect a client build to `127.0.0.1`, and exercise the changed behavior. State in the PR what you tested and on which platform. An untested change is an unfinished change. + +## Style (C / C++ / Obj-C++) + +- Run `make clang_format` before committing (the target exists once `qmake` has generated the Makefile). CI enforces a specific clang-format version — see `clangFormatVersion` in `.github/workflows/coding-style-check.yml`. +- If you add a new source directory or file extension, update `.github/workflows/coding-style-check.yml` and `.clang-format-ignore` as well as `Jamulus.pro` (see the comment above `CLANG_FORMAT_SOURCES`) — otherwise CI and local formatting silently diverge. +- Rules in brief: 4-space indent (no tabs), braces on their own line, space inside `()` and around `if`/`for`/`while` conditions, column limit 150, left pointer alignment (`int* p`). +- New files need an AGPL 3.0 (or later) license header; pre-3.12.1 files carry combined GPL/AGPL blocks — leave existing headers alone. Details in CONTRIBUTING.md. + +## Translations + +- Use substitution, never concatenation: `tr ( "Hello, %1!" ).arg ( name )` — concatenated fragments cannot be translated. +- Per-language `.ts` files live in `src/translation/`; see [docs/TRANSLATING.md](docs/TRANSLATING.md). + +## JSON-RPC + +- If you change RPC methods, regenerate `docs/JSON-RPC.md` with `tools/generate_json_rpc_docs.py` — CI (`check-json-rpcs-docs.yml`) fails otherwise. + +## Other tooling + +- `.sh` files: lint with `shellcheck --shell=bash` and `shfmt -d`. +- Python (under `tools/`): max line length 99, run `pylint` (< 3.0) with the repo's `.pylintrc`. +- ChangeLog: put `CHANGELOG: ` in the PR description (`CHANGELOG: SKIP` for changes users won't notice). Do **not** edit the `ChangeLog` file — it causes conflicts. + +## Version / portability constraints + +- Minimum Qt is **5.12.2**. Guard newer APIs with `#if QT_VERSION >= QT_VERSION_CHECK(...)`. +- Maintain C++11 compatibility and keep Windows, macOS, Linux, Android and iOS building. + +## PR expectations + +- Naming your branch `autobuild/` triggers CI builds of all targets on your fork — use it before opening the PR. +- Include the `CHANGELOG:` line, and for new dependencies or build changes add `AUTOBUILD: Please build all targets` to the PR description. +- PRs need two approving reviews to merge; the submitter is responsible for follow-up questions and agreed changes (see CONTRIBUTING.md). diff --git a/COMPILING.md b/COMPILING.md index c6ac38b098..f0c0190755 100644 --- a/COMPILING.md +++ b/COMPILING.md @@ -66,7 +66,19 @@ make sudo make install # optional ``` -To control the server with systemd, runtime options and similar, refer to the [Server manual](https://jamulus.io/wiki/Server-Linux). +To control the server with systemd, runtime options and similar, refer to the [Server manual](https://jamulus.io/wiki/Server-Linux). If you plan to copy the binary onto other machines, read [DEPLOY.md](DEPLOY.md) first — architecture and library mismatches between build and target hosts are the most common cause of broken deployments. + +### Troubleshooting: moc errors with GCC 13+ + +On distributions shipping GCC 13 or later (e.g. Ubuntu 24.04), Qt 5's `moc` can fail to parse the system headers because of the `_GLIBCXX_VISIBILITY` macro. Workaround — after `qmake`, generate `moc_predefs.h` and neutralize the macro, then build: + +```shell +make moc_predefs.h +printf "\n#undef _GLIBCXX_VISIBILITY\n#define _GLIBCXX_VISIBILITY(V)\n" >> moc_predefs.h +make +``` + +(If your Makefile builds into `release/`, the file is `release/moc_predefs.h` and is produced by `make -f Makefile.Release release/moc_predefs.h`.) --- diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d16f02122c..7657b0f115 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -7,6 +7,8 @@ We’d really appreciate your support! Please ensure that you understand the fol - Otherwise, please [post on the GitHub Discussions](https://github.com/jamulussoftware/jamulus/discussions) and say that you are planning to do some coding and explain why. Then we can discuss the specification. - Please begin coding only after we have agreed on a specification to avoid putting a lot of effort into something that may not be accepted later. +If you work with an AI coding agent, [AGENTS.md](AGENTS.md) is its entry point into this repository. Everything in this document applies to agent-assisted contributions without exception — you remain the author, and you are expected to understand and stand behind every line you submit. + ## Jamulus project/source code general principles @@ -39,7 +41,7 @@ Please see the [.clang_format file](https://github.com/jamulussoftware/jamulus/b - Insert a space before and after `(` and `)`. There should be no space between `)` and `;` or before an empty `()`. - Enclose all bodies of `if`, `else`, `while`, `for`, etc. in braces `{` and `}` on separate lines. -- Do not use concatinations in strings with parameters. Instead use substitutions. **Do:** `QString ( tr ( "Hello, %1. Have a nice day!" ) ).arg( getName() )` **Don't:** `tr ( "Hello " ) + getName() + tr ( ". Have a nice day!" )` ...to make translation easier. +- Do not use concatenations in strings with parameters. Instead use substitutions. **Do:** `QString ( tr ( "Hello, %1. Have a nice day!" ) ).arg( getName() )` **Don't:** `tr ( "Hello " ) + getName() + tr ( ". Have a nice day!" )` ...to make translation easier. #### Python Please install and use [pylint](https://pylint.org/) to scan any Python code. diff --git a/DEPLOY.md b/DEPLOY.md new file mode 100644 index 0000000000..a42ff02635 --- /dev/null +++ b/DEPLOY.md @@ -0,0 +1,49 @@ +# Deploying a Jamulus Server + +[COMPILING.md](COMPILING.md) ends when the binary exists. This document covers the step after that: putting a self-built headless server binary on a production host and verifying it actually runs. Most self-inflicted server outages happen in this step, and every rule below corresponds to a real-world failure. + +For configuring and operating a server (registration, recording, welcome message, etc.), see the [Server manual](https://jamulus.io/wiki/Running-a-Server). + +## Build for the target, not for the build machine + +- **CPU architecture must match.** A binary built on x86-64 will not run on an ARM host (and vice versa) — the service crash-loops with `Exec format error`. Before copying, compare `file ./Jamulus` on the build machine with `uname -m` on the target. +- **Build on the oldest OS release you deploy to.** Binaries depend on the glibc/libstdc++ of the build machine. A binary built on Ubuntu 24.04 fails on 22.04 with `GLIBCXX_3.4.32 not found`, while a 22.04 build runs fine on 24.04 and later. Newer hosts run older binaries; the reverse never holds. +- **Check shared libraries after every copy.** `ldd /path/to/jamulus | grep "not found"` must print nothing. This catches a missing Qt runtime package before systemd shows you a crash loop. The minimal headless runtime needs the Qt core, network, concurrent and xml libraries (see [COMPILING.md](COMPILING.md)). +- **Low-memory hosts:** on machines with ≤ 1 GB RAM, build with `make -j1`, or build on a bigger machine of the same OS/architecture and copy the binary. If you must add temporary swap to survive a build, remove it afterwards — see below. + +## Run under systemd + +Use the unit shipped in [`linux/debian/jamulus-headless.service`](linux/debian/jamulus-headless.service) as your starting point — it already encodes hard-won defaults (dedicated `jamulus` user, `Nice=-20`, real-time I/O scheduling, `MemorySwapMax=0`, `Restart=on-failure`). Manage the server only through `systemctl`; never kill the process by PID. + +## Host tuning + +- **Enlarge the UDP receive buffer.** Under load, default kernel buffers drop packets, which musicians hear as dropouts. Set in `/etc/sysctl.d/99-jamulus.conf`: + + ``` + net.core.rmem_max=4194304 + net.core.rmem_default=4194304 + ``` + +- **No swap on a live server.** Swapping causes latency spikes audible to everyone connected. Keep swap off (the shipped systemd unit sets `MemorySwapMax=0` for the service; better still, don't enable swap on the host at all). +- **Don't compete with the audio process.** Never compile, or run other CPU-heavy work, on a host while musicians are connected — CPU contention causes dropouts just like network loss does. + +## Firewall + +- Inbound UDP on the server port (default 22224) must be open. Remember that cloud providers filter *in front of* the host (AWS security groups, OCI security lists) in addition to any host firewall, and some images run `firewalld` by default — you may need to open the port in two places. +- If you enable JSON-RPC (`--jsonrpcport`), never expose that TCP port to the internet. Bind it to localhost or firewall it to specific trusted addresses, and always use `--jsonrpcsecretfile`. +- Corollary: a TCP probe of a firewalled port from outside proves nothing about the service. Verify RPC from an allowed host, not from the internet. + +## Verify every deploy + +A deploy is finished when all of these pass on the target host — not when the file lands: + +```bash +file /usr/bin/jamulus-headless # architecture matches `uname -m` +ldd /usr/bin/jamulus-headless | grep "not found" # no output +/usr/bin/jamulus-headless --version # the version you just built +systemctl status jamulus-headless # active (running) +``` + +Then check the restart counter is stable (`systemctl show -p NRestarts jamulus-headless`, again a minute later — a crash loop can look "active" in a single snapshot), and finally do an end-to-end test: if the server is registered with a directory, confirm it appears in the listing, and connect a client to confirm audio passes. + +When a change "isn't working" on a server, check the binary's timestamp against the commit you think it contains *before* debugging — a stale binary explains most such mysteries. From ada1a84704e0ee0ed07983aa8ac17ae489273ae5 Mon Sep 17 00:00:00 2001 From: jrd Date: Fri, 17 Jul 2026 20:22:13 +0000 Subject: [PATCH 2/3] Address review: move DEPLOY.md and architecture content into docs/ - DEPLOY.md -> docs/DEPLOY.md, links adjusted, license header added - The "Architecture laws" section of AGENTS.md moves to a new docs/ARCHITECTURE.md written for humans as well as agents: system overview, source map, threading model, connection/protocol model, and the three stability invariants. AGENTS.md now links to it. - Reworded the protocol rule to make clear it is preventive (message IDs are frozen), not a report of past renumbering - Dropped the merge-mechanics bullet from AGENTS.md Co-Authored-By: Claude Fable 5 --- AGENTS.md | 14 ++++---- COMPILING.md | 2 +- docs/ARCHITECTURE.md | 70 +++++++++++++++++++++++++++++++++++++ DEPLOY.md => docs/DEPLOY.md | 15 ++++++-- 4 files changed, 90 insertions(+), 11 deletions(-) create mode 100644 docs/ARCHITECTURE.md rename DEPLOY.md => docs/DEPLOY.md (81%) diff --git a/AGENTS.md b/AGENTS.md index 16fbdb67de..1db3550dbc 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -6,7 +6,8 @@ This file is the starting point for coding agents. It links to the detailed docs - [CONTRIBUTING.md](CONTRIBUTING.md) — process, code style, licensing, ownership. Its rules apply to agent-assisted work without exception. - [COMPILING.md](COMPILING.md) — building on every supported platform. -- [DEPLOY.md](DEPLOY.md) — moving a self-built server binary onto production hosts and verifying it. +- [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) — source map, threading model, and the invariants that protect live performances. +- [docs/DEPLOY.md](docs/DEPLOY.md) — moving a self-built server binary onto production hosts and verifying it. - [docs/JAMULUS_PROTOCOL.md](docs/JAMULUS_PROTOCOL.md) — the wire protocol. - [SECURITY.md](SECURITY.md) — report vulnerabilities to team@jamulus.io, never in a public issue. @@ -16,13 +17,13 @@ This file is the starting point for coding agents. It links to the detailed docs - Features must be agreed in a GitHub issue or Discussion **before** coding — see CONTRIBUTING.md. If no agreement exists, propose; don't implement. - Stability outranks everything: Jamulus runs live performances. Prefer not adding a feature over adding risk (KISS principles in CONTRIBUTING.md). -## Architecture laws +## Architecture invariants -Violating these causes audible failures for every connected musician, so they are non-negotiable: +Read [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) before changing code. Its three invariants, in brief: -1. **Never block the real-time path.** The audio capture/playback callbacks (`src/sound/`) and the server's UDP receive → mix → send path (`socket.cpp`, `channel.cpp`, `server.cpp`) must not perform blocking I/O (DNS, HTTP, disk, database), wait on locks contended by non-real-time threads, or do unbounded allocation per packet. Slow work must run asynchronously on another thread; when its result isn't ready, the real-time path fails open and continues. A single synchronous lookup here freezes the audio of everyone on the server. -2. **The wire protocol is a compatibility contract.** `protocol.cpp` must interoperate with every older client and server in the wild. Never renumber `PROTMESSID_*` values, never change the layout of an existing message — extend only by adding new message IDs. Read the header comment in `src/protocol.cpp` and [docs/JAMULUS_PROTOCOL.md](docs/JAMULUS_PROTOCOL.md) before touching it. -3. **All network input is adversarial.** UDP packets arrive from arbitrary internet hosts. Bounds-check every length and count field before use. A malformed packet must be dropped silently — it must never crash, block, or corrupt the server. +1. Never block the real-time audio path (sound callbacks, socket thread, server mix timer). +2. Existing wire-protocol message IDs and layouts are frozen — extend only by adding new message IDs. +3. All network input is untrusted — bounds-check everything; drop malformed packets silently. ## DO NOT edit (generated or third-party) @@ -69,4 +70,3 @@ Violating these causes audible failures for every connected musician, so they ar - Naming your branch `autobuild/` triggers CI builds of all targets on your fork — use it before opening the PR. - Include the `CHANGELOG:` line, and for new dependencies or build changes add `AUTOBUILD: Please build all targets` to the PR description. -- PRs need two approving reviews to merge; the submitter is responsible for follow-up questions and agreed changes (see CONTRIBUTING.md). diff --git a/COMPILING.md b/COMPILING.md index f0c0190755..1144600c50 100644 --- a/COMPILING.md +++ b/COMPILING.md @@ -66,7 +66,7 @@ make sudo make install # optional ``` -To control the server with systemd, runtime options and similar, refer to the [Server manual](https://jamulus.io/wiki/Server-Linux). If you plan to copy the binary onto other machines, read [DEPLOY.md](DEPLOY.md) first — architecture and library mismatches between build and target hosts are the most common cause of broken deployments. +To control the server with systemd, runtime options and similar, refer to the [Server manual](https://jamulus.io/wiki/Server-Linux). If you plan to copy the binary onto other machines, read [docs/DEPLOY.md](docs/DEPLOY.md) first — architecture and library mismatches between build and target hosts are the most common cause of broken deployments. ### Troubleshooting: moc errors with GCC 13+ diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 0000000000..fc3e05d62a --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,70 @@ +### Copyright (c) 2026 + +Author(s): +* The Jamulus Development Team + +Licensed under AGPL 3.0 or any later version. See [COPYING](../COPYING) for details. + +--- + +# Jamulus Architecture + +Jamulus is a client–server system for playing music together in real time. Each client captures local audio, OPUS-encodes it and sends it to a server over UDP. The server decodes every connected client, produces an individual mix for each one, re-encodes and sends it back. All audio flows through the server; clients never exchange audio with each other. One binary contains both roles — `src/main.cpp` starts a client or (with `--server`) a server. + +Audio is processed in frames of 64 or 128 samples at 48 kHz — one UDP packet roughly every 1.3–2.7 ms in each direction, per client. That timescale is why the threading rules below are strict: a delay of a few milliseconds anywhere in the path is audible to everyone. + +## Source map + +| Files | Responsibility | +| --- | --- | +| `src/main.cpp` | Command-line parsing; instantiates client or server | +| `src/client.*` | `CClient`: connects sound card, codec and network on the client side | +| `src/server.*` | `CServer`: the mix engine and channel bookkeeping | +| `src/channel.*` | `CChannel`: one connected client as seen by the server — jitter buffer, gains, per-channel protocol state | +| `src/protocol.*` | `CProtocol`: message framing, acknowledgement/retransmission, split messages | +| `src/socket.*` | `CSocket` / `CHighPrioSocket`: UDP I/O and packet classification | +| `src/buffer.*` | `CNetBuf` / `CNetBufWithStats`: the jitter buffer, including automatic sizing | +| `src/serverlist.*` | `CServerListManager`: directory registration and server lists (both the registering-server and the directory role) | +| `src/serverlogging.*`, `src/recorder/` | Server logging and the jam recorder | +| `src/rpcserver.*`, `src/clientrpc.*`, `src/serverrpc.*` | JSON-RPC interface (see [JSON-RPC.md](JSON-RPC.md)) | +| `src/sound/` | Audio backends (JACK, ASIO, CoreAudio, Oboe), all derived from `CSoundBase` | +| `src/util.*`, `src/settings.*` | Shared helpers and persistent settings | +| `src/*dlg*`, `src/audiomixerboard.*` | Qt GUI — excluded from `CONFIG+=headless` builds | + +## Threading model + +The server runs three execution contexts: + +1. **Socket thread** (`CHighPrioSocket`, started with `QThread::TimeCriticalPriority`): a receive loop classifies each datagram (see below). Audio packets are pushed into the owning channel's jitter buffer directly on this thread; protocol messages are re-emitted as queued Qt signals to the main thread. +2. **Mix timer** (`CHighPrecisionTimer` → `CServer::OnTimer`): the server's heartbeat. Every system frame it pulls one block per connected channel from the jitter buffers, decodes, mixes a personal output for each client, encodes and sends. +3. **Qt main event loop**: protocol message handling, chat, directory registration, JSON-RPC, GUI. + +The client is analogous: the sound backend's hardware callback (`CSoundBase`) drives encode → send and receive → decode, a socket thread handles incoming packets, and the main thread runs GUI and protocol. + +**Invariant 1 — never block the real-time path.** Contexts 1 and 2 and the sound callback must not perform blocking I/O (disk, DNS, HTTP, database), wait on locks contended by non-real-time threads, or allocate unboundedly per packet. Slow work belongs on the main thread or an async helper; when its result is not ready, the real-time path fails open and carries on. One synchronous lookup here freezes the audio of everyone on the server. + +## How a "connection" works + +There is no handshake — the protocol is pure UDP. For every received datagram, `CSocket::ProcessPacket()` first tries to parse it as a protocol frame (two zero tag bytes, valid CRC); anything that fails the parse is treated as audio and routed to a channel by source address. A valid audio packet from an unknown address *is* the connection request: the server allocates a channel for it. The channel times out when audio stops arriving (`CChannel::IsConnected()`); an orderly disconnect is the client repeating `CLM_DISCONNECTION` until the server stops streaming to it. The protocol setup exchange (client ID, transport properties, channel info) then runs concurrently with audio, in no guaranteed order — see [JAMULUS_PROTOCOL.md](JAMULUS_PROTOCOL.md). + +## The wire protocol is a compatibility contract + +Messages come in two classes: + +- **Connection-based** (ID < 1000): carried per-channel by a `CProtocol` instance, sequence-counted, acknowledged and retransmitted until acked. +- **Connectionless** (`CLM_*`, ID ≥ 1000): no channel, no acknowledgement — used where no connection exists yet: ping, directory registration, server list queries, NAT hole punching. + +**Invariant 2 — existing message IDs and layouts are frozen.** Every released client and server must keep interoperating, so the protocol is extended only by adding new message IDs, never by renumbering or changing the layout of existing ones. Read the header comment in `src/protocol.cpp` and [JAMULUS_PROTOCOL.md](JAMULUS_PROTOCOL.md) before touching protocol code. + +**Invariant 3 — all network input is untrusted.** Packets arrive from arbitrary internet hosts. Bounds-check every length and count field before use; a malformed packet must be dropped silently — it must never crash, block, or corrupt state. + +## Directory servers + +A directory is a Jamulus server acting as a registry. Ordinary servers register with it (`CLM_REGISTER_SERVER_EX`) and re-register periodically as a keepalive; clients ask it for the server list (`CLM_REQ_SERVER_LIST`). Because listed servers may sit behind NAT, the directory also brokers hole punching: it asks a server to send `CLM_SEND_EMPTY_MESSAGE` to a client's address so the client's follow-up packets can get through. Both sides of this are implemented in `CServerListManager`. + +## Where common changes go + +- **New protocol message**: add the `PROTMESSID_*` define in `protocol.h`, the create/parse functions in `protocol.cpp`, wire the signal into `channel`/`client`/`server`, and document it in [JAMULUS_PROTOCOL.md](JAMULUS_PROTOCOL.md). +- **New sound backend**: subclass `CSoundBase` under `src/sound/`. +- **New JSON-RPC method**: `clientrpc.*`/`serverrpc.*`, then regenerate [JSON-RPC.md](JSON-RPC.md) with `tools/generate_json_rpc_docs.py`. +- **GUI changes**: keep `CONFIG+=headless serveronly` building — no GUI code may be required by the core. diff --git a/DEPLOY.md b/docs/DEPLOY.md similarity index 81% rename from DEPLOY.md rename to docs/DEPLOY.md index a42ff02635..db6cd3b9ad 100644 --- a/DEPLOY.md +++ b/docs/DEPLOY.md @@ -1,6 +1,15 @@ +### Copyright (c) 2026 + +Author(s): +* The Jamulus Development Team + +Licensed under AGPL 3.0 or any later version. See [COPYING](../COPYING) for details. + +--- + # Deploying a Jamulus Server -[COMPILING.md](COMPILING.md) ends when the binary exists. This document covers the step after that: putting a self-built headless server binary on a production host and verifying it actually runs. Most self-inflicted server outages happen in this step, and every rule below corresponds to a real-world failure. +[COMPILING.md](../COMPILING.md) ends when the binary exists. This document covers the step after that: putting a self-built headless server binary on a production host and verifying it actually runs. Most self-inflicted server outages happen in this step, and every rule below corresponds to a real-world failure. For configuring and operating a server (registration, recording, welcome message, etc.), see the [Server manual](https://jamulus.io/wiki/Running-a-Server). @@ -8,12 +17,12 @@ For configuring and operating a server (registration, recording, welcome message - **CPU architecture must match.** A binary built on x86-64 will not run on an ARM host (and vice versa) — the service crash-loops with `Exec format error`. Before copying, compare `file ./Jamulus` on the build machine with `uname -m` on the target. - **Build on the oldest OS release you deploy to.** Binaries depend on the glibc/libstdc++ of the build machine. A binary built on Ubuntu 24.04 fails on 22.04 with `GLIBCXX_3.4.32 not found`, while a 22.04 build runs fine on 24.04 and later. Newer hosts run older binaries; the reverse never holds. -- **Check shared libraries after every copy.** `ldd /path/to/jamulus | grep "not found"` must print nothing. This catches a missing Qt runtime package before systemd shows you a crash loop. The minimal headless runtime needs the Qt core, network, concurrent and xml libraries (see [COMPILING.md](COMPILING.md)). +- **Check shared libraries after every copy.** `ldd /path/to/jamulus | grep "not found"` must print nothing. This catches a missing Qt runtime package before systemd shows you a crash loop. The minimal headless runtime needs the Qt core, network, concurrent and xml libraries (see [COMPILING.md](../COMPILING.md)). - **Low-memory hosts:** on machines with ≤ 1 GB RAM, build with `make -j1`, or build on a bigger machine of the same OS/architecture and copy the binary. If you must add temporary swap to survive a build, remove it afterwards — see below. ## Run under systemd -Use the unit shipped in [`linux/debian/jamulus-headless.service`](linux/debian/jamulus-headless.service) as your starting point — it already encodes hard-won defaults (dedicated `jamulus` user, `Nice=-20`, real-time I/O scheduling, `MemorySwapMax=0`, `Restart=on-failure`). Manage the server only through `systemctl`; never kill the process by PID. +Use the unit shipped in [`linux/debian/jamulus-headless.service`](../linux/debian/jamulus-headless.service) as your starting point — it already encodes hard-won defaults (dedicated `jamulus` user, `Nice=-20`, real-time I/O scheduling, `MemorySwapMax=0`, `Restart=on-failure`). Manage the server only through `systemctl`; never kill the process by PID. ## Host tuning From d46a77a1420345c4e4f058b677f3e5a97dc4d6f2 Mon Sep 17 00:00:00 2001 From: jrd Date: Fri, 17 Jul 2026 20:32:49 +0000 Subject: [PATCH 3/3] Address review: split new docs files into their own PRs; drop renumbering rule MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit docs/DEPLOY.md and docs/ARCHITECTURE.md move to separate one-file PRs so nothing here blocks their review. The protocol-renumbering rule is removed — it described a problem nobody has had. Co-Authored-By: Claude Fable 5 --- AGENTS.md | 5 ++-- docs/ARCHITECTURE.md | 70 -------------------------------------------- docs/DEPLOY.md | 58 ------------------------------------ 3 files changed, 2 insertions(+), 131 deletions(-) delete mode 100644 docs/ARCHITECTURE.md delete mode 100644 docs/DEPLOY.md diff --git a/AGENTS.md b/AGENTS.md index 1db3550dbc..5b7e23043e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -19,11 +19,10 @@ This file is the starting point for coding agents. It links to the detailed docs ## Architecture invariants -Read [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) before changing code. Its three invariants, in brief: +Read [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) before changing code. Its invariants, in brief: 1. Never block the real-time audio path (sound callbacks, socket thread, server mix timer). -2. Existing wire-protocol message IDs and layouts are frozen — extend only by adding new message IDs. -3. All network input is untrusted — bounds-check everything; drop malformed packets silently. +2. All network input is untrusted — bounds-check everything; drop malformed packets silently. ## DO NOT edit (generated or third-party) diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md deleted file mode 100644 index fc3e05d62a..0000000000 --- a/docs/ARCHITECTURE.md +++ /dev/null @@ -1,70 +0,0 @@ -### Copyright (c) 2026 - -Author(s): -* The Jamulus Development Team - -Licensed under AGPL 3.0 or any later version. See [COPYING](../COPYING) for details. - ---- - -# Jamulus Architecture - -Jamulus is a client–server system for playing music together in real time. Each client captures local audio, OPUS-encodes it and sends it to a server over UDP. The server decodes every connected client, produces an individual mix for each one, re-encodes and sends it back. All audio flows through the server; clients never exchange audio with each other. One binary contains both roles — `src/main.cpp` starts a client or (with `--server`) a server. - -Audio is processed in frames of 64 or 128 samples at 48 kHz — one UDP packet roughly every 1.3–2.7 ms in each direction, per client. That timescale is why the threading rules below are strict: a delay of a few milliseconds anywhere in the path is audible to everyone. - -## Source map - -| Files | Responsibility | -| --- | --- | -| `src/main.cpp` | Command-line parsing; instantiates client or server | -| `src/client.*` | `CClient`: connects sound card, codec and network on the client side | -| `src/server.*` | `CServer`: the mix engine and channel bookkeeping | -| `src/channel.*` | `CChannel`: one connected client as seen by the server — jitter buffer, gains, per-channel protocol state | -| `src/protocol.*` | `CProtocol`: message framing, acknowledgement/retransmission, split messages | -| `src/socket.*` | `CSocket` / `CHighPrioSocket`: UDP I/O and packet classification | -| `src/buffer.*` | `CNetBuf` / `CNetBufWithStats`: the jitter buffer, including automatic sizing | -| `src/serverlist.*` | `CServerListManager`: directory registration and server lists (both the registering-server and the directory role) | -| `src/serverlogging.*`, `src/recorder/` | Server logging and the jam recorder | -| `src/rpcserver.*`, `src/clientrpc.*`, `src/serverrpc.*` | JSON-RPC interface (see [JSON-RPC.md](JSON-RPC.md)) | -| `src/sound/` | Audio backends (JACK, ASIO, CoreAudio, Oboe), all derived from `CSoundBase` | -| `src/util.*`, `src/settings.*` | Shared helpers and persistent settings | -| `src/*dlg*`, `src/audiomixerboard.*` | Qt GUI — excluded from `CONFIG+=headless` builds | - -## Threading model - -The server runs three execution contexts: - -1. **Socket thread** (`CHighPrioSocket`, started with `QThread::TimeCriticalPriority`): a receive loop classifies each datagram (see below). Audio packets are pushed into the owning channel's jitter buffer directly on this thread; protocol messages are re-emitted as queued Qt signals to the main thread. -2. **Mix timer** (`CHighPrecisionTimer` → `CServer::OnTimer`): the server's heartbeat. Every system frame it pulls one block per connected channel from the jitter buffers, decodes, mixes a personal output for each client, encodes and sends. -3. **Qt main event loop**: protocol message handling, chat, directory registration, JSON-RPC, GUI. - -The client is analogous: the sound backend's hardware callback (`CSoundBase`) drives encode → send and receive → decode, a socket thread handles incoming packets, and the main thread runs GUI and protocol. - -**Invariant 1 — never block the real-time path.** Contexts 1 and 2 and the sound callback must not perform blocking I/O (disk, DNS, HTTP, database), wait on locks contended by non-real-time threads, or allocate unboundedly per packet. Slow work belongs on the main thread or an async helper; when its result is not ready, the real-time path fails open and carries on. One synchronous lookup here freezes the audio of everyone on the server. - -## How a "connection" works - -There is no handshake — the protocol is pure UDP. For every received datagram, `CSocket::ProcessPacket()` first tries to parse it as a protocol frame (two zero tag bytes, valid CRC); anything that fails the parse is treated as audio and routed to a channel by source address. A valid audio packet from an unknown address *is* the connection request: the server allocates a channel for it. The channel times out when audio stops arriving (`CChannel::IsConnected()`); an orderly disconnect is the client repeating `CLM_DISCONNECTION` until the server stops streaming to it. The protocol setup exchange (client ID, transport properties, channel info) then runs concurrently with audio, in no guaranteed order — see [JAMULUS_PROTOCOL.md](JAMULUS_PROTOCOL.md). - -## The wire protocol is a compatibility contract - -Messages come in two classes: - -- **Connection-based** (ID < 1000): carried per-channel by a `CProtocol` instance, sequence-counted, acknowledged and retransmitted until acked. -- **Connectionless** (`CLM_*`, ID ≥ 1000): no channel, no acknowledgement — used where no connection exists yet: ping, directory registration, server list queries, NAT hole punching. - -**Invariant 2 — existing message IDs and layouts are frozen.** Every released client and server must keep interoperating, so the protocol is extended only by adding new message IDs, never by renumbering or changing the layout of existing ones. Read the header comment in `src/protocol.cpp` and [JAMULUS_PROTOCOL.md](JAMULUS_PROTOCOL.md) before touching protocol code. - -**Invariant 3 — all network input is untrusted.** Packets arrive from arbitrary internet hosts. Bounds-check every length and count field before use; a malformed packet must be dropped silently — it must never crash, block, or corrupt state. - -## Directory servers - -A directory is a Jamulus server acting as a registry. Ordinary servers register with it (`CLM_REGISTER_SERVER_EX`) and re-register periodically as a keepalive; clients ask it for the server list (`CLM_REQ_SERVER_LIST`). Because listed servers may sit behind NAT, the directory also brokers hole punching: it asks a server to send `CLM_SEND_EMPTY_MESSAGE` to a client's address so the client's follow-up packets can get through. Both sides of this are implemented in `CServerListManager`. - -## Where common changes go - -- **New protocol message**: add the `PROTMESSID_*` define in `protocol.h`, the create/parse functions in `protocol.cpp`, wire the signal into `channel`/`client`/`server`, and document it in [JAMULUS_PROTOCOL.md](JAMULUS_PROTOCOL.md). -- **New sound backend**: subclass `CSoundBase` under `src/sound/`. -- **New JSON-RPC method**: `clientrpc.*`/`serverrpc.*`, then regenerate [JSON-RPC.md](JSON-RPC.md) with `tools/generate_json_rpc_docs.py`. -- **GUI changes**: keep `CONFIG+=headless serveronly` building — no GUI code may be required by the core. diff --git a/docs/DEPLOY.md b/docs/DEPLOY.md deleted file mode 100644 index db6cd3b9ad..0000000000 --- a/docs/DEPLOY.md +++ /dev/null @@ -1,58 +0,0 @@ -### Copyright (c) 2026 - -Author(s): -* The Jamulus Development Team - -Licensed under AGPL 3.0 or any later version. See [COPYING](../COPYING) for details. - ---- - -# Deploying a Jamulus Server - -[COMPILING.md](../COMPILING.md) ends when the binary exists. This document covers the step after that: putting a self-built headless server binary on a production host and verifying it actually runs. Most self-inflicted server outages happen in this step, and every rule below corresponds to a real-world failure. - -For configuring and operating a server (registration, recording, welcome message, etc.), see the [Server manual](https://jamulus.io/wiki/Running-a-Server). - -## Build for the target, not for the build machine - -- **CPU architecture must match.** A binary built on x86-64 will not run on an ARM host (and vice versa) — the service crash-loops with `Exec format error`. Before copying, compare `file ./Jamulus` on the build machine with `uname -m` on the target. -- **Build on the oldest OS release you deploy to.** Binaries depend on the glibc/libstdc++ of the build machine. A binary built on Ubuntu 24.04 fails on 22.04 with `GLIBCXX_3.4.32 not found`, while a 22.04 build runs fine on 24.04 and later. Newer hosts run older binaries; the reverse never holds. -- **Check shared libraries after every copy.** `ldd /path/to/jamulus | grep "not found"` must print nothing. This catches a missing Qt runtime package before systemd shows you a crash loop. The minimal headless runtime needs the Qt core, network, concurrent and xml libraries (see [COMPILING.md](../COMPILING.md)). -- **Low-memory hosts:** on machines with ≤ 1 GB RAM, build with `make -j1`, or build on a bigger machine of the same OS/architecture and copy the binary. If you must add temporary swap to survive a build, remove it afterwards — see below. - -## Run under systemd - -Use the unit shipped in [`linux/debian/jamulus-headless.service`](../linux/debian/jamulus-headless.service) as your starting point — it already encodes hard-won defaults (dedicated `jamulus` user, `Nice=-20`, real-time I/O scheduling, `MemorySwapMax=0`, `Restart=on-failure`). Manage the server only through `systemctl`; never kill the process by PID. - -## Host tuning - -- **Enlarge the UDP receive buffer.** Under load, default kernel buffers drop packets, which musicians hear as dropouts. Set in `/etc/sysctl.d/99-jamulus.conf`: - - ``` - net.core.rmem_max=4194304 - net.core.rmem_default=4194304 - ``` - -- **No swap on a live server.** Swapping causes latency spikes audible to everyone connected. Keep swap off (the shipped systemd unit sets `MemorySwapMax=0` for the service; better still, don't enable swap on the host at all). -- **Don't compete with the audio process.** Never compile, or run other CPU-heavy work, on a host while musicians are connected — CPU contention causes dropouts just like network loss does. - -## Firewall - -- Inbound UDP on the server port (default 22224) must be open. Remember that cloud providers filter *in front of* the host (AWS security groups, OCI security lists) in addition to any host firewall, and some images run `firewalld` by default — you may need to open the port in two places. -- If you enable JSON-RPC (`--jsonrpcport`), never expose that TCP port to the internet. Bind it to localhost or firewall it to specific trusted addresses, and always use `--jsonrpcsecretfile`. -- Corollary: a TCP probe of a firewalled port from outside proves nothing about the service. Verify RPC from an allowed host, not from the internet. - -## Verify every deploy - -A deploy is finished when all of these pass on the target host — not when the file lands: - -```bash -file /usr/bin/jamulus-headless # architecture matches `uname -m` -ldd /usr/bin/jamulus-headless | grep "not found" # no output -/usr/bin/jamulus-headless --version # the version you just built -systemctl status jamulus-headless # active (running) -``` - -Then check the restart counter is stable (`systemctl show -p NRestarts jamulus-headless`, again a minute later — a crash loop can look "active" in a single snapshot), and finally do an end-to-end test: if the server is registered with a directory, confirm it appears in the listing, and connect a client to confirm audio passes. - -When a change "isn't working" on a server, check the binary's timestamp against the commit you think it contains *before* debugging — a stale binary explains most such mysteries.