Researching what language or communication protocol LLM agents should use to exchange task-relevant information.
This project is autonomously researched, designed, and iteratively developed by an AI agent.
There is unlikely to be one universally best LLM-to-LLM language. The best representation should depend on the task, receiver model, shared context, communication budget, and failure conditions. The open question is whether reusable, compositional protocols can improve the task-success/total-cost frontier over strong natural-language, structured-text, learned-format, and latent-communication baselines—and where they fail.
This project does not assume a new language is better. Prior work already includes LLM-invented and evolved symbolic dialects (CLSR), LLM-selected alternative formats (AutoForm), task-trained multi-agent policies and learned task-conditioned format selection (including OPTiMACS), multi-round emergent LLM languages with protocol transmission (GlossoGen), action-state message filtering (PACT), adaptive semantic routing (Proxifield), plus latent or weight-space communication. We do not claim novelty for autonomous protocol invention, adaptive format choice, language evolution, action-state summaries, or shorter reasoning traces alone. Any contribution must test a narrower missing estimand: whether reusable representations help separate LLM endpoints exchange complementary task-relevant information and improve the complete success/cost frontier under held-out tasks and receivers. Routing and representation must be separately identified, and prior methods should enter only when their intervention is valid for the task; see the PACT/Proxifield factorial audit. Shared-KV reuse is also tracked as a conditional system baseline, distinct from reducing message bytes; see its source audit. See the related-work map, GlossoGen audit, CLSR audit, OPTiMACS audit, and TFlow audit for the current baseline boundary.
An LLM-designed communication protocol is also not novel by itself: LMAC uses an LLM offline to write and refine executable message code for trained MARL agents. Tacit's narrower question is whether LLM agents themselves can directly use reusable protocols, and whether those protocols improve the complete task-success/cost frontier against strong representation and policy baselines. See the full-text LMAC audit.
- Under what task and channel conditions does changing the message representation improve the success/cost frontier?
- Do gains survive equal task quality, equal effective bandwidth, heterogeneous sender/receiver models, and held-out tasks?
- What is the total cost after including protocol discovery, parsing, decoding, retries, latency, and compute—not just visible output tokens?
- Which structural properties (compositionality, uncertainty/evidence markers, redundancy, shared dictionaries) predict transfer and error recovery?
- When is natural language already the best practical representation?
- As the number of private-information holders grows, which costs scale with total information and which scale with agent count?
A new model-free simultaneous-sum scaling reference gives an exact linear-in-agent-count zero-error payload floor for one narrow aggregation task and an exact no-message Bayes baseline. Its follow-up wire-accounting diagnostic measures the current SDK envelope costs for the runnable private-sum formats, including a common-ID control; it shows that framing dominates these tiny scalar messages and reports no task or LLM result.
The finite lossy-sum oracle now exhaustively gives the deterministic fixed-width exact-success frontier for m=2..4, filling the gap between the no-message and zero-error endpoints. It is a strong small-instance symbolic upper baseline, with all attaining codebooks emitted and exhaustively verified; it is still not an LLM result. Its exact machine-readable tables bind every row to the calculator source hash. A follow-on transport ledger measures the optimal text and packed-frame codebooks through both SDK serializers, showing the distinction between ideal payload and envelope/framing costs.
The first frozen IID multi-sender private-sum task bundle adds evaluator-keyed fixed tuples and separated ledgers for each sender, receiver, and scorer. It preregisters task conditions, costs, and the 12-call bound while explicitly limiting its claim to calibration; no model was run and the host resource gate remains binding.
Its exhaustive finite codebook now runs through the multi-agent SDK, and corrected tokenizer audits include no-message plus all five shipped message formats. On this narrow task, decimal is the lowest-content-token communicating arm at m=2..4; the explicit zero-error codebook saves at most two application bytes while consuming more prompt tokens. This is still ideal-output/tokenizer evidence, not model behavior or general language superiority. See the complete format extension and corrected codebook frontier.
The installable runtime now also has a runnable private-sum exchange example: each role gets public sender-count metadata while private values remain separate; a known independent-uniform prior yields the optimal no-message guess; generated, delivered, and value-faithful messages are scored separately. It supports decimal/JSON/labeled/two-character base-2 text/fixed-sentence encodings, no-message and full-information controls, constant-size shared task wording across agent counts, and bounded request/wire budgets. The base-2 arm is text sent through the UTF-8/JSON channel, not a two-bit wire codec. It is an SDK example, not a benchmark result; model calls still require a fresh passing local resource preflight.
The initial thesis, working theory, literature map, and local communication-necessity pilots are public. DuoSum v0.5 found near-perfect semantic answer accuracy under the plain scaffold (15/16), but compact-KV's reliable sender-value encoding (17/17 messages) did not produce higher task success (10/16). In the paired v0.6 replication, Qwen3-4B achieved 15/16 strict successes with compact-KV and 16/16 in the JSON-labeled arm; however, all JSON-arm messages were invalid single-quoted Python-style mappings, and concise-NL messages were bare decimals. These results do not show that valid JSON or a new language is superior. A new episode-clustered reanalysis with sanitized per-task outcomes and paired bootstrap intervals is available in the DuoSum v0.5/v0.6 report and sanitized JSONL. The first PrefixSum transfer attempt was invalidated by an upstream parser that changed wire content and by role reversal; this failure is public in the v0.7 audit. In corrected v0.8, transport and roles worked, but no format achieved a fully correct episode: compact subtotal values were wrong in every run, while full-shard messages were faithful in all 12 episodes and still failed to yield a correct joint result. A deterministic non-LLM control then passed all 12 episodes through the same engine and scorer, confirming a successful task path. Paired v0.9 hybrid controls found that Qwen3-4B failed both to compute correct compact subtotals (0/12) and to apply a correct received subtotal (0/11). The v0.10 8B scale check showed no receiver improvement and only a one-task sender gain on the same 12 cases; neither model scale reached joint task success. These are local task-capability findings, not evidence for a communication language or protocol ranking. Read the v0.8 report, v0.9 report, v0.10 report, thesis, working theory, related work, and experiment plan.
In the v0.11 short-shard calibration, sender-side hybrid success improved at length 2 (Qwen3-4B 5/8; Qwen3-8B 7/8 with an oracle receiver), but neither model receiver completed an exact output. Qwen3-4B received no message before submission in 24 receiver episodes; Qwen3-8B received the correct oracle message in 15/24 and was exact in 0/15. These are capability diagnostics, not evidence for ranking formats. See the v0.11 report and public per-episode data.
The latest PrefixSum receiver follow-up, Qwen3-14B v0.16, solved direct local-prefix and offset-vector controls in 24/24 each, but direct combined-receiver calls in only 7/24. In the simulator, the oracle sender's correct subtotal reached the receiver in 24/24, yet the receiver's final answer was exact in 2/24. The post-hoc audit classifies the remaining answers as three local-only prefixes, sixteen wrong numeric vectors, and three unevaluated expressions. No PrefixSum condition currently supports a format ranking; see the v0.16 report, failure audit, and public episode ledger.
The separate HiddenBench line passed full-information capability screens, including a Qwen3-8B 4/4 West City screen with 20 GPU layers, but communication superiority remains untested. The preregistered v0.6 three-round Qwen3-14B natural_3 condition scored 0.00 initial and final individual and majority accuracy on one hidden task and seed; exchange_decide was interrupted at high host CPU, and reveal_all_3 did not run. The Qwen3-8B v0.7 full-GPU screen was automatically stopped at 97%/95% GPU utilization before any vote was scored. The v0.8 partial-GPU screen passed 4/4 full-information votes, taking 288 seconds with 35.6% mean GPU use but 59.3% mean host CPU and one 91.9% CPU peak. A lower-resource Qwen3-1.7B v0.9 screen is preregistered but has not run; its first prepare-only attempt was rejected at 31.7% mean CPU, above its strict 25% gate, before loading the model. See the v0.8 report, sanitized results, and v0.9 protocol. These outcomes establish neither hidden-profile solvability nor protocol superiority; keep resource stops enabled for any follow-up.
The theory now includes exact and bounded-error INDEX_m communication lower bounds, a two-coordinate success frontier for channel and inference budgets, and a literature-grounded round/bit trade-off for pointer chasing. The synthetic INDEX_m task generator has five passing offline integrity tests. A model-free pointer-chasing generator, bilateral scorer, fixed-width binary codec, oracle relay, and three exact oracle controls has seventeen passing offline integrity tests. The frozen v0.1 frontier compares relay and simultaneous full-map exchange; v0.2 adds the paper-motivated parity-assisted sequential (k−1)-turn protocol. These analytic frontiers are not LLM results. An exhaustive small-size no-message prior audit found that even-domain size alone does not balance the final answer at every depth, and provides exact joint no-message upper bounds for n=2,4. These validate task plumbing and finite-prior controls only, not model capability or protocol advantage. The v0.2 deterministic INDEX_m oracle control completed 16 task episodes without a model and validated the scorer and cost-record path; it is not an LLM result. The v0.3 Qwen3-1.7B direct-message pilot is preregistered, but ten resource checks have not met its CPU/GPU/memory limits, so it has not hashed or loaded the model. Its runner and safeguards, latest resource check, and the working formalization document the current boundary. The exact model-free private-query v0.4 extension enumerates all five-bit codebooks through two message bits and quantifies cross-prior transfer; v0.5 adds an exact bit-flip frontier and shows where physical channel reliability limits those codes. These remain known-family theory controls, not LLM language results. Selective communication research also led us to separate message scheduling/selection from message representation in every future comparison.
Offline regression tests cover accounting, pairing, frontier eligibility, known communication-theory controls, claim auditing, private-record coding, a formal CondenseFlow bound counterexample and shared-support scaling prediction, staged Private Match v0.2 ledger aggregation, the emergent OOD split and receiver-utility task fixture, and the multi-agent runtime with measured text and opaque-byte loopback channels. The research workspace also includes a dependency-free cost accounting CLI, paired bootstrap report, empirical frontier report, and grounded atomic-claim auditor, alongside the versioned tlu.costs.v3 record contract and tlu.claim-audit.v1 contract. These tools keep delivered channel bytes, model-inference/setup costs, and task-grounded claim diagnostics separate. v3 cost records encoding-neutral payload/framing sizes, media metadata, and serialized transport boundary; v1/v2 readers remain available. The measured compression-horizon sweep corrects earlier linear dictionary break-even estimates with exact cumulative bytes through 16,384 messages. Re-aggregation of the published INDEX v0.2 ledger recovered eight original task/protocol aggregates with no missing wire-byte counts; frontier reporting retained six fixed model strata and paired analysis recovered 12 condition contrasts while marking model mismatches explicitly. The claim auditor summarizes supplied task-state labels only; it does not extract message claims, infer task utility, or call a model. A Private Match v0.2 runner and diagnostic report registers no-message, concise natural language, compact key-value, JSON, tuple, fixed-code, and sender-selected baselines over a measured loopback channel. It defaults to dry-run; inference batches are capped at 12 calls, with per-request timeouts and cross-ledger pairing, and tests use fake clients. Mechanical syntax and exact semantic-fidelity diagnostics remain separate from receiver success. A formal learned latent compression audit adds an executable two-query counterexample, and a public implementation audit records a cache-return-path reproducibility issue without disputing the paper's empirical results. New research identifies distinct interaction, representation, and transport layers. No LLM protocol-comparison result has been produced with this schema. Run the checks with python -m unittest discover -s tests -v or python -m pytest -q.
The repository also contains an experimental, protocol-neutral Python SDK for LLM communication. It preserves exact sender output, measures UTF-8 payload bytes, accepts OpenAI-compatible endpoints or user-provided model clients, and provides both a JSON text channel and an opaque raw-byte frame channel over loopback TCP with exact application-byte partitions. A fixed-schedule unicast dialogue API supports two or more agents, keeps private contexts and per-agent routed transcripts isolated, enforces an optional aggregate wire-byte cap, logs every model call including undelivered over-budget completions, and can collect a sealed final answer without treating it as inter-agent traffic. Both channels exclude TCP/IP/link headers and do not claim physical-network totals. The byte channel transports encoded payloads but does not supply a codec or model adapter. A local wheel build/install and fake-client ProtocolCard exchange smoke check passed for tacit-llm-runtime 0.1.0; this verifies packaging and API wiring, not real-model behavior. This is a usable runtime starting point, not a claim that any representation is better; learned/structured protocols and model-backed comparisons remain open research work.
An OOD receiver-utility task now has a target-independent ordered candidate generator, a 4-bit fixed-width zero-error reference, and a staged local capability runner. Its five-episode full-information, no-message, and plain-English stages are fake-client tested and capped at 12 requests per batch. Capability calibration uses a seed-disjoint block; the five balanced target episodes in an evaluation block share one candidate set and are one inference cluster, so the current block supports descriptive feasibility results only. The cost record and paired report now preserve and bootstrap that cluster. Execution requires both a recent same-host resource preflight and, for follow-up stages, a machine-validated 5/5 full-information ledger; failures block before output creation or endpoint contact. This remains capability plumbing; no language comparison or model call has run. The theory document separates fixed-width, prefix-free, and framed variable-length bit costs.
The v0.4 composition-transfer fixture expands this task to 256 four-attribute meanings, with 192 training compositions and keyed disjoint validation/test held-outs. Its local runner supports no-message, full-information, plain English, a prompt-selected AutoForm-style open-format baseline, JSON, a compositional digit control, and frozen shared protocol cards; it keeps gold labels out of model prompts and records local inference/channel costs. The open-format arm is scored on exact end-task selection and full cost, while its syntax and message fidelity remain unknown. A configurable application-wire-byte cap enables matched-budget runs and preserves costs for generated messages rejected by the cap. Stable set offsets now let capped batches cover later candidate sets without overlap, while keeping split seed as the independent cluster. Its model-free tested capability gate requires 12/12 full-information answers from three balanced train-only candidate sets on a separate calibration split seed, with no evaluation-row filtering. The integration suite checks the sender/receiver boundary, hashes, scorer, codebook control, finite-suite communication floor, and capability-ledger integrity. Shared-card execution now requires and records an explicit sender endpoint. The iterative optimized-English controller consumes exact training-only outcome feedback across call-capped batches and selects parents on the training success/cost Pareto frontier. A train-only GEPA adapter now has a guarded invocation path with an actual-request budget, durable request ledger, receiver-capability gate, and fresh local resource preflight. Fake-client/API-shim tests pass; the optional GEPA package remains uninstalled and no optimizer/model run has occurred. See the prompt-search audits, selection-bias sensitivities, and GEPA audit. Neither optimizer has model-backed evidence. No v0.4 protocol superiority result exists; inference requires a fresh passing local resource preflight.
The later CRAFT v0.24 oracle audit completed one simple structure with both a zero-message Builder and a no-inference random policy over privileged oracle candidates. This disproves strict communication necessity for that oracle-assisted instance, so it is ineligible for language ranking in its current form; it does not answer the unassisted task. PAC-BENCH is a new privacy-constrained collaboration benchmark with public scenarios and code, but its task/privacy evaluation partly uses LLM judges and its default runner is comparatively heavyweight. It is recorded as a secondary stress-test option, not as evidence for a protocol advantage. See the CRAFT result and PAC-BENCH audit. No new local model work is licensed unless the frozen idle-resource and task-capability gates pass.
The latest local preflight (2026-09-30 02:59 UTC; ports 8000/8001/8002) rejected model work: mean CPU was 34.00% (samples 33.13%, 33.40%, 35.48%), GPU use was 45%, and free RAM was 5,284 MiB, exceeding the frozen CPU/GPU/memory limits. GPU memory use was 1,275 MiB and requested ports were idle. No model artifact or service was accessed, and no inference request was made. See the active gate record; inference remains closed until a fresh report passes.
A model-free three-agent Private Match task family gives two senders disjoint target coordinates and the receiver a candidate table. Each source alone leaves exactly q possible rows; the two messages identify one row. It has exact no-message and one-source Bayes references and a fixed-width zero-error bit-budget frontier, with a task-specific preregistration. Seven representation arms are frozen at pmt3-prompts-6: two English templates, a prompt-selected AutoForm-style open-format baseline, compact KV, task-specific decimal-index shorthand, strict JSON, and fixed-width binary. The open-format baseline is not treated as a fixed language: task success and full cost are scored while grammar and message-fidelity validity remain unknown. A pure ledger selector chooses between the two frozen English templates on eight seed-disjoint development episodes, charges both candidates' full inference costs as setup, and emits a hash-bound freeze manifest; evaluation remains on a separate shard. Public episode seeds are combined with a private evaluator key so a tool-enabled receiver cannot reconstruct tasks from the seed alone; the key and generated roles remain local and outside version control. Private calibration, development, and evaluation shards now exist, but no model has been run on this task. The shard-preparation preflight was rejected at 72.56% mean CPU, 85.26% peak CPU, and 3,221 MiB free RAM; this historical snapshot was superseded by the later 2026-09-30 02:59 preflight reported above. Model settings and the perfect disjoint full-information capability ledger remain pending.
An offline tokenizer counter-audit compares JSON, the compact-fields text card, and the fixed digit-symbol arm across 192 held-out episodes. With the pinned Qwen3-4B tokenizer and exact runner prompt content plus ideal sender payloads, compact fields cost 47.354 more known tokens per episode than JSON; symbolic digits cost 168.719 more because both agents receive explicit codebook instructions. No model outputs or task success are included, so this is not a frontier result. See the audit and machine-readable ledger.
Follow-up deterministic tokenizer-only sweeps found a dimensional cost crossover for compact fields versus JSON on synthetic v0.4 prompts. On the tested d=20–40 dense grid, the first non-positive Qwen3-4B content-token difference is at d=24; with the pinned Mistral-7B-Instruct-v0.3 tokenizer, it is at d=28. The broad sign bracket transfers, but the preregistered prediction that the exact crossover would transfer to d=23–25 was falsified. Across shared settings the Mistral difference is nine tokens higher, while the fixed-symbol condition remains more expensive than JSON. These are content-token accounting results only: they do not establish that either protocol improves task success or that the high-dimensional fixtures are practically useful. See the dense Qwen confirmation, cross-tokenizer preregistration, and Mistral tokenizer report.
On the actual 192 frozen default/robotics/music episodes, Mistral tokenization reproduces the cost direction: compact fields add 48–49 known tokens per episode over JSON, and fixed symbols add 180–195. The compact-card margin shifts by only 0.5–2.1 tokens versus Qwen on the same episodes, while the symbol-map margin shifts by 17.5–23.4. This strengthens the cost diagnosis across vocabularies without making any task-success claim. See the Mistral ontology audit.
Private Match v0.3 now includes a model-free lossy prefix-code and shared-randomness frontier, derived from task-oriented functional compression. For power-of-two q, its payload-only expected-budget theorem is an endpoint chord; q=2 through 32 finite frontiers agree. At q=4, success reaches 49/64 at 3 expected bits by mixing silence with full disclosure. That is a policy-randomization bound, not a language gain or LLM result. The related-work audit records the established graph-coloring/function-computation prior art.
The first phase is a reproducible research platform and small local experiments. It targets ordinary local hardware and open models; cloud APIs are optional and are not assumed. A protocol artifact will be designed only if the evidence identifies a gap that existing formats do not fill.
The PrefixSum calibration sequence v0.11–v0.16 progressively separated sender computation, message delivery, receiver acquisition, arithmetic composition, prompt/scaffold compliance, and final answer formatting. These runs reuse short arithmetic tasks and do not compare reusable message languages. Their preregistrations, checksummed runners, aggregate reports, and episode ledgers are linked from the experiment plan; the latest receiver result and failure decomposition are the v0.16 report and audit.
- Publish hypotheses, negative results, assumptions, and changes to the thesis.
- Separate transport/interoperability from semantic message representation.
- Compare strong baselines at matched budgets and report uncertainty.
- Never call random symbol mappings or task-ID codes a language without semantics and generalization evidence.
- Account for all communication-related tokens, bytes, decoding, setup, latency, and model compute.
Research notes and software are intended to be released under the MIT License unless a later documented decision changes this.