Skip to content

0.45: new memory crate - #86

Merged
MathisWellmann merged 13 commits into
mainfrom
memory
Oct 19, 2026
Merged

MathisWellmann merged 13 commits into
mainfrom
memory

Conversation

@MathisWellmann

@MathisWellmann MathisWellmann commented Oct 19, 2026 •

Copy link
Copy Markdown
Owner

Introduces the new crate: symbiont-memory

Archive (archive.rs, render.rs):

  • stores every evaluated attempt (source, finite Score, a report written for the model) up to a set capacity.
  • When full, it evicts the worst entry, but never from a protected recent tail. Bad recent results therefore still warn the next lane.
  • Memory::render shows the best attempts (the top ones with source) and the newest ones within a token budget.
  • Recurring failures (failures.rs): reads EvolutionTraces and caught panics.
  • Counts rustc error headlines once per lane. Free-standing numbers are normalized, so index 5 and index 7 count as one failure.
  • Failures seen at least min_occurrences times render as "Known pitfalls", in the order they were first seen. A rising count therefore never reorders text that is already cached.

ACE playbook (playbook.rs):

  • itemized lessons in ACE's format, [str-00003] helpful=X harmful=Y :: lesson (arXiv:2510.04618).
  • Duplicates are found by word-set Jaccard overlap, so no embedding model is needed.
  • A bullet is pruned when its harmful count exceeds its helpful count by prune_margin.
  • When the playbook is full, its weakest bullet is dropped.
  • By default, a new bullet must cite evidence, i.e. a measured attempt.
  • Hosts can add lessons they already know through playbook_mut.

Reflection (reflection.rs)

  • reflection_request builds one prompt from the attempts committed since the last review, covering each attempt's score against its parents, its report, its code, and the bullets that attempt's lane was shown.
  • The host sends this prompt to any model, then passes the answer to apply_reflection.
  • This combines ACE's Reflector and Curator into one call per round. Merge order is tags, then prune, then add.
  • Each request has an id, so an answer applies at most once. If the answer contains no JSON, the request stays outstanding. abandon_reflection puts the attempts back in the queue after a failed model call.

Islands (island.rs):

  • the classic counter to a population collapsing onto one program (FunSearch, OpenEvolve).
  • Archive capacity is enforced per island.
  • Playbook bullets have a Scope: global or one island.
  • Rendered gets an island part between the shared part and the lane part.
  • migrate moves each island's best attempts one hop around a ring. It also promotes island lessons whose net count reached promote_net to the global playbook.
  • reset_island clears a stagnant island.
  • novelty uses token-trigram Jaccard similarity, ignoring layout and comments, to reject near copies before an expensive evaluation.
  • With the default single island, behaviour is unchanged. The only difference is the explicit IslandId argument.

Observability

  • every state change emits a structured debug-level tracing event. Turn them on with RUST_LOG=symbiont_memory=debug. A traced_test checks the key events.
  • Config for hosts: every config struct is #[serde(default)], so a RON/JSON file only needs the fields it changes. MemoryConfig::validate checks the limits again, because deserializing skips the checking constructors. Memory::new panics on an invalid config so the error shows up immediately, not later.
  • When the host excludes ids it renders itself, the elite section is titled "Best attempts not shown above" instead of the wrong "Best attempts so far".

…ugh [workspace.dependencies]

The coming symbiont-memory crate uses all four, which makes them shared by
two or more crates; one version spec in the root keeps them from drifting.
…t prompt

The runtime drops chat history after every evolve call on purpose, so every
host hand-rolls 'last attempt' and 'best so far' into its prompt. This new,
optional crate makes that a reusable store: Archive keeps every evaluated
attempt (source, finite Score, report written for the model) up to a
capacity, evicting the worst entry outside a protected recent tail so bad
recent results still warn the next lane. Memory::render shows the best
attempts (top ones with source) and the newest ones within a token budget,
as a pure function of the state, so the bytes stay stable for a prefix cache
and a replay.

Plain data owned through &mut self and serializable without internally
tagged enums or non-string map keys, so a host can checkpoint it with
bincode next to its population.
…anics

Weak models repeat the same mistakes across lanes (an invented host
method, the same type mismatch, the same index out of bounds). Every
rejected attempt is already in the lane's EvolutionTrace, so Failures counts
the rustc error headlines (other kinds: their first line) once per lane,
plus the panics the host caught, with free-standing numbers normalized so
'index 5' and 'index 7' are one failure. Failures that reached
min_occurrences render as 'Known pitfalls' in a new shared part of
Rendered, listed in first-seen order so a rising count never reorders the
text a prefix cache holds. No model call is needed.
…l counters

The playbook is the context representation of ACE (arXiv:2510.04618,
github.com/ace-agent/ace): bullets with an id, two counters and one line
of content, in ACE's default sections, rendered in its
'[str-00003] helpful=X harmful=Y :: lesson' format. Bullets are only
added, tagged or removed by deterministic code, never rewritten as a
whole; ACE credits this for avoiding context collapse and brevity bias.

Grow-and-refine without an embedding model: a new bullet whose word set
overlaps an existing one by >= duplicate_percent (Jaccard) is a duplicate,
a bullet tagged harmful prune_margin more times than helpful is removed
(the paper's first defence against noisy reflections), a full playbook
drops its weakest bullet, and rendering keeps the strongest bullets that
fit the token budget. New bullets must cite evidence by default, so a
lesson always points at a measured attempt.

The rendered playbook goes into the shared prompt part before the known
pitfalls; Rendered::bullets records what a lane was shown. Hosts can seed
lessons they already know through Memory::playbook_mut. The model-facing
reflection step follows in the next revision.
…om scored attempts

Memory::reflection_request turns the attempts committed since the last
review into one prompt: each attempt's score against its parents, report,
code and the bullets its lane was shown (new Evaluated::bullets_shown),
plus the reference elites, known pitfalls and the whole playbook. The host
sends it to any model; Memory::apply_reflection merges the answer.

This adapts ACE's Reflector and Curator (prompts from ace-agent/ace) into
one call per round, the budget agreed for this harness: measured scores
replace ground truth, so diagnosis and curation fit one prompt. The deltas
are ACE's: helpful/harmful/neutral tags, counted only for bullets the
reviewed lanes were shown, and ADD operations (the only operation the
reference supports). Merge order is tags, prune, add, so a new bullet is
deduplicated against the survivors; ADDs must cite a reviewed or archived
attempt as evidence. The parser takes the last JSON object with a
reflection key, which survives fences, prose and thinking drafts.

Requests carry an id: an answer applies at most once, a NoJson answer
keeps the request outstanding, and abandon_reflection requeues its
attempts after a failed model call. Memory stays sync and owned by one
task.
… and a novelty check

A memory that every lane shares pulls the population onto one program.
Islands (FunSearch, OpenEvolve) are the classic counter, and the nexus
harness that consumes this crate plans them (agent-symbiont TODO #5).

- Evaluated carries an IslandId; archive entries record the islands that
  hold them and capacity is enforced per island, so a strong island cannot
  evict a weak island's history.
- Playbook bullets get a Scope (Global or one island). With several islands
  reflection adds lessons to the reviewed island; duplicates are checked
  against the global and own-island bullets; capacity is per scope.
- Rendered gains an island part between the shared and lane parts.
- Memory::migrate sends each island's best attempts to the next island in
  a ring (chosen before any moves, so one hop per call) and promotes island
  lessons whose net reached promote_net to the global playbook.
- Memory::reset_island forgets an island's members, lessons and queued
  attempts, for hosts that reset stagnant sub-populations.
- Memory::novelty finds the most similar attempt on an island by token
  trigram Jaccard, ignoring layout and comments: ShinkaEvolve-style
  rejection of near copies before an expensive evaluation, without an
  embedding model.

With the default single island nothing changes but the explicit IslandId
argument of render, reflection_due and reflection_request.
Shows every part of symbiont-memory on a cheap, graded task (symbolic
regression of a hidden 1-D function): four lanes on two islands per round
via evolve_batch, prompts built shared -> island -> lane so a prefix
cache reuses the most, candidates evaluated through RevisionFn handles,
near copies skipped before evaluation, traces and panics counted as
pitfalls, one reflection call per island and round through a second agent
whose preamble is the newly exported REFLECTION_SYSTEM_PROMPT, abandoned
requests on a failed call, and ring migration every other round.

Compiles clean with clippy -D warnings; not yet run end to end (the
inference endpoint was overloaded), so no CI smoke step is added yet.
…quests behind it

The README, the website and the UI said the runtime keeps the chat
history so follow-ups refine the current shader. It does not: evolve
drops its history on return by design, and the prompt carried only the
signature, so 'now make the palette cycle with time' asked for a fresh
shader with no palette to cycle.

The evolution worker now keeps a Session of the requests whose shaders
became active (the last 8; a failed evolution is not recorded). The first
request still prompts with only the signature, so the agent is not
anchored to the default body. Every later request carries the active
revision's code and the earlier requests, and tells the agent to refine,
or to replace the shader if the request asks for something entirely
different. Docs and the UI hint now describe this.
A harness operator could not see what the memory did between prompts.
Every state change now logs a structured tracing event at debug level
(verbose: one or more per lane and round): commits with seq, island,
score and eviction; re-scores; failures that recur often enough to become
pitfalls; token sizes of each rendered section with the entries and
bullets shown; reflection requests; each applied reflection with its
counts, every added lesson, every rejection reason and every pruned
bullet; unusable answers; abandoned requests; migrations and promotions;
island resets; novelty verdicts. RUST_LOG=symbiont_memory=debug turns
them on; the crate docs, README and memory-example say so.

Field values are only evaluated when the level is enabled. A traced_test
asserts the key events; tracing-test moves to [workspace.dependencies]
since two crates use it now. RequestId::get exposes the number for logs.
…mples, current quick start

Brings symbiont.rs up to date with the README:
- Quick start used APIs that no longer exist (Runtime::init(SYMBIONT_DECLS),
  init_agent()); it now matches the README (Runtime::new with prelude and
  profile, init_agent(None), timed evolve) and 'cargo run -p'.
- Development setup block (Nix, .env with API_KEY/BASE_URL/MODEL).
- New showcases: evolving a trading strategy (README's main showcase) and
  memory across rounds (archive, pitfalls, ACE playbook, islands, the
  usage loop, prefix-cache ordering, RUST_LOG=symbiont_memory=debug).
- Highlight cards the README has and the site lacked: typed revision
  handles, agent state between calls, memory, cheap repair rounds, tool
  calling, revision tools, observability. 'LLms' typo fixed.
- Example cards for memory, batched-evolution and fractal-studio, as
  text-only cards (no screenshots exist), placed after the image cards so
  rows pair up.
- Example cards use min-height instead of a fixed height: the tool-calling
  caption was clipped.

The README's 'Tiny Core: ~1000 LOC' claim is left off: runtime.rs alone is
2400 lines. Built with zola and checked in headless chromium.
'~1000 LOC' was long out of date. Counted without comments, blank lines
and #[cfg(test)] modules: the harness and constrained generation part
(runtime, parser, validation, edit, diagnostics, api_hints, compiler,
layout, evolution_agent, revision, state, decl, preambles, utils, error,
config and info types) is ~5.5k lines; the whole symbiont/src is ~12.4k,
adding doc_index/doc_string/system_prompt, tools, inference, dsh export,
evolution_trace and observability. The website gets the card it lacked
because the old number was wrong.
Workspace version and the internal path-dependency version specs
(symbiont -> symbiont-dsh-log, symbiont-macros; symbiont-memory ->
symbiont) move to 0.45.0 for the release that adds symbiont-memory. The
rig-* 0.44 specs are rig's own versions and stay.
…te, honest archive header

For hosts that read the memory config from a file (the nexus runners use
RON): every config struct is #[serde(default)], so a file names only what
it changes. Deserializing skips the checking constructors, so
MemoryConfig::validate re-checks every limit (archive capacity, island
count, zero budgets that would stall reflection or panic on a full
playbook, percentages over 100) and Memory::new panics on an invalid
config instead of misbehaving later.

When the host excludes ids (its own retained members, which it renders
itself and which rank higher), the elite section is titled 'Best attempts
not shown above' instead of the false 'Best attempts so far'.
@MathisWellmann MathisWellmann self-assigned this Oct 19, 2026
@MathisWellmann MathisWellmann changed the title Memory 0.45: new memory crate Oct 19, 2026
@MathisWellmann
MathisWellmann merged commit 47d6966 into main Oct 19, 2026
7 of 11 checks passed
@MathisWellmann
MathisWellmann deleted the memory branch October 19, 2026 20:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant