fix: reject LLM calls on a context that cannot serve them

[andinux]

Closes #33.

Supersedes #34, which diagnosed the bug correctly but classified contexts by which llm_context_create_* wrapper was called. That left #33 unfixed on the documented generic form and broke working configurations. This takes the same idea — record what the active context is for, reject operations that cannot work on it — and derives the classification from how the context is actually configured.

Three commits, each independently revertable.

ba55a1c — release the cursor when llm_chat() xOpen fails

llm_chat_cursor_open() allocated the ai_cursor before validating the context and returned SQLITE_ERROR without freeing it. SQLite does not call xClose for a cursor whose xOpen failed, so every rejected SELECT ... FROM llm_chat(...) leaked 48 bytes. Pre-existing; reachable today by querying the vtab with no context created.

46e68be — report failures to the caller being served, not to stale sinks

ai->context / ai->vtab are the error sink for the whole chat and sampler subsystem. About twenty sites report through them; only three ever assigned them, so at nearly every report site the sink was whatever the previous statement left behind.

SELECT llm_context_create_chat('context_size=512');
SELECT llm_chat_respond('hi');       -- publishes this statement's context
SELECT llm_context_free();
SELECT llm_chat_restore('x');        -- reports into the finalized statement

That last line called sqlite3_result_error() on freed memory — SIGSEGV. Without the llm_chat_respond() line the sink is NULL and the same call returned NULL with no error at all. After a vtab scan the message went into vtab->zErrMsg, which SQLite no longer imports: discarded, and the buffer leaked.

Every entry point now publishes its own sink first: the chat scalars, the vtab methods (xFilter and xNext matter as much as xOpen — an interleaved scalar statement can repoint the sink mid-scan), and the 16 llm_sampler_* scalars that reach llm_sampler_check(). llm_chat_disconnect() also left ai->vtab dangling at freed memory, which is what made the second case a use-after-free rather than a lost message.

Pre-existing and unrelated to #33, bundled here because the gate is unreportable without it.

22d3086 — reject generation on a context that cannot produce tokens

Pooled embeddings leave no per-token logits, so the first sampled token reads as EOG and generation returns '' with no error. The chat paths are worse: llm_chat_respond() and the llm_chat() vtab reach llama_sampler_sample(), which dereferences that buffer.

A context is an embedding context when either:

• generate_embedding=1 was passed — API.md documents llm_context_create('generate_embedding=1,...') as equivalent to llm_context_create_embedding(), so classifying by call site leaves llm_text_generate() returns '' with no error when the active model/context is an embedding one #33 unfixed on the documented form; or
• the model pools by default. A BERT-family model carries its pooling type in its own GGUF, so llm_context_create('context_size=512,embedding_type=FLOAT32') on all-MiniLM yields a pooling context with no embedding settings passed at all — and that is the flow API.md recommends for embeddings, so llm_text_generate() returns '' with no error when the active model/context is an embedding one #33 reproduced there too.

Resolved pooling alone cannot decide this: forcing pooling_type=mean on a generative model still generates correctly. What identifies an embedding model is pooling the caller did not ask for, which llama exposes by resolving LLAMA_POOLING_TYPE_UNSPECIFIED from the model's own hparams. (llama_model_has_decoder() is no use — it returns true for everything except T5ENCODER, BERT included.)

That made "did the caller ask for pooling" load-bearing, which exposed a parser bug: generate_embedding forced pooling_type = MEAN for any value, 0 included. The side effects now apply only when embeddings are actually enabled.

Embedding generation is deliberately not gated — it needs a context that pools, and llm_embed_generate_run() already checks the resolved pooling type, which is a better test than the declared kind.

Known limitation, documented in API.md: passing pooling_type explicitly on an embedding model opts out of the second test and lets generation return '' again. That is the cost of not misreading a caller-forced pooling type on a generative model, which is the more common case.

Compatibility

The gate fires only where generation was already broken — silent '' or a crash — so no working configuration starts failing. Verified across both model families: generic-constructor embedding contexts, caller-forced pooling, chat contexts used with llm_text_generate() (the README vision example), and generate_embedding=0 on a generative model all keep working.

Tests

56 passing, up from 51. Each new test was confirmed to fail against the pre-fix code before being accepted, including a 96-byte leak assertion for the cursor and a SIGSEGV reproduction for the error sink.

Covering the embedding-model half needs an encoder-style model, so the suite now pulls all-MiniLM-L6-v2 (25MB) alongside the existing generative model. That test skips cleanly without --embed-model, so existing checkouts are unaffected.

Follow-ups, not addressed here

• Zero-token generation still returns '' rather than an error for anything the classifier does not catch, including the pooling_type opt-out above. This is the general form of llm_text_generate() returns '' with no error when the active model/context is an embedding one #33 and has now surfaced twice in different guises.
• llm_chat_create() installs an unseeded dist sampler on ai->sampler that outlives the context, silently making later llm_text_generate() calls non-deterministic.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Ks738YbgnnJBio2wLtpbAJ