[Feat] Integrate and evaluate Julia-1

[Yunaik]

Why

Julia-1 (Apache-2.0, checkpoint dated 2026-09-24) is a 144M-parameter decision model fine-tuned from mmBERT-small, a multilingual ModernBERT encoder. It answers choice, score and noul questions and runs on CPU. The authors report 73.15% accuracy on typed decisions and 71.50% on MASSIVE across 52 locales. Neither result has been reproduced here.

At 144M parameters it is smaller than LAYA (0.4B) and multilingual. That makes it a candidate for a local backend on machines without a GPU.

Scope

Add a julia backend that runs in process on a thread, following the laya and cua backends, behind uv sync --extra julia. The request can reuse jev_question from the Jev backend because engine.predict(state=..., questions=...) takes the TypeSafe question shape.

Two limits need explicit handling:

1. Option count. Julia-1 accepts 2 to 20 options per question. Browser pages often offer more elements. As a minimum, a question with more than 20 options raises MODEL_SERVICE_CONFIG_ERROR, and the evaluation reports how often that happens. Pre-filtering or chunking is out of scope until those numbers exist, because chunking breaks the requirement that probabilities cover exactly the offered keys and sum to 1.
2. Context length. Combined state, question and options are capped at 8,192 tokens, and strict_encoding=True rejects overflow. Browser states on the Allrecipes run measure 18,785 to 23,654 tokens (docs/benchmarks.md). [decision_models] Compact the browser front's state for Laya's window #9 compacts the browser state for LAYA in laya_state(). Move that function out of laya.py into shared code and apply it to Julia-1 as well.

Packaging risk: the model installs as a local editable package named julia. PyJulia on PyPI uses the same import name. Pin the checkpoint by Hugging Face revision and confirm the extra resolves without a name clash.

Acceptance criteria

1. Installation and run instructions specify the checkpoint revision, dependencies, device settings, max_length and head_length, and a reproducible smoke test.
2. Behavioural tests with a fake engine cover option mapping, choice, noul and score answers, invalid outputs, more than 20 options, and context overflow.
3. Evaluation on the [Eval] Add repeatable browser and desktop task suites #13 suites reports completion rate, total task time, decision latency and failure reasons next to the models compared in [Feat] Integrate and evaluate Cua-S1 4B 0.2 #14, on the same tasks and budgets, together with the share of decisions refused for option count or context length.
4. Model loading, warm CPU inference and peak memory are measured separately.
5. Results state where Julia-1 is useful and list configurations that failed or remain unsupported.

Dependencies and references

Model card, ONNX export, decision-model interface, CUA backend.

State compaction: #9. Evaluation: #13. Comparison table: #14. Serving-side candidate list: ThinkFlowLab/system1-omni#9.

[GoDiao]

Hi @Yunaik, I’d like to take on #25. Could you assign it to me?

I see that the shared browser-state work needs #9 to merge, and the full comparison needs #13 and #14. In the meantime, I can start on the Julia package and backend, including the fake-engine tests, then finish the remaining parts when those dependencies are ready.

I’ve already opened PR #23 for #3. While we wait, I can work through any review feedback there and make a start on this issue. Thanks!

[Yunaik]

Hi @Yunaik, I’d like to take on #25. Could you assign it to me?

I see that the shared browser-state work needs #9 to merge, and the full comparison needs #13 and #14. In the meantime, I can start on the Julia package and backend, including the fake-engine tests, then finish the remaining parts when those dependencies are ready.

I’ve already opened PR #23 for #3. While we wait, I can work through any review feedback there and make a start on this issue. Thanks!

thanks, yes please do. It's independent from #13, 14. You might need #9 for best performance, yes

Staring with Julia meanwhile is reasonable

PS: #23 is a lot. I'll review it soon

[Yunaik]

you could also take #24 if you want. It's very similar to Julia.

Please also check https://github.com/ThinkFlowLab/system1-omni for performant inference of system 1 models

[GoDiao]

Thanks for clarifying that I can move ahead with #25 before #13 and #14. I'll start with the Julia-1 backend and fake-engine tests, then bring in the shared state compaction once #9 lands. I'll also take a look at system1-omni.

I appreciate the invitation to take #24. I only have a MacBook at the moment, so I'm not confident I could run the 8B setup and complete the evaluation it calls for. I'll focus on #25 for now. Thanks for taking a look at #23 when you can.