feat(decision): d3 System One contract library and its L0 test

.github/workflows/ubuntu-build.yml

@@ -12,6 +12,19 @@ on:
   workflow_dispatch:
 
 jobs:
+  # The d3 System One contract (src/common/decision) against fixtures written by the upstream
+  # package: prompt bytes, request/question errors and answer objects. CPU only, no NPU.
+  d3-contract:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+      - name: Build and run the d3 contract test
+        run: |
+          cmake -S src/test/d3_contract -B build-d3-contract -DCMAKE_BUILD_TYPE=Release
+          cmake --build build-d3-contract -j"$(nproc)"
+          ctest --test-dir build-d3-contract --output-on-failure
+
   package-deb:
     runs-on: ubuntu-latest
     strategy:

src/include/decision/d3_contract.hpp

@@ -0,0 +1,117 @@
+/// \file d3_contract.hpp
+/// \brief The d3 (Decision 3.0) System One contract: request validation, the prompt the model reads,
+///        the readout, and the answer objects. A byte-exact port of the upstream package's
+///        d3_format.py, d3_runtime.py (normalize_question, prepare, product_answer, respond) and
+///        d3_server.py (request checks) at vllm-sr/d3-lite v3.1.3.
+/// \note  No NPU code here: the engine returns one hidden row per question and everything else is
+///        this file. src/test/d3_contract checks it against the frozen goldens and fuzz requests.
+#pragma once
+
+#include <functional>
+#include <map>
+#include <stdexcept>
+#include <string>
+#include <vector>
+
+#include <nlohmann/json.hpp>
+
+#include "decision/py_json.hpp"
+
+namespace flm::d3 {
+
+using ojson = nlohmann::ordered_json;
+
+constexpr int MAX_OPTIONS = 255;
+constexpr const char* FORMAT_ID = "d3-code-readout-v1";
+extern const char* const SYSTEM_PROMPT;
+
+/// A problem with one question (`invalid_question`) or with a model output (`invalid_model_output`);
+/// the message is upstream's ValueError text.
+struct ContractError : std::runtime_error {
+    using std::runtime_error::runtime_error;
+};
+
+/// A request the server answers with 422 `{"detail": detail}` before running anything.
+struct RequestError : std::runtime_error {
+    ojson detail;
+    explicit RequestError(ojson d) : std::runtime_error(d.is_string() ? d.get<std::string>() : d.dump()), detail(std::move(d)) {}
+};
+
+/// One validated question (d3_runtime.Question).
+struct Question {
+    std::string kind;                       //!< choice | noul | score
+    std::vector<std::string> keys;          //!< answer keys, in option order ("0".."n-1" for score)
+    std::vector<std::string> option_texts;  //!< the text after each code in the prompt
+    std::vector<PyValue> levels;            //!< score: the original levels (legend)
+    PyValue instructions;                   //!< as given (null when absent)
+};
+
+/// `describe`: strings verbatim, anything else as `json.dumps(v, ensure_ascii=False)`.
+std::string describe(const PyValue& v);
+
+/// `normalize_question`; throws ContractError with upstream's message.
+Question normalize_question(const PyValue& question);
+
+/// The 255 candidate codes in preference order (A..Z, AA..ZZ).
+std::vector<std::string> candidate_codes();
+
+/// `d3_format.user_prompt` (state may be null).
+std::string user_prompt(const PyValue& state, const Question& q, const std::vector<std::string>& codes);
+
+/// The chat text of `[system, user]` (or `[user]` when system is empty) with
+/// `add_generation_prompt=True, enable_thinking=False`, as the package's chat_template.jinja renders
+/// it. AutoDecisionModel cross-checks this against minja at load.
+std::string chat_text(const std::string& system, const std::string& user);
+
+/// The whole prompt of one question (`d3_format.render`).
+std::string render(const PyValue& state, const Question& q, const std::vector<std::string>& codes);
+
+/// `d3_format.answer_codes`: the codes and token ids this tokenizer gives; throws ContractError if
+/// fewer than 255 qualify.
+void answer_codes(const std::function<std::vector<int>(const std::string&)>& encode,
+                  std::vector<std::string>& codes, std::vector<int>& token_ids);
+
+/// Masked code logits -> option probabilities, as the runtime computes them: fp32 hidden @ fp32
+/// readout^T, codes >= count masked, / temperature, fp32 softmax, cut to count.
+std::vector<float> readout_probabilities(const float* hidden, const float* readout, int hidden_size,
+                                         int count, float temperature);
+
+/// `product_answer`: the public answer object; throws ContractError (invalid_model_output) when the
+/// probabilities are unusable.
+ojson product_answer(const Question& q, const std::vector<double>& probabilities);
+
+/// `{"type": <type as given>, "error": error, "message": message}`.
+ojson error_answer(const ojson& type, const std::string& error, const std::string& message);
+
+/// A PyValue as nlohmann JSON (for echoing request values in replies).
+ojson to_ojson(const PyValue& v);
+
+/// A System One request after validation and rendering (d3_runtime.Prepared, text path).
+struct Prepared {
+    std::vector<std::string> keys;              //!< every question ID, request order
+    std::map<std::string, Question> questions;  //!< valid questions
+    std::map<std::string, std::string> texts;   //!< rendered prompts of valid questions
+    std::map<std::string, std::vector<int>> sequences;  //!< token ids of runnable questions
+    std::map<std::string, ojson> errors;        //!< per-question error objects
+    bool has_images = false;
+    bool has_videos = false;
+
+    std::vector<std::string> runnable() const;
+};
+
+/// Server-level checks of a decoded body (d3_server.Decider.arguments + D3.prepare's request
+/// checks) and per-question validation/rendering. Throws RequestError for what upstream answers 422.
+/// Tokenization and the length limit are applied by `tokenize()`.
+Prepared prepare(const PyValue& body, const std::vector<std::string>& codes);
+
+/// Tokenize every rendered prompt; prompts over `max_length` (> 0) become max_length_exceeded.
+void tokenize(Prepared& p, const std::function<std::vector<int>(const std::string&)>& encode, int max_length);
+
+/// `invalid_questions` of a prepared request, or an empty object.
+ojson invalid_questions(const Prepared& p);
+
+/// `D3.respond`: `{"model", "answers", "usage"}` from per-question probabilities.
+ojson respond(const Prepared& p, const std::map<std::string, std::vector<double>>& probabilities,
+              const std::string& model_name);
+
+}  // namespace flm::d3

src/include/decision/py_json.hpp

@@ -0,0 +1,77 @@
+/// \file py_json.hpp
+/// \brief JSON values with Python's json module semantics, for the d3 prompt contract.
+/// \note  The d3 prompt shows non-string states, instructions and descriptions as Python's
+///        `json.dumps(v, ensure_ascii=False)` text, so the bytes the model reads depend on details
+///        nlohmann::json does not keep: integers of any size, the shortest-repr float format
+///        (`1e-05`, `1e+16`, `100000.0`), `", "` / `": "` separators, and the key order and
+///        duplicate-key rule of `json.loads` (first position, last value). PyValue keeps exactly
+///        what `json.loads` would build, and the writers below reproduce `json.dumps` and `repr`.
+#pragma once
+
+#include <cstdint>
+#include <stdexcept>
+#include <string>
+#include <utility>
+#include <vector>
+
+namespace flm::d3 {
+
+class PyValue {
+public:
+    enum class Kind { Null, Bool, Int, Float, String, Array, Object };
+
+    Kind kind = Kind::Null;
+    bool b = false;
+    std::string text;   //!< Int: canonical decimal digits with an optional '-'; String: UTF-8
+    double f = 0.0;
+    std::vector<PyValue> items;                          //!< Array
+    std::vector<std::pair<std::string, PyValue>> members; //!< Object, in json.loads key order
+
+    static PyValue null() { return PyValue(); }
+    static PyValue boolean(bool v) { PyValue p; p.kind = Kind::Bool; p.b = v; return p; }
+    static PyValue integer(std::string digits);
+    static PyValue number(double v) { PyValue p; p.kind = Kind::Float; p.f = v; return p; }
+    static PyValue string(std::string v) { PyValue p; p.kind = Kind::String; p.text = std::move(v); return p; }
+    static PyValue array() { PyValue p; p.kind = Kind::Array; return p; }
+    static PyValue object() { PyValue p; p.kind = Kind::Object; return p; }
+
+    bool is_null() const { return kind == Kind::Null; }
+    bool is_string() const { return kind == Kind::String; }
+    bool is_object() const { return kind == Kind::Object; }
+    bool is_array() const { return kind == Kind::Array; }
+
+    /// Object member lookup (nullptr when absent); `dict.get(key)`.
+    const PyValue* get(const std::string& key) const;
+    /// Insert or overwrite, keeping the first position (`d[key] = value`).
+    void set(const std::string& key, PyValue value);
+
+    /// Python truthiness: None, False, 0, 0.0, "", [] and {} are false.
+    bool truthy() const;
+};
+
+struct JsonError : std::runtime_error {
+    using std::runtime_error::runtime_error;
+};
+
+/// `json.loads(text, parse_constant=<reject>)`: strict JSON (no raw control characters in strings),
+/// NaN / Infinity literals rejected, numbers with a fraction or exponent become floats (and may
+/// overflow to +-inf, as in Python), integers keep every digit. Throws JsonError on bad input,
+/// including invalid UTF-8 and lone surrogate escapes (which Python would carry along and then
+/// fail to encode).
+PyValue parse_json(const std::string& text);
+
+/// `json.dumps(v, ensure_ascii=False)` with the default `", "` / `": "` separators, or compact
+/// `(",", ":")` separators and sorted keys (score legends). Throws JsonError for a non-finite float
+/// (`allow_nan=False`) with Python's message.
+std::string dumps(const PyValue& v, bool compact = false, bool sort_keys = false);
+
+/// `repr(v)` of the Python object `json.loads` builds (used in error messages: `'rank'`, `None`).
+std::string repr(const PyValue& v);
+
+/// `repr(float)`: the shortest round-trip digits in Python's layout.
+std::string float_repr(double v);
+
+/// `_json_value` of d3_runtime: raises (JsonError) when `json.dumps(..., allow_nan=False)` would.
+void check_json_value(const PyValue& v);
+
+}  // namespace flm::d3

src/test/d3_contract/CMakeLists.txt

@@ -0,0 +1,29 @@
+# d3 contract tests: no NPU, no XRT, so this builds and runs on Linux CI as well as Windows.
+#   cmake -S src/test/d3_contract -B build-d3c && cmake --build build-d3c && ctest --test-dir build-d3c
+# -DFLM_D3_TEST_TOKENIZER=ON also links tokenizers-cpp (needs cargo) so a local run with the model package
+# checks token ids and the answer codes:  test_d3_contract <fixtures> <D3-Lite-NPU2 dir>
+cmake_minimum_required(VERSION 3.22)
+project(d3_contract_tests LANGUAGES CXX)
+
+set(CMAKE_CXX_STANDARD 20)
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+option(FLM_D3_TEST_TOKENIZER "Check token ids with tokenizers-cpp" OFF)
+
+set(FLM_SOURCE_DIR "${CMAKE_CURRENT_LIST_DIR}/../..")
+
+add_executable(test_d3_contract
+    test_d3_contract.cpp
+    "${FLM_SOURCE_DIR}/common/decision/py_json.cpp"
+    "${FLM_SOURCE_DIR}/common/decision/d3_contract.cpp")
+target_include_directories(test_d3_contract PRIVATE "${FLM_SOURCE_DIR}/include")
+
+if(FLM_D3_TEST_TOKENIZER)
+    add_subdirectory("${FLM_SOURCE_DIR}/../third_party/tokenizers-cpp" tokenizers EXCLUDE_FROM_ALL)
+    target_link_libraries(test_d3_contract PRIVATE tokenizers_cpp)
+    target_include_directories(test_d3_contract PRIVATE "${FLM_SOURCE_DIR}/../third_party/tokenizers-cpp/include")
+    target_compile_definitions(test_d3_contract PRIVATE FLM_D3_TEST_TOKENIZER)
+endif()
+
+enable_testing()
+add_test(NAME test_d3_contract
+         COMMAND test_d3_contract "${CMAKE_CURRENT_LIST_DIR}/fixtures")

src/test/d3_contract/README.md

@@ -0,0 +1,34 @@
+# d3 contract test
+
+`test_d3_contract` checks FastFlowLM's port of the d3 (Decision 3.0) System One contract
+(`src/common/decision`) against the upstream package it reproduces, `vllm-sr/d3-lite` at `v3.1.3`
+(`d3_format.py`, `d3_runtime.py`, `d3_server.py`). No NPU and no model weights are involved.
+
+For every fixture request it requires:
+
+- the rendered prompt of every question, byte for byte;
+- the same request-level refusal (the 422 `detail` text d3_server sends) or the same `invalid_questions`;
+- from the same fp32 option probabilities, the same answer objects: keys, key order, types, and every number
+  bit-identical (Python 3.12's compensated `sum()` included).
+
+With `-DFLM_D3_TEST_TOKENIZER=ON` and the model package as the second argument, it also checks every token id and
+that the tokenizer yields the package's 255 answer codes.
+
+```bash
+cmake -S src/test/d3_contract -B build-d3-contract && cmake --build build-d3-contract
+ctest --test-dir build-d3-contract --output-on-failure
+```
+
+## Fixtures
+
+`fixtures/` is generated by `gen_fixtures.py` with the upstream package's own Python runtime:
+
+- `goldens.jsonl`: 308 hand-built golden requests (all question types, 1-255 options, the 26/27 two-letter code
+  boundary, JSON states and descriptions, unicode and control characters, special-token text, invalid questions)
+  with their prompts, token ids, BF16 probabilities and in-process responses;
+- `fuzz.jsonl`: 1,000 fixed-seed random request *bodies* (raw JSON text, so number spellings and duplicate keys
+  reach the parser): every JSON value type, floats around Python's repr switch points, integers beyond 64 bits,
+  falsy values, invalid shapes, unknown fields, malformed JSON -- with what upstream answers for each;
+- `decision_config.json`: the package's answer codes.
+
+Regenerating them is a reviewed change: the fixtures are the definition of "identical to upstream".