From d909601f4160d771eda77180cbd89b62dc5c30a3 Mon Sep 17 00:00:00 2001 From: VickyXAI <115643921+VickyXAI@users.noreply.github.com> Date: Sun, 20 Sep 2026 21:04:55 +0900 Subject: [PATCH 1/3] =?UTF-8?q?feat(skills):=20`decide`=20=E2=80=94=20try?= =?UTF-8?q?=20the=20free=20typed-judgment=20endpoint=20from=20Claude=20Cod?= =?UTF-8?q?e,=20without=20an=20MCP=20tool?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit POST api.blockrun.ai/v1/decide (OpenJev) is free behind a registered key and not x402. It is deliberately NOT a blockrun_decide tool: every MCP caller is already a frontier model and a better one-off judge than a 4B NLI cross-encoder, the default wallet install has no brk_ key, and a tool taxes every user's context on every turn for an experiment few run. The skill costs nothing until triggered and carries the whole recipe: curl, the three question types and their limits, the response shape, and the eval loop over the user's own labelled rows. Two things every surface has to say, and this one does: the `confidence` on a choice is a share of agreement, not a probability (equal-weak and equal-strong support are indistinguishable after normalisation), and OpenJev is not Jev — unaffiliated, MIT, no published comparison, none invented. Also tells the agent to export the key in the shell rather than write ~/.blockrun/.api-key, because that file flips every paid tool to account billing at the next start. Co-Authored-By: Claude Opus 5 (1M context) --- .claude-plugin/marketplace.json | 5 + skills/decide/SKILL.md | 208 ++++++++++++++++++++++++++++++++ 2 files changed, 213 insertions(+) create mode 100644 skills/decide/SKILL.md diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index a943ab5..add4857 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -84,6 +84,11 @@ "name": "gentech-blockrun", "source": "./skills/gentech-blockrun", "description": "GenTech Labs' integration patterns for BlockRun from Hermes Agent — daily usage patterns, cost-optimized workflows, multi-tool pipelines and error handling across the full toolset." + }, + { + "name": "decide", + "source": "./skills/decide", + "description": "Use when the user wants to try BlockRun's free typed-judgment endpoint (POST api.blockrun.ai/v1/decide, served by OpenJev) on their own data — yes/no, labelled choice, or scored-rung questions over a text or JSON state. Not an MCP tool; called with curl. Explains what the confidence number means and why OpenJev is not Jev." } ] } diff --git a/skills/decide/SKILL.md b/skills/decide/SKILL.md new file mode 100644 index 0000000..8e08dc0 --- /dev/null +++ b/skills/decide/SKILL.md @@ -0,0 +1,208 @@ +--- +name: decide +description: "Use when the user wants to try BlockRun's free typed-judgment endpoint (POST api.blockrun.ai/v1/decide, served by OpenJev) on their own data from Claude Code — yes/no, labelled choice, or scored-rung questions over a text or JSON state, up to 64 per call. Not an MCP tool: call it with curl from the shell. Covers the request shape, what the confidence number does and does not mean, and why OpenJev is not Jev." +triggers: + - "decide" + - "v1/decide" + - "typed judgment" + - "typed judgments" + - "openjev" + - "open jev" + - "jev" + - "nli" + - "natural language inference" + - "entailment" + - "classify with a small model" + - "cheap classifier" + - "judgment model" + - "score these" + - "label these" + - "triage tickets" +--- + +# Decide — free typed judgments from Claude Code + +`POST https://api.blockrun.ai/v1/decide` takes a **state** (a message, a ticket, a +diff, a tool result, a row of data) and up to **64 questions**, and returns a typed +answer with a number beside each one. No prose to parse. The backend is +**OpenJev**, an open-source natural-language-inference model BlockRun hosts. + +It is **not a BlockRun MCP tool**, on purpose: you are already a frontier model, +and for a one-off "is this urgent?" you are the better judge. The endpoint earns +its place when the user wants the *same fixed ruler over many items*, or is +prototyping a judgment they will later run from a pipeline **without** a model. +From Claude Code, call it with `curl` from the shell. + +## What it is not + +- **Not paid, not x402.** It is free behind a registered API key. There is no + payment header, no wallet path, and it is served by `api.blockrun.ai`, not the + x402 gateway at `blockrun.ai` (a POST to `blockrun.ai/v1/decide` is a 404). +- **OpenJev is not Jev.** Jev is TypeSafe's model. OpenJev is an unaffiliated + open-source NLI cross-encoder (MIT, published by AlexWortega on Hugging Face, + Qwen3.5 4B base). It is not made by the people who make Jev and it is not a + smaller or free tier of it. BlockRun does not sell or resell Jev. There is one + backend, and the `x-blockrun-backend` response header names it on every call. +- **No quality comparison exists.** BlockRun expects OpenJev to be materially + weaker than Jev, has not benchmarked the two, and will not put a number on the + gap. Do not invent one. The model authors' own zero-shot NLI figures, with + attribution and a read date, are at . + +## Getting a key without changing how the MCP pays + +Keys are minted at (`brk_live_…`, +shown once; registration, not a card). + +For experiments, **export the key in the shell** and leave the MCP server alone: + +```bash +export BLOCKRUN_API_KEY=brk_live_… +``` + +Do **not** write it to `~/.blockrun/.api-key` just to try `decide`. The MCP +server reads that file at startup and a present key moves **every** paid tool +from wallet mode to account billing — the same switch `BLOCKRUN_API_KEY` in the +MCP server's own config makes. That is fine if the user wants account billing +(see the `blockrun-setup` skill); it is a surprise if they only wanted a free +judgment. If the MCP is already on account billing, the same key works for both. + +## Request + +```bash +curl -sS -X POST https://api.blockrun.ai/v1/decide \ + -H "authorization: Bearer $BLOCKRUN_API_KEY" \ + -H "content-type: application/json" \ + -D /dev/stderr \ + -d '{ + "state": "Help! My payouts have been failing for 3 days.", + "questions": { + "is_urgent": { "type": "noul", "instructions": "Does this convey urgency?" }, + "department": { "type": "choice", "instructions": "Which team should handle this?", + "criteria": { "billing": "Payments, refunds", + "technical": "Bugs, outages" } }, + "frustration": { "type": "score", "instructions": "How frustrated is the customer?", + "criteria": ["Calm", "Frustrated", "Very angry"] } + } + }' +``` + +`-D /dev/stderr` shows the response headers (backend, rate-limit) without mixing +them into the JSON on stdout. + +| Field | Type | Required | Notes | +|---|---|---|---| +| `state` | string \| object \| array | yes | What to judge. Text and JSON both work. If a fact matters, put it in the state — the model looks nothing up. | +| `questions` | object | yes | Your own id → question. **1 to 64** per call. | +| `model` | string | no | Defaults to `openjev`, the only backend. Leave it out. | + +Three question types, one operation underneath (state = premise, question = +hypothesis, answer = how strongly the premise entails it): + +| Type | `criteria` | You get back | +|---|---|---| +| `noul` | none — it is true or false | one number for how strongly the state supports the claim | +| `choice` | map of label → its meaning, **2 to 255** entries | the top label, plus a share per option | +| `score` | array of rung descriptions in words, **2 to 255** | a weighted position across the rungs, plus the distribution | + +`instructions` — the question itself, in plain language — is required on all three. + +## Response + +```json +{ + "model": "openjev", + "answers": { + "is_urgent": { "type": "noul", "noul": 0.986 }, + "department": { "type": "choice", "choice": "billing", + "probabilities": { "billing": 0.71, "technical": 0.29 }, + "confidence": 0.71 }, + "frustration": { "type": "score", "score": 1.42, + "legend": { "0": "Calm", "1": "Frustrated", "2": "Very angry" }, + "probabilities": { "0": 0.11, "1": 0.36, "2": 0.53 }, + "confidence": 0.53 } + } +} +``` + +`score` is the probability-weighted rung index (`1.42` sits between +"Frustrated" and "Very angry"); `legend` maps index → rung text. + +## Read this before you build a threshold + +**The `confidence` on a `choice` is not the probability the answer is correct.** + +Every option is scored against the state, and the scores are then **divided by +their total** so they sum to one. That step throws away how strong the scores +were: + +- three options each scoring 0.1 — the model supporting **none** of them — come + back as `0.33 / 0.33 / 0.33`; +- two options scoring 0.9 and 0.85 — **both** strongly supported — come back as + `0.51 / 0.49`. + +Opposite situations, identical response. The number is a **share of the +agreement found** — an ordering with a margin. Wide means the model clearly +preferred one option; narrow means it did not. It says nothing about whether +*any* option fits, and nothing has been fitted against real outcomes the way a +calibrated model's probabilities are. The same applies to `score`'s +`probabilities` and `confidence`. A `noul` is the raw entailment score and is +not normalised — but it is not calibrated either. + +Set thresholds by running the user's **own labelled examples** through the +endpoint and looking at where the errors land, never by reading the number as a +percentage. An agent that gates on `confidence > 0.8` as if it were accuracy +will be wrong with no error to tell it so. + +## Trying it on the user's data — the shape that works + +1. **Build the state** with everything the judgment needs: the message, the + records it refers to, the policy that governs it. Fetch with `blockrun_exa` + or `blockrun_search` first if a fact is missing; `decide` cannot look it up. +2. **Ask the independent questions together** in one call — one round trip, + one rate-limit hit, up to 64 answers. +3. **Combine with deterministic checks** the code already has (amount, account + age, "was a refund already issued"). +4. **Route by margin.** Act automatically where the margin is wide *and* the + checks agree; hand narrow cases to a reasoning model — `blockrun_chat` with + a `claude-*`, `o-`series or DeepSeek model — or to a person. + +A quick evaluation loop from Claude Code: put 20–50 labelled rows in a JSON +file, loop `curl` over them with `jq`, and tabulate agreement per question. +That is the only number that should decide whether the endpoint is good enough +for the user's task. + +```bash +jq -c '.[]' examples.json | while read -r row; do + state=$(jq -c '.state' <<<"$row"); want=$(jq -r '.label' <<<"$row") + got=$(curl -sS -X POST https://api.blockrun.ai/v1/decide \ + -H "authorization: Bearer $BLOCKRUN_API_KEY" -H "content-type: application/json" \ + -d "{\"state\": $state, \"questions\": {\"q\": {\"type\": \"choice\", + \"instructions\": \"Which team should handle this?\", + \"criteria\": {\"billing\": \"Payments, refunds\", \"technical\": \"Bugs, outages\"}}}}" \ + | jq -r '.answers.q.choice') + echo "$want,$got" +done | sort | uniq -c +``` + +## What it does not do + +It does not write, summarise, or explain — there is no free-text field in the +response. It has no memory between calls. Image input exists in the model's v2 +checkpoint but the endpoint does not expose it. + +## Limits and errors + +| Status | Meaning | What to do | +|---|---|---| +| 400 | Body did not match the schema; the response names the field | Fix the field. Common: a `choice` with `criteria` as an array, a `score` with a map, fewer than 2 criteria, more than 64 questions. | +| 401 | Missing or invalid key | `BLOCKRUN_API_KEY` is unset in this shell, or wrong. Check . | +| 429 | Per-key hourly limit reached | Wait for the interval the response carries (`retry-after`). The limit is enforced server-side and reported in the response — read it there; do not assume a number. | + +None of these cost anything; nothing here ever does. + +## Related + +- API reference: +- OpenJev, with the authors' published figures: +- Account billing for the whole MCP (if the user wants the key to pay for the + paid tools too): `blockrun-setup` skill, "API key" section. From cd5a0009290e9b5c241f9aa607d46c7e0256b930 Mon Sep 17 00:00:00 2001 From: VickyXAI <115643921+VickyXAI@users.noreply.github.com> Date: Sun, 20 Sep 2026 21:04:56 +0900 Subject: [PATCH 2/3] docs: seventeen skills, and the router skill points a fixed-ruler question at `decide` Co-Authored-By: Claude Opus 5 (1M context) --- CONTRIBUTING.md | 2 +- README.md | 2 +- skills/blockrun-setup/SKILL.md | 2 +- skills/blockrun/SKILL.md | 3 ++- 4 files changed, 5 insertions(+), 4 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 16936fd..8a5289f 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,6 +1,6 @@ # Contributing to BlockRun MCP -PRs welcome. This server is small (~4,400 lines of tool code + 16 skills); reviews are usually fast. +PRs welcome. This server is small (~4,400 lines of tool code + 17 skills); reviews are usually fast. ## Setup diff --git a/README.md b/README.md index a11faa5..468d93f 100644 --- a/README.md +++ b/README.md @@ -334,7 +334,7 @@ Claude reads the odds with `blockrun_markets`. In wallet mode, and only after yo ### 5. Install the agent skills (optional) -The package ships 16 skills — which tool answers what, worked examples, and a setup / debug / upgrade trio so the agent can install, troubleshoot and update the server on its own. +The package ships 17 skills — which tool answers what, worked examples, and a setup / debug / upgrade trio so the agent can install, troubleshoot and update the server on its own. ```bash /plugin marketplace add BlockRunAI/blockrun-mcp # Claude Code diff --git a/skills/blockrun-setup/SKILL.md b/skills/blockrun-setup/SKILL.md index d17a839..4d70558 100644 --- a/skills/blockrun-setup/SKILL.md +++ b/skills/blockrun-setup/SKILL.md @@ -146,7 +146,7 @@ keypair and are unavailable; everything else works. ## 5. Optional: install the agent skills -The package ships 16 skills (which tool to use, worked examples, this one). Claude Code +The package ships 17 skills (which tool to use, worked examples, this one). Claude Code users: `/plugin marketplace add BlockRunAI/blockrun-mcp`. Everyone else: ```bash diff --git a/skills/blockrun/SKILL.md b/skills/blockrun/SKILL.md index 17c06a6..3d71a31 100644 --- a/skills/blockrun/SKILL.md +++ b/skills/blockrun/SKILL.md @@ -116,6 +116,7 @@ Deeper wallet, budget and x402 mechanics — including calling the HTTP API dire | Run code in a remote container / on a GPU | `blockrun_modal` | Prefer local for normal repo work | | Which models exist, and what they cost | `blockrun_models` | | | Balance, funding, spend caps | `blockrun_wallet` | | +| A fixed yes/no, choice or score over many items, free | *(no tool)* — `curl api.blockrun.ai/v1/decide` | Needs a free key; see the `decide` skill. You are the better one-off judge | The crypto tools overlap heavily. Prefer the free ones (`blockrun_price`, `blockrun_dex`) when they already answer the question, and reach for `blockrun_defi` or `blockrun_markets` only when @@ -177,7 +178,7 @@ authoritative** — this list would go stale within a release. Install from the same marketplace for depth on one area: `search`, `exa-research`, `crypto-data`, `surf`, `rpc`, `prediction-markets`, `polymarket-trading`, `image-prompting`, -`phone`, `modal`. +`phone`, `modal`, `decide`. ```bash /plugin marketplace add BlockRunAI/blockrun-mcp From 8a64a781afb0ef5e491df880befea03f6b711789 Mon Sep 17 00:00:00 2001 From: VickyXAI <115643921+VickyXAI@users.noreply.github.com> Date: Sun, 20 Sep 2026 21:08:53 +0900 Subject: [PATCH 3/3] =?UTF-8?q?chore(release):=200.52.0=20=E2=80=94=20a=20?= =?UTF-8?q?judgment=20endpoint=20as=20a=20skill,=20not=20a=20tool?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 5 (1M context) --- CHANGELOG.md | 31 +++++++++++++++++++++++++++++++ VERSION | 2 +- package-lock.json | 4 ++-- package.json | 2 +- 4 files changed, 35 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fa23a7d..c8ad14a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,37 @@ All notable changes to BlockRun MCP will be documented in this file. +## 0.52.0 + +**A judgment endpoint as a skill, not a tool.** BlockRun now serves +`POST api.blockrun.ai/v1/decide` — typed yes/no, labelled-choice and +scored-rung judgments over a text or JSON state, served by OpenJev, free +behind a registered key. The obvious move was a `blockrun_decide` tool. It was +not made: every caller of this server is already a frontier model and a better +one-off judge than a 4B NLI cross-encoder, the default install is a wallet with +no key, and a tool taxes every user's context on every turn for an experiment +few run. The endpoint earns its place when someone wants one fixed ruler over +many items, or is prototyping a judgment they will later run from a pipeline +without a model — and that is a `curl` recipe, which is what a skill is for. + +### Skills + +- **`feat(skills)` — `decide`.** The seventeenth skill: the request shape, the + three question types and their limits (1–64 questions, 2–255 criteria), the + response, an eval loop over the user's own labelled rows, and the errors. Two + things it says because every surface has to: the `confidence` on a choice is + a share of the agreement found, not the probability the answer is correct + (equal-weak and equal-strong support are indistinguishable once the scores + are normalised), and OpenJev is not Jev — unaffiliated, MIT, no published + comparison and none invented. It also tells the agent to export the key in + the shell rather than write `~/.blockrun/.api-key`, because that file moves + every paid tool to account billing at the next start. +- **`docs` — the router skill points a fixed-ruler question at `decide`**, and + the README, CONTRIBUTING and setup skill count seventeen. + +Nothing in `src/` changed: the tool count, the schema cost and every price +table are as they were in 0.51.1. + ## 0.51.1 **Round four, second half.** 0.51.0 shipped with seven of round four's ten diff --git a/VERSION b/VERSION index 1e0c609..4f9b378 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.51.1 +0.52.0 diff --git a/package-lock.json b/package-lock.json index 30fbad8..4e88cd8 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "@blockrun/mcp", - "version": "0.51.1", + "version": "0.52.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@blockrun/mcp", - "version": "0.51.1", + "version": "0.52.0", "license": "MIT", "dependencies": { "@anthropic-ai/sdk": "^0.123.0", diff --git a/package.json b/package.json index 118e335..a01e8d6 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@blockrun/mcp", - "version": "0.51.1", + "version": "0.52.0", "mcpName": "io.github.BlockRunAI/blockrun-mcp", "description": "BlockRun MCP Server - Give your AI agent web search, deep research, prediction markets, and crypto data. Pay per call from a USDC wallet (Solana or Base) or a BlockRun API key.", "type": "module",