feat(judge): 以 Claim 审查指纹结果(合约下沉 utils/jev),并恢复 Go 1.17 支持 - #35
Merged
Merged
Conversation
Jev judges the rule result instead of being another engine: rules and code recall candidates, Jev only judges evidence, code makes the final call. - jev.Page is the page entry, jev.Round is one request of questions with callbacks; Verify / Classify / Version are the capabilities - results annotate common.Frameworks in place with jev:* tags; a failed round leaves frames untouched - Client cache keyed by exact questions plus a page simhash, so similar pages can share answers - Engine.AttachJev / Engine.Refine; WebMatch and DetectContent unchanged - cmd/jevbench compares rules vs rules+Jev; cmd/engine gets --jev Also fix MergeFrameworks sharing alias Attributes across pages: a version written on one page leaked into every later hit of the same alias. Refs #34 Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
1 task
Jev is one provider of a generic judgement and rerank layer, not the layer itself. - package judge: Provider interface (ID + Judge), Judge (cache, request merging, thresholds, optional provider Calibration), neutral question types Binary / Choice / Score - package judge/jev: the TypeSafe Jev provider, protocol and transport only - marks are one enum (Rejected, Duplicate, Primary, Recalled) read with judge.Is / LayerOf / Accepted, stored as judge:* tags; page Kind is an enum - cache is on by default and per question: any grouping of questions hits, partial hits send only the rest, concurrent rounds on similar pages merge - similar pages must share a title and differ by <= 1 signature bit; text is shingled by two runes. On 1070 real pages every answer reused at distance <= 1 matched (159/159), 96.9% at the old default of 3, and unrelated sparse CJK pages collided without the title guard - Verify skips judged frames, so Refine on the same frames is free - version subject without a primary: the only application, else server, else device (fixes directory listings next to an OS hit) - Engine.AttachJudge; cmd/judgeeval -provider; cmd/engine --judge jev - docs: judge/README.md Refs #34 Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…rator Snapshot of the working tree before the judge layer cleanup: explicit Judge methods (Refine/Inspect/Classify/Version/...), replayable judgeeval (-manifest/-history/-maintain/-offline), the fingerprint generator and the validation reports. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The replace pinned github.com/chainreactors/go-re2 with a pseudo-version whose timestamp did not match the commit, so go.sum carried another commit and builds without -tags goregexp failed with a missing go.sum entry. Pin commit 8121b6cd261e with its real pseudo-version. Require utils/parsers at the commit adding Framework.Judge (chainreactors/utils#11), to be re-pinned to parsers/v0.0.4 once tagged. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…n judge/gen - Verdicts (layer, confidence, rejected, duplicate, primary, recalled) are written to parsers.Framework.Judge instead of judge:* tags; callers use Frameworks.Accepted/Primary without importing judge. - Public API: Refine (report), Inspect (explain), Classify (page), Version, IsUnknownProduct, SuggestNames, Yes/Choose/Score. Judge.Verify, the knownNames variadics, the Round layer and Page.Kind/Generic are gone. - Judge.Known recalls fingerprint names (Engine.Names) automatically; SimilarDistance is a Judge field instead of a mutable global. - Refine resolves the version of every kept product. Versions a response binds by name (Server: nginx/1.24.0, X-Jenkins: 2.401.3, generator meta) are taken without asking; the provider picks among candidates ranked by closeness to the product name, versions bound to another product are not offered, and a string occurring once goes to one product only. - The fingerprint generator moves to judge/gen, so package judge depends only on common; shared text primitives live in judge/internal/evidence. - judgeeval replay drops the separate per-product Version stage. - Code in judge, gen, jev and judgeeval is Go 1.17 compatible. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
- fingerprinthub: baseline and case tests use the passive template loader and matcher (buildInternalEvent/loadTemplates were removed). - nmap: NoGuess moved to parsers. - fingers bench: constant format string for vet. go vet ./... and go test -short ./... pass with and without -tags goregexp. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
go.mod had drifted to go 1.24.0 with a toolchain line, which Go 1.17-1.20
cannot build ("invalid go version", "cannot compile Go 1.24 code") and
which raises the go line of every downstream module. The cause was
go-re2/wazero; build tags keep it out of compilation but not out of the
module graph, so go mod tidy refused go 1.17.
- fingers.RegexpCompiler (standard library by default) replaces the
goregexp/re2 build-tag pair; -tags goregexp is now a no-op.
- New module github.com/chainreactors/fingers/re2 (go 1.24) sets RE2 on
import: import _ "github.com/chainreactors/fingers/re2". The RE2
comparison benchmark moves there, on the public API.
- Downgrade to versions declaring go <= 1.17: x/net v0.17.0,
x/text v0.13.0, x/sys v0.13.0, go-flags v1.5.0, jsonschema v0.7.0;
pin kr/text v0.2.0. go mod tidy -go=1.17 -compat=1.17 is clean.
- CI: tidy must keep go 1.17; vet/test on stable Go; a real Go 1.17
module-mode build and test replaces the GOPATH job; the re2 module is
tested on Go 1.24 with the cgo tags.
Verified with go1.17.13 (build, vet, go test ./...), go1.20.14 and
go1.26.5 as downstream toolchains: downstream go.mod stays at go 1.17 and
no wazero enters its module graph.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…ta egress Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Replace directives of a dependency are ignored downstream, so the re2 module must require a fingers version that exists; the local replace stays for development and CI. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
M09Ic
force-pushed
the
feat/jev-judgement-layer
branch
2 times, most recently
from
September 25, 2026 18:03
a341ad1 to
1f42439
Compare
…/jev
Every judgement is a Claim: a statement with finite options, each mapped
to holds / refuted / insufficient, and an explicit abstention. The
generic part (Claim/Option/Outcome/Ruling, validation, Provider, the
exact batch cache and the Jev HTTP client) now lives in
github.com/chainreactors/utils/jev; fingers keeps only what is about
fingerprints:
- judge: Judge{Provider, MinConfidence, DropInsufficient}, New,
Inspect (presence + version in at most two batches), Version, and the
presence options declared/absent/running/mentioned/unrelated.
Code-established facts are rulings of the same claims.
- judge/maintain: Ledger (junk-rule audit from presence annotations) and
Discover (clusters pages shared by hosts, Coverage claim per cluster).
- judge/gen: rule generation for a human-given name.
- Engine.EnableMatchDetail: fingers, fingerprinthub, wappalyzer, ehole
and goby record the matched text so claims can quote it.
Removed: judge/jev, the generic j.Judge, Refine/Classify/Recall and the
Identify/Ask/Yes/Choose/Score question types. judge.New now defaults to
jev.DefaultMinConfidence (0.3). Requires utils/parsers with the
Judgement {Verdict, Outcome, Confidence, Evidence, Rejected, Duplicate}.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
- Framework.Judge.Verdict is now Judge.Option (utils/parsers fd113e9), the same word as the Ruling it mirrors. - cmd/judgeeval: testdata/labels.json is converted once to productLabel arrays (keep/drop/version, plus the Classify-era page_kind/generic fields, are gone; notes become basis), so loadLabels no longer carries the old-format converter. The converted file loads identically. - Tests speak claims/rulings/options instead of questions/answers/ choice/verdicts/refine; the unused testClaim helper is removed. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The same response could produce different merged results between runs, because several places iterated maps: the enabled engines, the fingers engine's candidate rules, fingerprinthub's matched templates, and wappalyzer's apps and their header / cookie / meta patterns. When several rules hit one product, which rule's MatchDetail and Attributes survived depended on that order (and fingers' stopAtFirst returned a random finger). For the judge this changed the evidence, the claims sent to the provider and therefore the cache keys. Engines now run in AllEngines order (others by name), rules and templates in index order, and wappalyzer apps and pattern keys in sorted order, precomputed at compile time. The ledger's report breaks ties on the full rule key, and judgeeval's directory mode adds pages to it in file order, so its samples are reproducible too. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Simplification, verified byte for byte with a deterministic provider
over the full judgeeval pipeline (189 provider requests and all outputs
unchanged):
- Rulings are validated once, by jev.Judge; code-built fact rulings are
valid by construction, so applyPresence / applyVersion return nothing.
- Presence facts (declared / absent) use one package-level factClaim.
- Inspect and Version share one ready check; declares loses its nested
loop; the test-only containsWord is gone.
- gen.New(j, name) takes the required name; Name() and the ProbeWith
wrapper are removed.
- Tests are grouped by mechanism (inspect / presence / version); the
legacy-field guard in judgeeval is dropped.
Behaviour change: the version candidates no longer include the HTTP
version that starts each Via entry ("Via: 1.1 varnish") or the format
version of an SVG / XML declaration (<svg version="1.1">). On the real
retest Jev had picked them as varnish's and caddy's versions.
Retest on the 69-page labelled set with jev-1.13.0
(docs/jev-claim-retest-20260927.md): false hits 24 -> 0, true hits
removed 0, versions 36 correct / 0 wrong with 0 unsupported fills,
ledger top three git / ipeakcms / webp_server_go; without the added
library Discover marks SearXNG, Wakapi and IT Tools missing, and with it
raises no false alarm.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
engine.EnableJudge(apiKey) is the only thing a user configures: every web match (WebMatch, Match, DetectContent, DetectResponse, WebMatchWithEngines) then reviews its rule results and returns those the evidence supports - false positives and duplicate spellings dropped, versions filled. An empty key reads TYPESAFE_API_KEY; EnableJudge turns on match detail and wraps the client in the exact cache. If a request fails the rule results come back unchanged. Engine.Judge stays open for another provider or policy; with it unset matching is unchanged. The example CLI's --judge flag becomes --jev-key and simply calls EnableJudge, so its output is the normal output, reviewed. README leads with the one-line setup. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
- The 2026-09-25/26 reports and the 1070-site report say up front that
they describe removed interfaces (Refine, SuggestNames, Classify) and
were not re-measured with the current implementation.
- Product authenticity review of the three added fingerprints: SearXNG,
Wakapi and IT Tools are real open-source projects; the IT Tools rule
identifies the product family and cannot tell the original from forks.
fingerprints/README.md states that scope.
- Archive of the System One ("habit") design discussion behind Jev.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
re2 replaces fingers with the parent directory, so its go.mod lists the parent's dependencies: add utils/jev and move utils/parsers to the merged Judgement. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
概要
在规则引擎之上加一层指纹判定
judge,设计见 #34。它只审查规则给出的结论,不做指纹识别,模型从不说出产品名。它不是新的引擎,WebMatch和DetectContent的行为不变。与指纹无关的通用机制放在 chainreactors/utils#12 的
github.com/chainreactors/utils/jev,fingers 不新增与指纹识别无关的抽象。通用机制包括 Claim 合约、校验、Provider 接口、批次缓存和 Jev HTTP 客户端。分层
utils/jevClaim/Option/Outcome/Ruling、Resolve、Validate*、Provider、jev.Judge、Cached、Client、DefaultMinConfidence=0.3utils/parsers(#11)Framework.Judge *Judgement,字段为 Option、Outcome、Confidence、Evidence、Rejected、Duplicate;另有Frameworks.Accepted()fingers/judgeJudge{Provider, MinConfidence, DropInsufficient}、New、Inspect、Version;presence 选项有 declared、absent、running、mentioned、unrelated。代码确定的事实也表达为同一 Claim 的 Rulingfingers/judge/maintainLedger:根据 presence 注解统计垃圾规则。Discover:把多个主机上的同一页面聚成簇,每簇提出一个 Coverage Claimfingers/judge/genfingers引擎Engine.EnableMatchDetail():fingers、fingerprinthub、wappalyzer、ehole、goby 记录命中原文,供 Claim 引用证据用法
普通用户只需配置一个 key:
命令行:
engine -u <url> --jev-key <KEY>。请求失败时返回纯规则结果;不调用EnableJudge时行为不变。需要全部注解或自定义策略时直接使用judge包:漏报在离线维护中处理(
judge/maintain.Discover→ 人工命名 →judge/gen),更新到指纹库。文档见 judge/README.md 和 utils 的jev/README.md。本轮变更
gen.New(j, name);Via 头的协议版本和 SVG/XML 声明的格式版本不再作为版本候选。judge/jev、通用的j.Judge、Refine/Classify/ 召回,以及 Identify、Ask、Yes、Choose、Score 这几种题型。judge.New的默认 MinConfidence 改为jev.DefaultMinConfidence(0.3),与 Jev 标定值统一。Go 1.17(沿用前几轮)
go 1.17。RE2 拆到子模块github.com/chainreactors/fingers/re2,主模块默认使用标准库 regexp,通过fingers.RegexpCompiler切换。go build ./...与 judge 测试通过。效果
2026-09-27 用当前代码在 69 份已标注真实响应上做真实 Jev 复测(jev-1.13.0,复测报告):
规则引擎的合并结果现在可复现(原来依赖 map 遍历顺序)。确定性回归显示,judge 精简前后 Provider 请求与输出逐字节一致。
数据外发
发给 Provider 的是响应的精简视图:响应头、Cookie 名、标题、generator/description、脚本和样式路径、内联脚本开头、HTML 注释、表单字段名、正文前 1500 字,以及规则命中上下文。README 中已写明。
依赖与后续
jev)和 feat(parsers): Framework 新增 Judge 判定结论字段 utils#11(parsers),目前固定在两个 PR 的 commit 伪版本上,无 replace。合并后需要打jev/v0.1.0、parsers/v0.0.4并重新固定版本。测试
go vet ./judge/... ./cmd/...go test ./judge/... ./cmd/judgeeval .(go1.26)go build ./...与go test ./judge/...🤖 Generated with Claude Code