Skip to content

feat(judge): 以 Claim 审查指纹结果(合约下沉 utils/jev),并恢复 Go 1.17 支持 - #35

Merged
M09Ic merged 16 commits into
masterfrom
feat/jev-judgement-layer
Sep 27, 2026
Merged

M09Ic merged 16 commits into
masterfrom
feat/jev-judgement-layer

Conversation

@M09Ic

@M09Ic M09Ic commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

概要

在规则引擎之上加一层指纹判定 judge,设计见 #34。它只审查规则给出的结论,不做指纹识别,模型从不说出产品名。它不是新的引擎,WebMatch 和 DetectContent 的行为不变。

与指纹无关的通用机制放在 chainreactors/utils#12 的 github.com/chainreactors/utils/jev,fingers 不新增与指纹识别无关的抽象。通用机制包括 Claim 合约、校验、Provider 接口、批次缓存和 Jev HTTP 客户端。

分层

位置 内容
utils/jev Claim / Option / Outcome / Ruling、Resolve、Validate*、Provider、jev.Judge、Cached、Client、DefaultMinConfidence=0.3
utils/parsers(#11) Framework.Judge *Judgement,字段为 Option、Outcome、Confidence、Evidence、Rejected、Duplicate;另有 Frameworks.Accepted()
fingers/judge Judge{Provider, MinConfidence, DropInsufficient}、New、Inspect、Version;presence 选项有 declared、absent、running、mentioned、unrelated。代码确定的事实也表达为同一 Claim 的 Ruling
fingers/judge/maintain Ledger:根据 presence 注解统计垃圾规则。Discover:把多个主机上的同一页面聚成簇,每簇提出一个 Coverage Claim
fingers/judge/gen 为人给定的产品名,用正反样本生成原生规则
fingers 引擎 Engine.EnableMatchDetail():fingers、fingerprinthub、wappalyzer、ehole、goby 记录命中原文,供 Claim 引用证据

用法

普通用户只需配置一个 key:

engine, _ := fingers.NewEngine()
engine.EnableJudge("")                 // 空字符串读取 TYPESAFE_API_KEY
frames, _ := engine.DetectContent(raw) // WebMatch、Match、DetectResponse 同样生效:去掉误报和重复、补全版本

命令行:engine -u <url> --jev-key <KEY>。请求失败时返回纯规则结果;不调用 EnableJudge 时行为不变。需要全部注解或自定义策略时直接使用 judge 包:

j := judge.New(jev.Cached(c, jev.DefaultCacheSize))
all, err := j.Inspect(ctx, raw, hits) // 含被剔除条目及原因

漏报在离线维护中处理(judge/maintain.Discover → 人工命名 → judge/gen),更新到指纹库。文档见 judge/README.md 和 utils 的 jev/README.md。

本轮变更

  • gen 改为 gen.New(j, name);Via 头的协议版本和 SVG/XML 声明的格式版本不再作为版本候选。
  • 所有判断都统一为 Claim。删除了 judge/jev、通用的 j.Judge、Refine / Classify / 召回,以及 Identify、Ask、Yes、Choose、Score 这几种题型。
  • judge.New 的默认 MinConfidence 改为 jev.DefaultMinConfidence(0.3),与 Jev 标定值统一。
  • 修复 alias 的 Attributes 被跨页面共享、导致版本号泄漏到后续页面的问题。

Go 1.17(沿用前几轮)

  • go.mod 为 go 1.17。RE2 拆到子模块 github.com/chainreactors/fingers/re2,主模块默认使用标准库 regexp,通过 fingers.RegexpCompiler 切换。
  • 实测:go1.17.13 下 go build ./... 与 judge 测试通过。

效果

2026-09-27 用当前代码在 69 份已标注真实响应上做真实 Jev 复测(jev-1.13.0,复测报告):

指标 结果
有标注的误报 24 → 0
真实命中误删 0
版本 正确 36、错误 0,无依据填值 0
Ledger 前三 git、ipeakcms、webp_server_go
Discover,不加载新增规则 SearXNG、Wakapi、IT Tools 全部判 missing
Discover,加载新增规则 无误报警

规则引擎的合并结果现在可复现(原来依赖 map 遍历顺序)。确定性回归显示,judge 精简前后 Provider 请求与输出逐字节一致。

数据外发

发给 Provider 的是响应的精简视图:响应头、Cookie 名、标题、generator/description、脚本和样式路径、内联脚本开头、HTML 注释、表单字段名、正文前 1500 字,以及规则命中上下文。README 中已写明。

依赖与后续

测试

  • go vet ./judge/... ./cmd/...
  • go test ./judge/... ./cmd/judgeeval .(go1.26)
  • go1.17.13 下 go build ./... 与 go test ./judge/...

🤖 Generated with Claude Code

Jev judges the rule result instead of being another engine: rules and code
recall candidates, Jev only judges evidence, code makes the final call.

- jev.Page is the page entry, jev.Round is one request of questions with
  callbacks; Verify / Classify / Version are the capabilities
- results annotate common.Frameworks in place with jev:* tags; a failed
  round leaves frames untouched
- Client cache keyed by exact questions plus a page simhash, so similar
  pages can share answers
- Engine.AttachJev / Engine.Refine; WebMatch and DetectContent unchanged
- cmd/jevbench compares rules vs rules+Jev; cmd/engine gets --jev

Also fix MergeFrameworks sharing alias Attributes across pages: a version
written on one page leaked into every later hit of the same alias.

Refs #34

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Jev is one provider of a generic judgement and rerank layer, not the
layer itself.

- package judge: Provider interface (ID + Judge), Judge (cache, request
  merging, thresholds, optional provider Calibration), neutral question
  types Binary / Choice / Score
- package judge/jev: the TypeSafe Jev provider, protocol and transport only
- marks are one enum (Rejected, Duplicate, Primary, Recalled) read with
  judge.Is / LayerOf / Accepted, stored as judge:* tags; page Kind is an enum
- cache is on by default and per question: any grouping of questions hits,
  partial hits send only the rest, concurrent rounds on similar pages merge
- similar pages must share a title and differ by <= 1 signature bit; text
  is shingled by two runes. On 1070 real pages every answer reused at
  distance <= 1 matched (159/159), 96.9% at the old default of 3, and
  unrelated sparse CJK pages collided without the title guard
- Verify skips judged frames, so Refine on the same frames is free
- version subject without a primary: the only application, else server,
  else device (fixes directory listings next to an OS hit)
- Engine.AttachJudge; cmd/judgeeval -provider; cmd/engine --judge jev
- docs: judge/README.md

Refs #34

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@M09Ic M09Ic changed the title feat(jev): 引入 Jev 判定层(去误报、去重、召回、分层、版本号、页面分类) feat(judge): 引入可替换 Provider 的判定层(去误报、去重、召回、分层、版本号、页面分类),首个 Provider 为 Jev Sep 25, 2026
M09Ic and others added 7 commits September 25, 2026 09:48
…rator

Snapshot of the working tree before the judge layer cleanup: explicit
Judge methods (Refine/Inspect/Classify/Version/...), replayable judgeeval
(-manifest/-history/-maintain/-offline), the fingerprint generator and
the validation reports.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The replace pinned github.com/chainreactors/go-re2 with a pseudo-version
whose timestamp did not match the commit, so go.sum carried another
commit and builds without -tags goregexp failed with a missing go.sum
entry. Pin commit 8121b6cd261e with its real pseudo-version.

Require utils/parsers at the commit adding Framework.Judge
(chainreactors/utils#11), to be re-pinned to parsers/v0.0.4 once tagged.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…n judge/gen

- Verdicts (layer, confidence, rejected, duplicate, primary, recalled) are
  written to parsers.Framework.Judge instead of judge:* tags; callers use
  Frameworks.Accepted/Primary without importing judge.
- Public API: Refine (report), Inspect (explain), Classify (page), Version,
  IsUnknownProduct, SuggestNames, Yes/Choose/Score. Judge.Verify, the
  knownNames variadics, the Round layer and Page.Kind/Generic are gone.
- Judge.Known recalls fingerprint names (Engine.Names) automatically;
  SimilarDistance is a Judge field instead of a mutable global.
- Refine resolves the version of every kept product. Versions a response
  binds by name (Server: nginx/1.24.0, X-Jenkins: 2.401.3, generator meta)
  are taken without asking; the provider picks among candidates ranked by
  closeness to the product name, versions bound to another product are
  not offered, and a string occurring once goes to one product only.
- The fingerprint generator moves to judge/gen, so package judge depends
  only on common; shared text primitives live in judge/internal/evidence.
- judgeeval replay drops the separate per-product Version stage.
- Code in judge, gen, jev and judgeeval is Go 1.17 compatible.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
- fingerprinthub: baseline and case tests use the passive template loader
  and matcher (buildInternalEvent/loadTemplates were removed).
- nmap: NoGuess moved to parsers.
- fingers bench: constant format string for vet.

go vet ./... and go test -short ./... pass with and without -tags goregexp.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
go.mod had drifted to go 1.24.0 with a toolchain line, which Go 1.17-1.20
cannot build ("invalid go version", "cannot compile Go 1.24 code") and
which raises the go line of every downstream module. The cause was
go-re2/wazero; build tags keep it out of compilation but not out of the
module graph, so go mod tidy refused go 1.17.

- fingers.RegexpCompiler (standard library by default) replaces the
  goregexp/re2 build-tag pair; -tags goregexp is now a no-op.
- New module github.com/chainreactors/fingers/re2 (go 1.24) sets RE2 on
  import: import _ "github.com/chainreactors/fingers/re2". The RE2
  comparison benchmark moves there, on the public API.
- Downgrade to versions declaring go <= 1.17: x/net v0.17.0,
  x/text v0.13.0, x/sys v0.13.0, go-flags v1.5.0, jsonschema v0.7.0;
  pin kr/text v0.2.0. go mod tidy -go=1.17 -compat=1.17 is clean.
- CI: tidy must keep go 1.17; vet/test on stable Go; a real Go 1.17
  module-mode build and test replaces the GOPATH job; the re2 module is
  tested on Go 1.24 with the cgo tags.

Verified with go1.17.13 (build, vet, go test ./...), go1.20.14 and
go1.26.5 as downstream toolchains: downstream go.mod stays at go 1.17 and
no wazero enters its module graph.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…ta egress

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Replace directives of a dependency are ignored downstream, so the re2
module must require a fingers version that exists; the local replace stays
for development and CI.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@M09Ic M09Ic changed the title feat(judge): 引入可替换 Provider 的判定层(去误报、去重、召回、分层、版本号、页面分类),首个 Provider 为 Jev feat(judge): 可替换 Provider 的判定层(Framework.Judge、Refine/Inspect/Classify),并恢复 Go 1.17 支持 Sep 25, 2026
@M09Ic
M09Ic force-pushed the feat/jev-judgement-layer branch 2 times, most recently from a341ad1 to 1f42439 Compare September 25, 2026 18:03
…/jev

Every judgement is a Claim: a statement with finite options, each mapped
to holds / refuted / insufficient, and an explicit abstention. The
generic part (Claim/Option/Outcome/Ruling, validation, Provider, the
exact batch cache and the Jev HTTP client) now lives in
github.com/chainreactors/utils/jev; fingers keeps only what is about
fingerprints:

- judge: Judge{Provider, MinConfidence, DropInsufficient}, New,
  Inspect (presence + version in at most two batches), Version, and the
  presence options declared/absent/running/mentioned/unrelated.
  Code-established facts are rulings of the same claims.
- judge/maintain: Ledger (junk-rule audit from presence annotations) and
  Discover (clusters pages shared by hosts, Coverage claim per cluster).
- judge/gen: rule generation for a human-given name.
- Engine.EnableMatchDetail: fingers, fingerprinthub, wappalyzer, ehole
  and goby record the matched text so claims can quote it.

Removed: judge/jev, the generic j.Judge, Refine/Classify/Recall and the
Identify/Ask/Yes/Choose/Score question types. judge.New now defaults to
jev.DefaultMinConfidence (0.3). Requires utils/parsers with the
Judgement {Verdict, Outcome, Confidence, Evidence, Rejected, Duplicate}.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@M09Ic M09Ic changed the title feat(judge): 可替换 Provider 的判定层(Framework.Judge、Refine/Inspect/Classify),并恢复 Go 1.17 支持 feat(judge): 以 Claim 审查指纹结果(合约下沉 utils/jev),并恢复 Go 1.17 支持 Sep 27, 2026
M09Ic and others added 6 commits September 26, 2026 20:32
- Framework.Judge.Verdict is now Judge.Option (utils/parsers fd113e9),
  the same word as the Ruling it mirrors.
- cmd/judgeeval: testdata/labels.json is converted once to productLabel
  arrays (keep/drop/version, plus the Classify-era page_kind/generic
  fields, are gone; notes become basis), so loadLabels no longer carries
  the old-format converter. The converted file loads identically.
- Tests speak claims/rulings/options instead of questions/answers/
  choice/verdicts/refine; the unused testClaim helper is removed.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The same response could produce different merged results between runs,
because several places iterated maps: the enabled engines, the fingers
engine's candidate rules, fingerprinthub's matched templates, and
wappalyzer's apps and their header / cookie / meta patterns. When several
rules hit one product, which rule's MatchDetail and Attributes survived
depended on that order (and fingers' stopAtFirst returned a random
finger). For the judge this changed the evidence, the claims sent to the
provider and therefore the cache keys.

Engines now run in AllEngines order (others by name), rules and
templates in index order, and wappalyzer apps and pattern keys in sorted
order, precomputed at compile time. The ledger's report breaks ties on
the full rule key, and judgeeval's directory mode adds pages to it in
file order, so its samples are reproducible too.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Simplification, verified byte for byte with a deterministic provider
over the full judgeeval pipeline (189 provider requests and all outputs
unchanged):
- Rulings are validated once, by jev.Judge; code-built fact rulings are
  valid by construction, so applyPresence / applyVersion return nothing.
- Presence facts (declared / absent) use one package-level factClaim.
- Inspect and Version share one ready check; declares loses its nested
  loop; the test-only containsWord is gone.
- gen.New(j, name) takes the required name; Name() and the ProbeWith
  wrapper are removed.
- Tests are grouped by mechanism (inspect / presence / version); the
  legacy-field guard in judgeeval is dropped.

Behaviour change: the version candidates no longer include the HTTP
version that starts each Via entry ("Via: 1.1 varnish") or the format
version of an SVG / XML declaration (<svg version="1.1">). On the real
retest Jev had picked them as varnish's and caddy's versions.

Retest on the 69-page labelled set with jev-1.13.0
(docs/jev-claim-retest-20260927.md): false hits 24 -> 0, true hits
removed 0, versions 36 correct / 0 wrong with 0 unsupported fills,
ledger top three git / ipeakcms / webp_server_go; without the added
library Discover marks SearXNG, Wakapi and IT Tools missing, and with it
raises no false alarm.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
engine.EnableJudge(apiKey) is the only thing a user configures: every
web match (WebMatch, Match, DetectContent, DetectResponse,
WebMatchWithEngines) then reviews its rule results and returns those the
evidence supports - false positives and duplicate spellings dropped,
versions filled. An empty key reads TYPESAFE_API_KEY; EnableJudge turns
on match detail and wraps the client in the exact cache. If a request
fails the rule results come back unchanged. Engine.Judge stays open for
another provider or policy; with it unset matching is unchanged.

The example CLI's --judge flag becomes --jev-key and simply calls
EnableJudge, so its output is the normal output, reviewed. README leads
with the one-line setup.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
- The 2026-09-25/26 reports and the 1070-site report say up front that
  they describe removed interfaces (Refine, SuggestNames, Classify) and
  were not re-measured with the current implementation.
- Product authenticity review of the three added fingerprints: SearXNG,
  Wakapi and IT Tools are real open-source projects; the IT Tools rule
  identifies the product family and cannot tell the original from forks.
  fingerprints/README.md states that scope.
- Archive of the System One ("habit") design discussion behind Jev.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
re2 replaces fingers with the parent directory, so its go.mod lists the
parent's dependencies: add utils/jev and move utils/parsers to the
merged Judgement.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@M09Ic
M09Ic merged commit e870395 into master Sep 27, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant