Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ this package adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm

## [Unreleased]

- Reuse unchanged discovery presentation decisions during reconciliation instead of reassembling every existing agent when one is created (#333). Invalidate on live authority inputs, registry/configuration and ownership changes, and host-service lifecycle; keep pre-step and execution checks fresh. Avoid empty worker-provider notifications and retain asynchronous invalidation and unload safety.
- 发现组件 reconcile 复用未变化的 presentation 判定,不再因新增一个 agent 而重组全部旧 agent(#333)。实时权限输入、注册表/配置、ownership 与宿主服务生命周期变化均使缓存失效;pre-step 和执行检查保持新鲜。避免空 worker provider 发送无效通知,保留异步失效和卸载安全性。

## [0.4.1] — 2026-10-20 — Update locks stop leaving files behind

- Stop the update lock from leaving `.dsh-ops-update-*.lock` sidecars behind (#319). The writer that takes the lock removes it on release, and only after proving the name still leads to the inode it locked, so a waiter that opened the old inode can never publish under it. A published write, a request refused after the lock and a directory-wide `replace` now leave the directory exactly as they found it; a sidecar left by a killed writer is not a stuck lock — the next write of that name binds to it, publishes under it and removes it.
Expand Down
12 changes: 11 additions & 1 deletion docs/discovery.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,19 @@ Every activated file tool keeps its declared path base, argument schema, Native

Only this plugin's scoped `ctx.tools.register`, `restrict`, prompt sections and lifecycle listeners are used. No host method or shared tool registry is replaced. Publication compares the desired tool names before changing registrations so it can coexist with PTC lifecycle listeners.

## Incremental presentation checks

Adding an agent probes its presentation without reassembling unchanged agents. Background reconciliation reuses only a successful presentation decision, keyed by the agent context, session, cwd and freshly resolved sandbox policy (mode, workspace root and session identity). Starting the discovery worker with no resident tools does not emit a no-op tool-registry change. The inexpensive policy/ownership and desired-tool diff still visits existing entries; this is not a claim of constant-time reconciliation.

`tools/change`, `system-prompt/change`, runtime configuration updates and discovery-row ownership changes invalidate decisions. The global runtime-update hook awaits `next()` before invalidating: the pinned host's default returns an asynchronous `Fiber.restart()` despite its `void` declaration, while an in-place update can be synchronous. The pinned host's registry notifications have no affected-scope payload, so genuine registry changes conservatively invalidate all entries, including a new scoped presentation declaration. Discovery registers its own prompt provider once per row, not once per agent. A session's `sandbox/mode` event invalidates only its entries. Required host-service unload removes the row's registrations and cache; reload rebuilds them. Optional PTC-runtime mount/unmount also refreshes decisions, and failed probes are not cached.

Arbitrary dynamic prompt providers need not emit any change event. For that reason, each model pre-step and every tool execution performs a fresh check for its own agent; this cache never authorizes a call. Runtime policy and cwd remain live inputs, including when the same agent object is reused. Native and `both` can publish discovery; pure PTC cannot. Invalidation during an asynchronous probe retires that result before publication; queued work and unload cleanup remain authoritative.

`node test/run.mjs discovery-incremental discovery-composition presentation-readiness progressive-loop` covers counting, invalidation, scoped ownership and lifecycle races. The incremental fixture adds one agent beside twelve existing agents and asserts **zero old-agent probes and one new-agent probe**; this is a deterministic assembly count, not a wall-clock benchmark.

## Measurement boundaries

The deterministic entry measurement uses `[email protected]` with `cl100k_base`, encoding compact JSON of the pinned host `ToolSchema` fields (`name`, `description`, `parameters`). The discovery entry is **94 tokens / 410 UTF-8 bytes**. This is a reproducible schema measurement, not provider billing, and excludes tool-message framing, prompt guidance and other OPS components. The tokenizer is a test-only dependency published on 2025-08-09; its locked `base64-js` dependency is also test-only.
The deterministic entry measurement uses `[email protected]` with `cl100k_base`, encoding compact JSON of the pinned host `ToolSchema` fields (`name`, `description`, `parameters`). The discovery entry is **95 tokens / 416 UTF-8 bytes**. This is a reproducible schema measurement, not provider billing, and excludes tool-message framing, prompt guidance and other OPS components. The tokenizer is a test-only dependency published on 2025-08-09; its locked `base64-js` dependency is also test-only.

The default file/discovery surface is now `ops_read_files` plus `ops_discover_tools`: **242 tokens / 1101 bytes** in actual host assembly order. The reader accepts multiple file ranges in one startup call; `ops_inspect_local_file` (including hex), search, mutations and Native result paging are discoverable. Routing names only published tools. The final Windows provider order with the full Bash/jobs contract measures **662 tokens / 3226 bytes** (the pre-rename snapshot was 660/3219; the older catalogue-order figure was 661 tokens). These full totals remain visible under the explicitly approved BM-03 scope: the ≤400-token gate applies to file/discovery schemas, while Bash remains at startup. Numeric bounds, defaults, path bases and permission contracts remain intact. Operator always-loaded choices expand the initial surface. See [benchmark evidence](benchmark-gates.md).

Expand Down
12 changes: 11 additions & 1 deletion docs/工具发现.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,19 @@ English: [Advanced tools on demand](discovery.md).

只使用本插件自己的作用域 `ctx.tools.register`、`restrict`、提示词段落和生命周期监听;不替换宿主方法或共享工具注册表。发布前比较目标工具名称,避免与 PTC 生命周期监听器互相触发重复注册。

## 增量 presentation 检查

新增 agent 时只探测它的 presentation,不重新组装未变化的旧 agent。后台 reconcile 仅复用成功的 presentation 判定,匹配 agent context、session、cwd 以及实时解析的沙箱策略(mode、workspace root、session identity)。发现组件以空常驻工具集启动 worker 时,不再发送无实际变化的工具注册表事件。低成本的策略、ownership 与目标工具差异检查仍遍历已有条目;这不代表 reconcile 整体变为常数时间。

`tools/change`、`system-prompt/change`、运行时配置更新和发现行 ownership 变更会使判定失效。全局 runtime-update hook 等待 `next()` 完成后再失效:固定宿主虽将其声明为 `void`,默认路径实际返回异步 `Fiber.restart()`;原地更新则可以同步完成。固定宿主版本的注册表通知没有受影响作用域参数,因此真实注册表变化会保守地使所有条目失效,包括新增作用域的 presentation 声明。发现组件自己的 prompt provider 每行只注册一次,而不是每个 agent 注册一次。会话的 `sandbox/mode` 事件只使自身条目失效。必需宿主服务卸载时移除本行注册及缓存,重载后重新建立;可选 PTC runtime 挂载、卸载也会刷新判定,探测失败不缓存。

任意动态提示词 provider 不保证发送变化事件,所以每次模型 pre-step 和每次工具执行仍对自己的 agent 做新鲜检查,缓存绝不用于授权调用。即使复用同一个 agent 对象,运行时策略与 cwd 也保持实时。Native 和 `both` 可以发布发现工具,纯 PTC 不发布。异步探测期间发生失效时,旧结果在发布前作废;后续排队任务和卸载清理仍具有最终效力。

`node test/run.mjs discovery-incremental discovery-composition presentation-readiness progressive-loop` 覆盖计数、失效、作用域 ownership 和生命周期竞争。增量 fixture 在十二个已有 agent 旁新增一个,断言**旧 agent 探测零次、新 agent 一次**;这是确定性的组装计数,不是耗时基准。

## 测量边界

确定性的入口测量固定 `[email protected]` 和 `cl100k_base`,编码固定宿主 `ToolSchema` 字段(`name`、`description`、`parameters`)的紧凑 JSON。发现入口为 **94 tokens / 410 UTF-8 字节**。这是可复现的 schema 测量,不是供应商账单;不含消息封装、提示词指引和其它 OPS 组件。分词器是测试依赖,发布于 2025-08-09;锁定的 `base64-js` 也仅供测试使用。
确定性的入口测量固定 `[email protected]` 和 `cl100k_base`,编码固定宿主 `ToolSchema` 字段(`name`、`description`、`parameters`)的紧凑 JSON。发现入口为 **95 tokens / 416 UTF-8 字节**。这是可复现的 schema 测量,不是供应商账单;不含消息封装、提示词指引和其它 OPS 组件。分词器是测试依赖,发布于 2025-08-09;锁定的 `base64-js` 也仅供测试使用。

默认文件/发现工具面为 `ops_read_files` 加 `ops_discover_tools`,**242 tokens / 1101 字节**。完整 Windows Bash/jobs 契约按供应商实际顺序为 **662 tokens / 3226 字节**(改名前快照为 660/3219,更早的目录顺序口径为 661 tokens)。已明确批准 BM-03 的 ≤400 token 门禁仅作用于文件/发现 schema;Bash 保持启动常驻,其成本及完整总数继续单列。高级读取、搜索、修改、文档与结果分页按需发现;路由只列出已发布工具。数值边界、默认值、路径基准与权限契约保持不变;始终加载项会扩大初始面。见[基准证据](基准门禁.md)。

Expand Down
62 changes: 52 additions & 10 deletions src/discovery/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ interface Entry {
disposers: (() => void)[]
owned: Set<string>
ready: Promise<void>
revision: number
presentation?: { inputs: unknown[]; native: boolean }
releaseSelection?: () => void
}
const DISCOVER = publicToolName('discover_tools')
Expand Down Expand Up @@ -68,38 +70,59 @@ export function apply(root: Context, raw: unknown): void {
} catch { return false }
}
const ptcOnly = (assembly: PromptAssembly): boolean => assembly.sections.some(section => section.name === 'tools:ptc-only' && Boolean(section.text.trim()))
const native = (agent: Agent): Promise<boolean> => isNativePresentation(prompt, agent)
const invalidate = (entry: Entry) => { ++entry.revision; entry.presentation = undefined }
const inputs = (agent: Agent): unknown[] => {
const policy = policyService.resolve({ session: agent.session })
return [agent.ctx, agent.session, agent.session.header.cwd, policy.mode, policy.workspaceRoot, policy.sessionId]
}
const same = (left: unknown[], right: unknown[]) => left.length === right.length && left.every((value, index) => value === right[index])
// Only background reconciliation reuses a probe. A step or execution can
// observe arbitrary dynamic prompt providers with no registry notification.
const native = async (agent: Agent, fresh = false): Promise<boolean> => {
const entry = entries.get(agent)
if (!entry) return false
if (fresh) invalidate(entry)
const current = inputs(agent), revision = entry.revision
if (entry.presentation && same(entry.presentation.inputs, current)) return entry.presentation.native
const result = await isNativePresentation(prompt, agent)
if (!stopped && entries.get(agent) === entry && revision === entry.revision && same(current, inputs(agent))) {
entry.presentation = { inputs: current, native: result }
}
return result
}
const own = async (agent: Agent, entry: Entry, tool: ToolDefinition): Promise<void> => {
if (!entry.ctx) return
const execute = tool.execute
const registration: ToolDefinition = { ...tool, execute: async (args, exec) => {
await chain
const version = epoch
const available = () => ownership?.owns(agent) && !stopped && entries.get(agent) === entry && exec.agent === agent && entry.owned.has(tool.name)
if (!available() || !await native(agent).catch(() => false) || !available() || version !== epoch) {
if (!available() || !permitted(agent) || !await native(agent, true).catch(() => false) || !available() || version !== epoch) {
return failure('POLICY_DENIED', 'Discovery capability belongs to a different or inactive scope.')
}
return execute(args, exec)
} }
entry.disposers.push(await registerNativeProvider(entry.ctx, [registration], tool => entry.ctx!.tools.register(tool), { inheritable: false, priority: 10 }))
const dispose = await registerNativeProvider(entry.ctx, [registration], tool => entry.ctx!.tools.register(tool), { inheritable: false, priority: 10 })
if (stopped || entries.get(agent) !== entry || !ownership?.owns(agent)) { dispose(); return }
entry.disposers.push(dispose)
entry.owned.add(tool.name)
}
const inheritedNative = (agent: Agent, entry: Entry): string[] => nativeProviders(ctx, agent)
.map(provider => provider.definition.name).filter(name => (advanced.includes(name) || name === DISCOVER) && !entry.owned.has(name))
const reconcile = (): Promise<void> => {
if (stopped || mutating) return chain
if (stopped) return chain
const version = ++epoch
chain = chain.then(async () => {
if (stopped || version !== epoch) return
const ready: [Agent, Entry][] = []
for (const [agent, entry] of entries) if (entry.ctx && ownership?.owns(agent) && permitted(agent) && await native(agent).catch(() => false)) ready.push([agent, entry])
const ready = new Set<Agent>()
for (const [agent, entry] of entries) if (entry.ctx && ownership?.owns(agent) && permitted(agent) && await native(agent).catch(() => false)) ready.add(agent)
if (stopped || version !== epoch) return
mutating = true; ownership?.begin()
try {
for (const [agent, entry] of entries) {
if (!ownership?.owns(agent)) { clear(entry); entry.releaseSelection?.(); entry.releaseSelection = undefined; continue }
entry.releaseSelection ??= claimNativeSelection(ctx, agent, new Set(advanced))
const isReady = ready.some(([candidate]) => candidate === agent)
const isReady = ready.has(agent)
const wanted = isReady ? [...new Set([...config.alwaysLoaded, ...entry.selected])].filter(tool => permitted(agent, tool)) : []
const desired = new Set([...(isReady && config.enabled ? [DISCOVER] : []), ...wanted.map(publicToolName)])
const inherited = inheritedNative(agent, entry)
Expand Down Expand Up @@ -142,7 +165,7 @@ export function apply(root: Context, raw: unknown): void {
const attach = (agent: Agent): Promise<void> => {
if (stopped || entries.has(agent) || !agent.ctx) return entries.get(agent)?.ready ?? Promise.resolve()
const entry: Entry = { runtime: new NativeFileTools({ ctx, config: { ...config, workerPath: worker }, report: undefined }),
selected: new Set(), pending: new Set(), disposers: [], owned: new Set(), ready: Promise.resolve() }
selected: new Set(), pending: new Set(), disposers: [], owned: new Set(), ready: Promise.resolve(), revision: 0 }
entries.set(agent, entry); ownership?.track(agent)
entry.fiber = agent.ctx.plugin({ inject: ['tools'], apply(scoped) { entry.ctx = scoped } })
entry.ready = Promise.resolve(entry.fiber).then(async () => { await entry.runtime.start([]); await reconcile() })
Expand All @@ -152,7 +175,7 @@ export function apply(root: Context, raw: unknown): void {
ctx.on('agent/pre-step', async (payload, next) => {
await attach(payload.agent)
const entry = entries.get(payload.agent)
if (entry && ownership?.owns(payload.agent) && !payload.signal.aborted && await native(payload.agent).catch(() => false)) {
if (entry && ownership?.owns(payload.agent) && !payload.signal.aborted && await native(payload.agent, true).catch(() => false)) {
for (const tool of entry.pending) if (permitted(payload.agent, tool)) entry.selected.add(tool)
entry.pending.clear()
await reconcile()
Expand All @@ -166,7 +189,24 @@ export function apply(root: Context, raw: unknown): void {
try { clear(entry); entry.releaseSelection?.() } finally { mutating = false }
await entry.runtime.stop(); await entry.fiber?.dispose(); await reconcile()
})
ctx.on('tools/change', () => { if (!ownership?.publishing()) void reconcile() })
const refresh = () => {
if (stopped) return
for (const entry of entries.values()) invalidate(entry)
void reconcile()
}
ctx.on('tools/change', () => { if (!mutating && !ownership?.publishing()) refresh() })
ctx.on('system-prompt/change', refresh)
// Observe in-place hooks and the default asynchronous Fiber.restart().
// Await the runtime result even though the pinned next() type says void.
ctx.on('internal/update', async (_config, _noSave, next) => {
try { return await next() } finally { refresh() }
}, { global: true, prepend: true })
// This optional service is not an injection prerequisite: native agents
// work without it, while failed PTC probes must recover when it arrives.
ctx.inject(['ptcRuntime'], inner => {
refresh()
inner.effect(() => () => { refresh() })
})
ctx.on('system-prompt/assemble', async (_assembly, context, next) => {
// The pinned host assembles BEFORE agent/pre-step. Wait for publication
// queued by the previous committed step, then obtain a fresh public assembly.
Expand Down Expand Up @@ -202,12 +242,14 @@ export function apply(root: Context, raw: unknown): void {
mutating = true
try {
for (const [agent, entry] of entries) if (agent.session === session) {
invalidate(entry)
entry.pending.clear(); entry.runtime.cancel('Sandbox authority changed'); clear(entry)
}
} finally { mutating = false }
void reconcile()
})
ownership = await ownScopes(ctx, 'discovery', () => {
for (const entry of entries.values()) invalidate(entry)
mutating = true
try {
for (const [agent, entry] of entries) if (!ownership?.owns(agent)) {
Expand Down
4 changes: 3 additions & 1 deletion src/file/runtime.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,9 @@ export class NativeFileTools {
return []
}
const definitions = names.map(rawName => this.definition(rawName))
this.unregisterProvider = await registerNativeProvider(this.ctx, definitions, tool => this.ctx.tools.register(tool))
// Discovery starts a worker without a resident surface. Do not emit a
// registry change for an empty provider on every newly attached agent.
if (definitions.length) this.unregisterProvider = await registerNativeProvider(this.ctx, definitions, tool => this.ctx.tools.register(tool))
return names.map(publicToolName)
}

Expand Down
Loading
Loading