Skip to content

[api][plan][runtime] Introduce AGENT resource type and sub-agent invocation API - #938

Open
pltbkd wants to merge 11 commits into
apache:mainfrom
pltbkd:subagent-framework
Open

[api][plan][runtime] Introduce AGENT resource type and sub-agent invocation API#938
pltbkd wants to merge 11 commits into
apache:mainfrom
pltbkd:subagent-framework

Conversation

@pltbkd

@pltbkd pltbkd commented Jul 29, 2026

Copy link
Copy Markdown

Design discussion: #909
Linked issue: #1082

Purpose of change

Introduces AGENT as a first-class resource type and the caller-facing sub-agent invocation API across Java / Python / YAML, per discussion #909.

SubagentSetup is the general sub-agent framework. This PR lands the framework and the one way to plug into it today: a custom SubagentSetup implementation. Registering an Agent directly as an internal sub-agent is a follow-up addition.

Included:

  • ResourceType.AGENT; register via addResource(name, AGENT, setup) and via YAML subagents:.
  • SubagentSetup (single caller-facing surface) + SubagentFuture / SubagentFutures (handles) + SubagentResult (outcome). Implementations capture their failures into a result instead of throwing, so callers check isSuccess(); getResult(Class) re-binds the payload identically on a first execution and on a replay.
  • submit(ctx, prompt) is the preferred form and returns a handle; passing a sessionId continues an earlier conversation. Python mirrors the surface as await setup.submit(...) then await future.
  • Every invocation runs through durable execution under a deterministic (sessionId, callId) identity, framework-assigned from the caller-side facts of the action execution, so failover replay reuses the persisted outcome instead of re-invoking.
  • Two execution modes ship on the runtime base, so an implementation supplies only its transport: deferred (the request is issued when the handle is first resolved) and async pub/sub (one durable POST wired to a reconciler, then poll status and fetch, paced by the status_poll_interval_millis descriptor argument).
  • Unresolved handles are caught: a task finishing while a submitted handle is still pending fails the action instead of dropping the call. This generalized two runtime hooks — TaskLifecycleListener / ComponentExecutionListener replace trace/ExecutionEventLogger + ExecutionEventSink, and ResourceCache.eagerMaterialize registers setups of either language while the operator opens (Python-owned ones reach Java as a non-owning PythonRuntimeResource handle).

Evolved from the #909 proposal during review:

  • SubagentResult (renamed from Result) carries the failure's type and message instead of a live Exception, so it survives durable persistence; the full stack trace is logged rather than persisted, keeping the payload bounded.
  • The separate Subagent caller interface was dropped and merged into SubagentSetup, to avoid a second public type with "agent" in the name meaning something other than the authored, event-driven Agent.
  • BaseSubagentCallable was replaced by the runtime setup base plus the two execution modes, which wire the reconciler by default instead of leaving it to each implementation.
  • The durable id is derived solely from (sessionId, callId); on the Python side durable execution gained explicit id keying, since deriving the id from the callable alone collapsed every sub-agent call onto one key.
  • YAML subagents: is treated as first-class API (schema + loader), not doc-only.

Notes

  • Follow-up additions: ChatModel calling sub-agents (sub-agent-as-tool dispatch), and internal sub-agent (registering an Agent directly as an AGENT resource, compiled into a scoped child plan, and its isolated execution).
  • Cross-language execution (an action calling a sub-agent implemented in the other language) is not supported yet; registration round-trips across the plan-JSON boundary, and execution is planned to route through a same-language delegate so the coroutine boundary stays inside one language.
  • Timeout / cancellation semantics are not included yet; planned as a follow-up.
  • Group resolution prepares every pending deferred handle up front and then executes them one by one; batching them into a single durable submission builds on Parallel Tool Execution ([Feature] Parallel Tool Call Execution #926) and is left as a follow-up. It stays internal to group resolution and does not affect this API.
  • A sessionId may be passed to continue an earlier conversation, but whether that works across action executions is up to the implementation: the framework does not carry the per-session ordinal across action tasks, because branch and diamond structures make the acquisition order unreproducible.

Tests

  • Java unit: AGENT registration and plan compilation (SubagentSetup + YAML descriptor), typed result on first execution and on replay, deterministic id assignment and its namespace digest, the per-task pending-call registry, task lifecycle notification including replay pairing, eager materialization in both languages, and both execution modes (deferred submit / cancel / group resolve, async POST-probe-fetch with reconcile paths and the configured poll interval).
  • Python unit: mirrors the Java surface (registration, plan resources, id assignment, both execution modes, durable id keying, eager materialization, lifecycle bridge).
  • Python e2e: both modes end to end as Python actions running on the Java runtime over pemja in a MiniCluster, including a failed run surfaced via SubagentResult.
  • Java integration: ExternalAgentSubagentSetupTest drives both modes against an external agent service over real HTTP — success, remote failure, the two resolved as one group, and the reconcile paths across a crash window (the POST landed, and the POST was lost). An in-process stub service runs by default and needs nothing installed; -Dexternal.agent.url=... points the same suite at a real deployment, which is how it was also verified against a demo service.

API

Additive public API, kept semantically aligned across Java / Python / YAML: ResourceType.AGENT, SubagentSetup, SubagentFuture, SubagentFutures, SubagentResult, and the YAML subagents: block.

No breaking changes to existing APIs. Runtime-internal only: the tracing hooks became the TaskLifecycleListener / ComponentExecutionListener extension points.

Documentation

  • doc-needed but deferred: add once the internal sub-agent lands and the API stabilizes
  • doc-not-needed
  • doc-included

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-needed Your PR changes impact docs. fixVersion/0.4.0 priority/major Default priority of the PR or issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants