Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,11 @@
## [Unreleased]

### Added
- `azure` model provider for Azure Foundry / Azure OpenAI with Microsoft Entra ID
auth (managed identity, AKS workload identity, `az login`); no API key needed
- Structured output support for the agentic loop
- Harvest MCP integration example (`examples/harvest/`) with config and auth docs
- Human-in-the-loop tool approval: `safety.approval` (`mode: none | all | selected`

Check warning on line 15 in CHANGELOG.md

View workflow job for this annotation

GitHub Actions / QualOps Code Quality

MEDIUM: bug

Module-level `azureTokenProvider` singleton is never reset, causing test isolation problems and preventing credential refresh on config changes within a process
with glob tool-name patterns). Gated tool calls pause the run via a
`tool_approval_requested` event and are resolved by the client on the next
request; optional `TOOL_APPROVAL_SECRET` HMAC-signs approvals
Expand All @@ -20,13 +22,13 @@
- End-user documentation guide at `docs/user-guide.md`
- `run_paused` SSE event carrying the paused run's messages, so a stateless
client can faithfully resume after a tool-approval decision
- Log/trace correlation: JSON logs include `trace_id`/`span_id`/`trace_flags`

Check failure on line 25 in CHANGELOG.md

View workflow job for this annotation

GitHub Actions / QualOps Code Quality

HIGH: bug

`cfg.baseUrl` is typed `string | undefined` at the TypeScript level and is passed directly to `createAzure` without a non-null assertion or type guard
of the active span plus `service.name`/`service.version`; each `/invoke`
request runs in its own span (parented on an incoming `traceparent`) and
returns an `x-request-id` header
- `OTEL_RECORD_CONTENT=0` strips prompts/tool contents from exported spans
- `SHUTDOWN_TIMEOUT_MS` bounds the graceful-shutdown drain
- `model.baseUrl` now works for the hosted providers too (anthropic, openai,

Check notice on line 31 in CHANGELOG.md

View workflow job for this annotation

GitHub Actions / QualOps Code Quality

LOW: maintainability

The fetch-URL assertion in the Azure model test is tightly coupled to `@ai-sdk/azure` internals and will silently break on SDK upgrades
google) to route traffic through a gateway or proxy
- `/invoke` request bodies are capped (`MAX_REQUEST_BODY_BYTES`, default 10 MiB)
- MCP connect + discovery is bounded by `MCP_DISCOVERY_TIMEOUT_MS` (default
Expand Down Expand Up @@ -82,7 +84,7 @@
split point required a user message inside the recent window, so agentic runs
never compacted and eventually exceeded the provider context limit
- Provider errors are sanitized before reaching SSE clients: no more raw
request bodies (prompts/system prompt) or provider response bodies/headers;

Check notice on line 87 in CHANGELOG.md

View workflow job for this annotation

GitHub Actions / QualOps Code Quality

LOW: bug

`vi.stubEnv('AZURE_API_KEY', '')` in the server test stubs a non-existent Azure environment variable, making the test setup misleading
the 429 path emits only rate-limit headers. Server logs use a whitelist
error serializer for the same reason
- `safety.toolOutput.tailChars: 0` no longer leaks the entire raw oversized
Expand Down
28 changes: 28 additions & 0 deletions docs/specs/016_azure_entra_auth_instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# 016 — Azure Foundry with Entra ID auth (instructions)

## Original request

> We are running the configurable agent on Azure, now as a Container App,
> later in AKS. OpenAI key authentication is turned off. Let's first plan:
> what do we need to add to this project to be able to auth to Azure Foundry?

Follow-ups during planning:

> Currently the app is calling OpenAI directly. In Azure I see a
> `<name>.openai.azure.com` URL for the OpenAI API.

> Keep MCP as a separate step, but keep it in mind — we will probably need
> that later.

> Write plan and instructions to add Entra / Entra token support, in the
> simplest way possible.

## Context

- Today the agent only authenticates with API keys from env vars
(`OPENAI_API_KEY`, …). The Foundry resource has key auth disabled, so the
agent must use Microsoft Entra ID tokens.
- Runtime identity: managed identity on Azure Container Apps now, AKS
Workload Identity later.
- `@ai-sdk/azure` (≥ 4.0) supports Entra natively via a `tokenProvider`
option; `@azure/identity` supplies the token.
127 changes: 127 additions & 0 deletions docs/specs/016_azure_entra_auth_plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# 016 — Azure Foundry with Entra ID auth (plan)

## Principle

Add one provider, `azure`, that always authenticates with Microsoft Entra ID.
No API keys and no custom auth code: `@ai-sdk/azure` sends the bearer token,
`@azure/identity` gets and refreshes it. The same code runs on Container
Apps, AKS and a dev laptop.

## Why code is needed

Without code changes, the only way to send an Entra token is a static token
in `OPENAI_API_KEY` with `provider: openai` and the Azure v1 `baseUrl`. The
`openai` provider sends it as `Authorization: Bearer`, which the Azure v1
endpoint accepts. However, the token expires, and nothing in the process can
refresh it. A proxy that adds the token (APIM or a sidecar) adds an infra
component and moves the auth problem to the agent → proxy hop. Neither is fit
for production.

## Smoke test before implementing

Use the static-token path to check network, RBAC, the deployment name and
Responses API support, with no code changes:

```sh
export OPENAI_API_KEY=$(az account get-access-token \
--resource https://cognitiveservices.azure.com --query accessToken -o tsv)
# config: provider: openai, name: <deployment>,
# baseUrl: https://<name>.openai.azure.com/openai/v1
CONFIG_PATH=./config.yaml pnpm dev serve &
curl -s 'localhost:3000/ready?deep=1' # {"status":"ok","probe":"passed"}
```

## Config

```yaml
model:
provider: azure
name: <deployment-name> # Foundry deployment name, not the model name
baseUrl: https://<name>.openai.azure.com/openai
```

- `baseUrl` is the `/openai` prefix. The SDK appends `/v1/...` and
`?api-version=v1`. The latest `@ai-sdk/azure` also accepts this form, so
configs survive an SDK upgrade.

- `baseUrl` is required for `azure` (schema refine with a clear error).
- No other new keys. The token scope is fixed to
`https://cognitiveservices.azure.com/.default`, the scope for
`*.openai.azure.com`.

## Changes

- **Dependencies:** `pnpm add @ai-sdk/[email protected] @azure/identity`.
`@ai-sdk/azure` 4.0.11 is the release that matches the installed
`@ai-sdk/openai` 4.0.11 and the pinned `@ai-sdk/provider` 4.0.3. Newer
releases bring a newer `@ai-sdk/provider` and fail typecheck. Upgrading all
`@ai-sdk/*` packages together is a separate task.
- **`src/config/schema.ts`:** add `azure` to `ModelProvider` and its error
message; refine `model` so `provider: azure` requires `baseUrl`.
- **`src/agent/model.ts`:**
- `case 'azure'` returns
`createAzure({ baseURL: cfg.baseUrl, tokenProvider })(cfg.name)`. This
uses the Responses API, the same as the current `openai` provider.
- `tokenProvider` is
`getBearerTokenProvider(new DefaultAzureCredential(), scope)`. It is
created lazily on first use and kept in a module-level variable, so the
token cache is shared by the main loop, compaction, tool summaries and
the `/ready` probe.
- `requiredEnvVarFor('azure')` returns `null`. There is no key to check.
`/ready?deep=1` checks the real path: token, then endpoint, then
deployment.
- **Credential selection** is done by `DefaultAzureCredential`, with no code
per environment:
- Container Apps: `ManagedIdentityCredential`. For a user-assigned
identity, set `AZURE_CLIENT_ID`.
- AKS: `WorkloadIdentityCredential`, using env vars injected by the
Workload Identity webhook.
- Local dev: `AzureCliCredential`, after `az login`.
- Optional in production: `AZURE_TOKEN_CREDENTIALS=prod` skips the dev
credentials, for faster and predictable startup.

## Azure setup (documented, not code)

- Give the Container App a managed identity. User-assigned is recommended
because it can be reused in AKS.
- Assign that identity the **Cognitive Services OpenAI User** role on the
Foundry / Azure OpenAI resource.
- AKS later: enable Workload Identity, add a federated credential on the same
identity for the pod's ServiceAccount, annotate the ServiceAccount with the
client ID, and label the pod `azure.workload.identity/use: "true"`.

## Docs

- `docs/user-guide.md`:
- Add `azure` to the provider lists.
- Add a short "Azure Foundry (Entra ID)" subsection covering config, the
role assignment, `AZURE_CLIENT_ID`, `AZURE_TOKEN_CREDENTIALS` and local
`az login`.
- Add the Azure env vars to the env table.
- Note that `/ready` has no key check for `azure`.

## Tests

- `tests/config.test.ts`: `provider: azure` is accepted with `baseUrl` and
rejected without it.
- `tests/model.test.ts`:
- Mock `@azure/identity` with a fake credential that returns a fixed token.
- Assert that `buildModel` returns an Azure model.
- Assert that a call through a stub `fetch` sends
`Authorization: Bearer <token>`, sends no `api-key` header, and uses the
configured `baseUrl`.
- Assert that `requiredEnvVarFor('azure')` is `null`.
- `tests/server.test.ts`: `/ready` returns 200 for `azure` with no API-key
env vars set.

## Out of scope / follow-ups

- **MCP over HTTP with Entra.** HTTP MCP servers behind Entra will need a
per-request bearer token in `mcpTools[].headers`. When needed, reuse the
same `DefaultAzureCredential` with a per-server scope
(e.g. `api://<app-id>/.default`). Specify this separately.
- **Non-OpenAI Foundry models** (Mistral, Llama, …) need Chat Completions
(`azure.chat(...)`) instead of Responses. Add an option when such a model
is actually used.
- **Claude on Foundry** is not covered by `@ai-sdk/azure`.
- **API-key auth for `azure`.** Not needed while key auth is disabled.
46 changes: 39 additions & 7 deletions docs/user-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ introduction and install steps, see the [project README](../README.md).
- [Running](#running)
- [Configuration](#configuration)
- [Model and providers](#model-and-providers)
- [Azure Foundry (Entra ID)](#azure-foundry-entra-id)
- [Prompt templating](#prompt-templating)
- [Tools](#tools)
- [Structured output](#structured-output)
Expand All @@ -25,7 +26,7 @@ introduction and install steps, see the [project README](../README.md).
## Features

- **One-file configuration** — prompt, model, tools, and safety knobs in YAML.
- **Any major provider** — Anthropic, OpenAI, Google, or any OpenAI-compatible
- **Any major provider** — Anthropic, OpenAI, Azure Foundry, Google, or any OpenAI-compatible
endpoint (including local [ollama](https://ollama.com)).
- **External tools via MCP** — connect any Model Context Protocol server over
stdio or HTTP; the agent discovers and uses its tools.
Expand Down Expand Up @@ -73,7 +74,7 @@ systemPrompt: |
You are a helpful assistant...

model:
provider: anthropic # anthropic | openai | google | ollama | openai-compatible
provider: anthropic # anthropic | openai | azure | google | ollama | openai-compatible
name: claude-sonnet-4-6
temperature: 0.2
# topP, maxOutputTokens also supported
Expand Down Expand Up @@ -112,10 +113,11 @@ output:

### Model and providers

Set `model.provider` to one of `anthropic`, `openai`, `google`, `ollama`, or
`openai-compatible`, and `model.name` to a model the provider offers. Each
hosted provider reads its API key from an environment variable (see
[Environment variables](#environment-variables)).
Set `model.provider` to one of `anthropic`, `openai`, `azure`, `google`,
`ollama`, or `openai-compatible`, and `model.name` to a model the provider
offers. Each hosted provider reads its API key from an environment variable
(see [Environment variables](#environment-variables)), except `azure`, which
uses Entra ID (see below).

`model.baseUrl` works for every provider: for `ollama` and `openai-compatible`
it points at your endpoint (these often need no API key); for the hosted
Expand All @@ -124,6 +126,35 @@ API endpoint. See
[Connecting to a local ollama](#connecting-to-a-local-ollama) for a Kubernetes
tip.

#### Azure Foundry (Entra ID)

`provider: azure` calls OpenAI models deployed in Azure Foundry / Azure
OpenAI and authenticates with Microsoft Entra ID tokens, not API keys. It works
when key auth is disabled on the resource.

```yaml
model:
provider: azure
name: my-gpt-deployment # the deployment name, not the model name
baseUrl: https://<resource>.openai.azure.com/openai # required
```

The token comes from `DefaultAzureCredential` and is refreshed
automatically. It picks the identity from the environment:

- **Azure Container Apps:** the app's managed identity. For a user-assigned
identity, set `AZURE_CLIENT_ID` to its client ID.
- **AKS:** Workload Identity. Federate the identity with the pod's
ServiceAccount and label the pod `azure.workload.identity/use: "true"`. The
webhook injects the env vars.
- **Local:** `az login`.

The identity needs the **Cognitive Services OpenAI User** role on the
resource. In production, set `AZURE_TOKEN_CREDENTIALS=prod` so developer
credentials (Azure CLI and similar) are never tried. `/ready` has no key to
check for `azure`. Use `/ready?deep=1` to verify the token, endpoint and
deployment.

### Prompt templating

`systemPrompt` is a [Handlebars](https://handlebarsjs.com) template. These
Expand Down Expand Up @@ -244,7 +275,7 @@ run — see [Resolving a tool approval](#resolving-a-tool-approval).
| Route | Method | Purpose |
|-------|--------|---------|
| `/health` | GET | Liveness — returns 200 as soon as the process is up. |
| `/ready` | GET | Readiness — 200 when the config is loaded and the provider key is present. Add `?deep=1` to also make one tiny provider call that verifies the credentials, URL, and model actually work (503 on failure; the provider error appears in the server logs, not the response). Probe results are cached briefly, so frequent polling can't hammer the provider. |
| `/ready` | GET | Readiness — 200 when the config is loaded and the provider key is present (no key check for `azure`). Add `?deep=1` to also make one tiny provider call that verifies the credentials, URL, and model actually work (503 on failure; the provider error appears in the server logs, not the response). Probe results are cached briefly, so frequent polling can't hammer the provider. |
| `/invoke` | POST | Run the agent and stream the result over SSE. Bodies larger than the configured limit are rejected with 413. |

### Request format
Expand Down Expand Up @@ -399,6 +430,7 @@ Make sure ollama listens on all interfaces (`OLLAMA_HOST=0.0.0.0`).
| `PORT` | `3000` | HTTP port. |
| `LOG_LEVEL` | `info` | Log level (`trace`…`fatal`, or `silent`). |
| `ANTHROPIC_API_KEY` · `OPENAI_API_KEY` · `GOOGLE_GENERATIVE_AI_API_KEY` | — | Provider credentials, per `model.provider`. |
| `AZURE_CLIENT_ID` · `AZURE_TOKEN_CREDENTIALS` | — | Entra ID for `provider: azure`: user-assigned managed identity client ID; `prod` to skip developer credentials. See [Azure Foundry (Entra ID)](#azure-foundry-entra-id). |
| `OLLAMA_BASE_URL` · `OPENAI_BASE_URL` | provider default | Base URL for `ollama` / `openai-compatible` when `model.baseUrl` is unset. |
| `TOOL_APPROVAL_SECRET` | — | Signs tool-approval requests. Required (server refuses to start without it) whenever approval is enabled. |
| `MAX_REQUEST_BODY_BYTES` | `10485760` (10 MiB) | Maximum `/invoke` request body size; larger bodies get 413. |
Expand Down
2 changes: 2 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,14 @@
},
"dependencies": {
"@ai-sdk/anthropic": "^4.0.11",
"@ai-sdk/azure": "^4.0.11",
"@ai-sdk/google": "^4.0.11",
"@ai-sdk/mcp": "^2.0.10",
"@ai-sdk/openai": "^4.0.11",
"@ai-sdk/openai-compatible": "^3.0.7",
"@ai-sdk/otel": "^1.0.19",
"@ai-sdk/provider": "4.0.3",
"@azure/identity": "^4.14.0",
"@hono/node-server": "^2.0.8",
"@opentelemetry/api": "^1.9.1",
"@opentelemetry/auto-instrumentations-node": "^0.78.0",
Expand Down
Loading
Loading