Stop requesting Observability API permissions for blueprint agents by default

CHANGELOG.md

@@ -22,6 +22,8 @@ Agents provisioned before this release need `Agent365.Observability.OtelWrite` g
 
 **Option B — CLI** (`a365 setup admin`) has been removed in this release. Use Option A above, or copy the PowerShell instructions printed in the `a365 setup all` summary output.
 
+Blueprint agents that export telemetry through the app-only S2S endpoint do not need these permissions: `a365 setup all` no longer requests them in the default auth mode, and agent registration authorizes the agent instead. Agents that still export through the delegated (OBO) route can request them with `--authmode both` (#501).
+
 ### Added
 - Setup and bootstrap now use Microsoft's first-party Agent 365 CLI application when it is present in your tenant, validating it without changing Microsoft's app registration, and fall back to a tenant-owned "Agent 365 CLI" app when it is not (#489).
 - Log separator written at the start of each CLI invocation now redacts values for secret-bearing options (e.g. `--idp-client-secret`) so they are not written to the log file in plain text.
@@ -59,6 +61,7 @@ Agents provisioned before this release need `Agent365.Observability.OtelWrite` g
 - `a365 develop get-token --device-code` — forces device code auth for Microsoft Graph scopes the Windows WAM broker rejects (e.g. Exchange `MailboxSettings.ReadWrite`, `ExchangeMessageTrace.Read.All`).
 
 ### Fixed
+- `a365 setup all --agent-registration-only` now exits with code 1 when agent registration fails (#501).
 - Setup no longer fails to detect the Agent 365 CLI application in tenants where it is not yet provisioned, and reports lookup errors instead of silently switching your configured client app (#489).
 - The first-party Agent 365 CLI app now uses device code authentication when Windows Account Manager is unavailable, avoiding unsupported browser-response errors in WSL, macOS, and Linux (#489).
 - `setup all --authmode s2s` no longer prints spurious "Action Required" PowerShell steps when the agent identity already inherits its app roles from the blueprint, and now retries the grant automatically before falling back to manual steps (#460).
@@ -104,6 +107,7 @@ Agents provisioned before this release need `Agent365.Observability.OtelWrite` g
 
 ### Changed
 
+- `a365 setup all` no longer requests Observability API permissions for blueprint agents in the default auth mode, so registered agents export telemetry through the app-only S2S endpoint without admin consent (#501).
 - Hardened token storage: the CLI no longer writes access tokens to a plaintext file — they live only in the OS-protected MSAL cache (DPAPI/Keychain/owner-only file). Any legacy plaintext cache is removed automatically; sign-in prompts are unchanged.
 - `develop-mcp register-external-mcp-server` now sets `exit code 1` on failure paths (validation errors, tenant detection failure, Graph unavailable, Entra app creation failure, MCP-Platform AddMcpServer failure). Previously these paths logged an error and exited `0`, which made the command's success/failure status undetectable from scripts and CI. Successful dry-run and user-initiated cancellation at the y/N prompt continue to exit `0`.
 - Admin consent canary path (when the caller lacks `DelegatedPermissionGrant.Read.All`) no longer prompts for Enter immediately. The CLI now polls every 5 seconds, prints a friendly progress message at 30 seconds, and responds promptly to Enter or Ctrl+C. The previous jargon-heavy message about `oauth2PermissionGrants` was rewritten in plain English; technical details are demoted to `Debug`.

docs/agent365-guided-setup/a365-observability-instructions.md

@@ -775,6 +775,8 @@ This skill is safe to rerun. On subsequent runs:
 
 ### OtelWrite App Role Assignment
 
+> **Blueprint agents (default):** `a365 setup all` does not request Observability API permissions for blueprint agents in the default auth mode — the S2S endpoint authorizes registered agent instances without the `OtelWrite` role, so no admin consent is needed. Setup exits with code 1 if registration fails; retry with `a365 setup all --agent-registration-only`. Use `--authmode both` if the agent still exports through the delegated (OBO) route. Permissions granted by earlier runs are not revoked.
+
 `a365 setup all` **attempts** to grant `Agent365.Observability.OtelWrite` to the Agent Identity SP, but this requires **Global Administrator** privileges. If the logged-in user is not a Global Admin, the assignment silently fails with 403 and trace exports will return HTTP 403 from the observability service.
 
 **The CLI prints a PowerShell admin consent script** in its output when the assignment fails. When running `a365 setup all`, **always scan the output for this script block** and display it to the user in a fenced code block so they can copy it and hand it to a Global Admin.

src/Microsoft.Agents.A365.DevTools.Cli/Commands/SetupSubcommands/AllSubcommand.cs

@@ -397,13 +397,18 @@ effectiveAuthModeForValidation is not ("obo" or "s2s" or "both"))
                 return;
             }
 
+            // Registered blueprint agents export telemetry app-only over S2S, so only the app-role modes
+            // (s2s/both) still request OtelWrite; AI Teammate setup is unchanged.
+            var skipObservabilityPermissions = nonDwConfig is not null
+                && effectiveAuthModeForValidation is not ("s2s" or "both");
+
             if (nonDwConfig is not null)
             {
                 if (dryRun)
                 {
                     var rawArgs = context.ParseResult.Tokens.Select(t => t.Value).ToArray();
                     var effectiveAuthMode = authMode ?? nonDwConfig.AuthMode;
-                    NonDwBlueprintSetupOrchestrator.PrintDryRunPlan(nonDwConfig, logger, isBootstrap, rawArgs, skipRequirements, isM365, agentRegistrationOnly, effectiveAuthMode, messagingEndpointFlag);
+                    NonDwBlueprintSetupOrchestrator.PrintDryRunPlan(nonDwConfig, logger, isBootstrap, rawArgs, skipRequirements, isM365, agentRegistrationOnly, effectiveAuthMode, messagingEndpointFlag, skipObservabilityPermissions);
                     return;
                 }
 
@@ -442,7 +447,8 @@ effectiveAuthModeForValidation is not ("obo" or "s2s" or "both"))
                     confirmationProvider: confirmationProvider,
                     skipSpProvisioning: skipSpProvisioning,
                     messagingEndpointOverride: messagingEndpointFlag,
-                    nonInteractive: Console.IsInputRedirected);
+                    nonInteractive: Console.IsInputRedirected,
+                    skipObservabilityPermissions: skipObservabilityPermissions);
 
                 context.ExitCode = await NonDwBlueprintSetupOrchestrator.ExecuteAsync(nonDwCtx);
                 return;
@@ -1018,7 +1024,8 @@ await PermissionsSubcommand.RemoveStaleCustomPermissionsAsync(
         // for both DW and non-DW agents; serverNamesByAudience drives the per-server display
         // names so V2 audiences read as e.g. "mcp_MailTools" rather than "Agent 365 Tools".
         var specs = await SetupHelpers.BuildConfiguredPermissionSpecsAsync(
-            ctx.Config, setInheritable: true, isM365: ctx.IsM365, scopesByAudience, serverNamesByAudience);
+            ctx.Config, setInheritable: true, isM365: ctx.IsM365, scopesByAudience, serverNamesByAudience,
+            includeObservability: !ctx.SkipObservabilityPermissions);
 
         // Return the full scopesByAudience map alongside the V1-compat mcpScopes so V2
         // callers (ApplyConsentUrlsIfNeeded) can route per-server audiences to the bare

src/Microsoft.Agents.A365.DevTools.Cli/Commands/SetupSubcommands/NonDwBlueprintSetupOrchestrator.cs

@@ -20,8 +20,8 @@ namespace Microsoft.Agents.A365.DevTools.Cli.Commands.SetupSubcommands;
 ///   1. Requirements validation
 ///   2. Blueprint creation (shared with DW)
 ///   3. Batch permissions on the blueprint (shared with DW pipeline; non-DW spec set:
-///      Observability API, Power Platform API, custom). MAC reads from the blueprint,
-///      so stamping here gives the same set visibility there.
+///      Power Platform API, custom, and Observability API only for authMode s2s/both). MAC reads
+///      from the blueprint, so stamping here gives the same set visibility there.
 ///   4. Agent Identity creation via POST /beta/servicePrincipals/Microsoft.Graph.AgentIdentity
 ///   5. Agent Identity permission grants (same spec set as step 3) — OBO or S2S
 ///   6. Agent registration via Graph API (copilot/agentRegistrations)
@@ -33,7 +33,7 @@ internal static class NonDwBlueprintSetupOrchestrator
     /// Prints a dry-run plan showing all resources that would be created or configured,
     /// using actual names and values from the loaded config. Makes no API calls.
     /// </summary>
-    public static void PrintDryRunPlan(Agent365Config config, ILogger logger, bool isBootstrap = false, string[]? rawArgs = null, bool skipRequirements = false, bool isM365 = false, bool agentRegistrationOnly = false, string? authMode = null, string? messagingEndpointOverride = null)
+    public static void PrintDryRunPlan(Agent365Config config, ILogger logger, bool isBootstrap = false, string[]? rawArgs = null, bool skipRequirements = false, bool isM365 = false, bool agentRegistrationOnly = false, string? authMode = null, string? messagingEndpointOverride = null, bool skipObservabilityPermissions = false)
     {
         var sub = new string(' ', SetupHelpers.DryRunValCol);
         // --messaging-endpoint flag (if supplied) wins over the init-only config value for the plan.
@@ -117,14 +117,17 @@ public static void PrintDryRunPlan(Agent365Config config, ILogger logger, bool i
             logger.LogInformation(sub + "create managed identity");
         }
 
-        // 3. Inheritable Permissions — non-DW spec set (Observability API, Power Platform API, custom)
-        //    stamped on the blueprint via SetInheritablePermissionsAsync so MAC and other dependent
-        //    systems can see them. The same set is applied to the agent identity SP in step 5.
+        // 3. Inheritable Permissions — non-DW spec set (Power Platform API, custom, plus Observability API
+        //    only for authMode s2s/both) stamped on the blueprint via SetInheritablePermissionsAsync so MAC
+        //    and other dependent systems can see them. The same set is applied to the agent identity SP in step 5.
         var selectedAuthMode = authMode ?? config.AuthMode;
         var effectiveMode = string.IsNullOrWhiteSpace(selectedAuthMode)
             ? "obo"
             : selectedAuthMode.Trim().ToLowerInvariant();
-        logger.LogInformation(SetupHelpers.DryRunRow(3, "Inheritable Permissions") + "configure for Observability API, Power Platform API, and custom permissions (Global Administrator required; consent URL printed if absent)");
+        logger.LogInformation(SetupHelpers.DryRunRow(3, "Inheritable Permissions") + "configure for {Resources} (Global Administrator required; consent URL printed if absent)",
+            skipObservabilityPermissions ? "Power Platform API and custom permissions" : "Observability API, Power Platform API, and custom permissions");
+        if (skipObservabilityPermissions)
+            logger.LogInformation(sub + "Observability API not requested (registered agents export telemetry with an app-only token)");
 
         // 4. Blueprint Permission Grants — per authMode. The consent URL targets the blueprint
         //    app, and S2S app-role assignments are persisted as grants flowing from the blueprint;
@@ -266,6 +269,7 @@ await ctx.ClientAppValidator.GrantConsentForPermissionsAsync(
     public static async Task<int> ExecuteAsync(SetupContext ctx)
     {
         ctx.Results.IsNonDwBlueprintFlow = true;
+        ctx.Results.ObservabilityPermissionsSkipped = ctx.SkipObservabilityPermissions;
         ctx.Results.TenantId = ctx.Config.TenantId;
         // Bootstrap already printed the "Running..." banner before auth steps; skip here to avoid duplication.
         if (!ctx.IsBootstrap)
@@ -364,6 +368,8 @@ public static async Task<int> ExecuteAsync(SetupContext ctx)
 
                 // Step 4: Build permission specs — stamps Graph, manifest MCP audiences, Observability,
                 // Power Platform, custom permissions, and Messaging Bot (only when isM365). Mirrors DW.
+                if (ctx.SkipObservabilityPermissions)
+                    ctx.Logger.LogInformation("Observability API permissions not requested: registered agents export telemetry with an app-only token.");
                 var buildResult = await AllSubcommand.BuildPermissionSpecsAsync(ctx);
                 specs = buildResult.specs;
 
@@ -435,7 +441,7 @@ await AllSubcommand.ExecuteBatchPermissionsStepAsync(
     /// When <paramref name="skipIdentityAndPermissions"/> is true (--agent-registration-only),
     /// identity creation and permission grants are skipped — only registration and project settings run.
     /// </summary>
-    private static async Task ExecuteAgentIdentityAndRegistrationAsync(
+    internal static async Task ExecuteAgentIdentityAndRegistrationAsync(
         SetupContext ctx,
         List<ResourcePermissionSpec> specs,
         bool skipIdentityAndPermissions = false)
@@ -549,15 +555,23 @@ private static async Task ExecuteAgentIdentityAndRegistrationAsync(
         ctx.Logger.LogInformation("");
         ctx.Logger.LogInformation("Registering agent...");
 
+        // Registration is the sole purpose of --agent-registration-only and, with OtelWrite skipped,
+        // the agent's only Observability authorization, so its failure must fail setup.
+        var registrationRequired = skipIdentityAndPermissions || ctx.SkipObservabilityPermissions;
+        void RecordRegistrationFailure(string message)
+        {
+            ctx.Results.AgentRegistrationFailed = true;
+            (registrationRequired ? ctx.Results.Errors : ctx.Results.Warnings).Add(message);
+            ctx.Logger.Log(registrationRequired ? LogLevel.Error : LogLevel.Warning, message);
+        }
+
         if (string.IsNullOrWhiteSpace(ctx.Config.AgenticAppId))
         {
             var registrationSkippedMessage =
                 "Agent registration failed: agent identity ID is not available. " +
                 "Ensure the agent identity was created successfully, then retry with: a365 setup all --agent-registration-only";
-            ctx.Results.Warnings.Add(registrationSkippedMessage);
             using (ctx.Logger.Indent())
-                ctx.Logger.LogWarning(registrationSkippedMessage);
-            ctx.Results.AgentRegistrationFailed = true;
+                RecordRegistrationFailure(registrationSkippedMessage);
         }
         else
         {
@@ -633,9 +647,7 @@ private static async Task ExecuteAgentIdentityAndRegistrationAsync(
         }
         else
         {
-            ctx.Results.AgentRegistrationFailed = true;
-            ctx.Results.Warnings.Add("Agent registration failed via Graph copilot/agentRegistrations API.");
-            ctx.Logger.LogWarning("Agent registration failed via Graph copilot/agentRegistrations API.");
+            RecordRegistrationFailure("Agent registration failed via Graph copilot/agentRegistrations API.");
         }
 
         } // end else (AgenticAppId present)

src/Microsoft.Agents.A365.DevTools.Cli/Commands/SetupSubcommands/README.md

@@ -92,6 +92,12 @@ a365 setup all --authmode s2s
 a365 setup all --authmode both
 ```
 
+### Observability permissions
+
+For blueprint agents in the default `obo` auth mode, `setup all` does not request `Agent365.Observability.OtelWrite`. Registered agents export telemetry with an app-only token through the S2S endpoint, which authorizes them by their agent registration, so no Observability admin consent is needed. Registration is then the agent's only authorization, so a registration failure is reported as an error (exit code 1).
+
+`--authmode s2s|both` still request `OtelWrite` (the app role, plus the delegated scope with `both`); use `both` for agents whose SDK still exports through the delegated (OBO) route. AI Teammate setup is unchanged. Re-running setup does not revoke permissions granted earlier.
+
 ---
 
 ### Messaging endpoint (M365 agents)

src/Microsoft.Agents.A365.DevTools.Cli/Commands/SetupSubcommands/SetupContext.cs

@@ -100,6 +100,12 @@ internal sealed class SetupContext
     /// </summary>
     public bool NonInteractive { get; }
 
+    /// <summary>
+    /// When true, Observability API permissions are omitted from every grant and consent URL (non-DW blueprint only),
+    /// making agent registration the agent's only Observability authorization.
+    /// </summary>
+    public bool SkipObservabilityPermissions { get; }
+
     /// <summary>
     /// Overrides the az CLI login hint resolver used during blueprint creation.
     /// Null in production — injected as a no-op in tests to avoid spawning 'az account show'.
@@ -154,7 +160,8 @@ public SetupContext(
         IConfirmationProvider? confirmationProvider = null,
         bool skipSpProvisioning = false,
         string? messagingEndpointOverride = null,
-        bool nonInteractive = false)
+        bool nonInteractive = false,
+        bool skipObservabilityPermissions = false)
     {
         Config = config;
         Results = results;
@@ -172,6 +179,7 @@ public SetupContext(
         MessagingEndpointOverride = string.IsNullOrWhiteSpace(messagingEndpointOverride) ? null : messagingEndpointOverride.Trim();
         SkipSpProvisioning = skipSpProvisioning;
         NonInteractive = nonInteractive;
+        SkipObservabilityPermissions = skipObservabilityPermissions;
         ConfigService = configService;
         Executor = executor;
         BackendConfigurator = backendConfigurator;