From f69bb3b1177723ec017f72fa17aa21a33082dfa6 Mon Sep 17 00:00:00 2001 From: Roland Krummenacher Date: Mon, 17 Aug 2026 09:04:47 +0200 Subject: [PATCH] fix(hubs): assign requested storage roles to apps that don't create storage Managed exports never created any Cost Management exports. Every export creation failed with an Unauthorized error because Cost Management needs the caller to write role assignments on the destination storage account (exports are created with a system-assigned identity that must be granted access to the destination). The Managed Exports app requests the Role Based Access Control Administrator role via the storageRoles parameter of fx/hub-app.bicep, but the role assignment loop was gated on the "Storage" feature. Apps only declare that feature when they create the publisher storage account, and Managed Exports uses the account created by Microsoft.CostManagement.Exports, so every role it requested was silently dropped at build time. Roles requested via storageRoles are now assigned whenever the app uses Data Factory, since the storage account is shared across all apps from the same publisher. The base data management roles (Storage Account Contributor, Storage Blob Data Contributor, Reader) stay tied to the "Storage" feature, so no other app changes. Also corrects the user-facing role name, which has referred to the removed User Access Administrator role since it was replaced by RBAC Administrator in v13. Fixes #2253 Co-Authored-By: Claude Opus 5 (1M context) --- docs-mslearn/toolkit/changelog.md | 3 +- docs-mslearn/toolkit/hubs/template.md | 6 +- .../Tests/Unit/HubsAppStorageRoles.Tests.ps1 | 116 ++++++++++++++++++ .../finops-hub/createUiDefinition.json | 2 +- src/templates/finops-hub/main.bicep | 2 +- .../finops-hub/modules/fx/hub-app.bicep | 8 +- src/templates/finops-hub/modules/hub.bicep | 2 +- 7 files changed, 129 insertions(+), 10 deletions(-) create mode 100644 src/powershell/Tests/Unit/HubsAppStorageRoles.Tests.ps1 diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 50bba2964..2457e1819 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -3,7 +3,7 @@ title: FinOps toolkit changelog description: Review the latest features and enhancements in the FinOps toolkit, including updates to FinOps hubs, Power BI reports, and more. author: MSBrett ms.author: brettwil -ms.date: 08/13/2026 +ms.date: 08/17/2026 ms.topic: reference ms.service: finops ms.subservice: finops-toolkit @@ -33,6 +33,7 @@ The following section lists features and enhancements that are currently in deve - Replaced redundant `tolower()` comparisons in hub KQL with case-insensitive operators (`has`, `=~`, `!~`) so the engine can use the term index instead of scanning every row ([#2213](https://github.com/microsoft/finops-toolkit/issues/2213)). - Replaced whole-term `contains` matches with `has` across hub KQL and the query catalog (resource ID paths, licensing phrases, SKU description terms) and added a per-row operator-equivalence regression harness with unit test coverage ([#2220](https://github.com/microsoft/finops-toolkit/pull/2220)). - **Fixed** + - Fixed managed exports failing with an `Unauthorized` error because the Role Based Access Control Administrator role was never assigned to the Data Factory identity. Roles requested by a hub app are now assigned on the publisher storage account even when the app doesn't create the storage account itself ([#2253](https://github.com/microsoft/finops-toolkit/issues/2253)). - Fixed the `ContractedCost` recompute guard to compare with a null-safe tolerance instead of exact float equality, eliminating millions of no-op rewrites that polluted the `x_SourceValues` audit trail while preserving the null-cost backfill and no longer overwriting an existing cost when the unit price is missing ([#2216](https://github.com/microsoft/finops-toolkit/issues/2216)). - Fixed the SQL VMs without Azure Hybrid Benefit recommendation query to join on the SQL VM `virtualMachineResourceId` instead of a case-sensitive VM name match that skipped VMs with uppercase names and dropped duplicate names, and made all Azure Resource Graph join kinds explicit so no query relies on the `innerunique` default ([#2225](https://github.com/microsoft/finops-toolkit/pull/2225)). - Switched dimension enrichment in the v1_0/v1_2 ingestion transforms (`PricingUnits`, `Regions`, `ResourceTypes`, `Services`) from `join` to the broadcast-optimized `lookup` operator and deduplicated the `Services` mapping per resource type to prevent cost row fan-out ([#2225](https://github.com/microsoft/finops-toolkit/pull/2225)). diff --git a/docs-mslearn/toolkit/hubs/template.md b/docs-mslearn/toolkit/hubs/template.md index 3073275b1..92c1a31d1 100644 --- a/docs-mslearn/toolkit/hubs/template.md +++ b/docs-mslearn/toolkit/hubs/template.md @@ -3,7 +3,7 @@ title: FinOps hub template description: Learn about what's included in the FinOps hub template including parameters, resources, and outputs. author: flanakin ms.author: micflan -ms.date: 06/03/2026 +ms.date: 08/17/2026 ms.topic: concept-article ms.service: finops ms.subservice: finops-toolkit @@ -65,7 +65,7 @@ Ensure the following prerequisites are met before you deploy the template: | [Storage Account Contributor](/azure/role-based-access-control/built-in-roles#storage-account-contributor) | Assigned to Data Factory to manage data in storage. | | [Storage Blob Data Contributor](/azure/role-based-access-control/built-in-roles#storage-blob-data-contributor) | Assigned to Data Factory and Data Explorer to manage data in storage. | | [Storage File Data Privileged Contributor](/azure/role-based-access-control/built-in-roles/storage#storage-file-data-privileged-contributor) | Assigned to the deployment file upload identity that uploads files to the config container. | - | [User Access Administrator](/azure/role-based-access-control/built-in-roles#user-access-administrator) | Assigned to Data Factory to manage data in storage. Not applied when **enableManagedExports** is disabled. | + | [Role Based Access Control Administrator](/azure/role-based-access-control/built-in-roles/privileged#role-based-access-control-administrator) | Assigned to Data Factory on the storage account so it can create Cost Management exports, which grant their own identity access to the export destination. Not applied when **enableManagedExports** is disabled. | - The Microsoft.EventGrid resource provider must be registered in your subscription. For more information, see [Register a resource provider](/azure/azure-resource-manager/management/resource-providers-and-types#register-resource-provider). @@ -97,7 +97,7 @@ Here are the parameters you can use to customize the deployment: | **dataExplorerFinalRetentionInMonths** | Int | Optional. Number of months of data to retain in the Data Explorer \*_final_v\* tables. | 13 | | **remoteHubStorageUri** | String | Optional. Data Lake storage endpoint from the remote (primary) hub storage account. Used for cross-tenant cost data collection where this hub sends processed data to a central hub. Example: `https://primaryhub.dfs.core.windows.net/` | | | **remoteHubStorageKey** | String | Optional. Storage account access key for the remote (primary) hub. Used with remoteHubStorageUri for cross-tenant scenarios. Must be kept secure as it provides full storage access. | | -| **enableManagedExports** | Bool | Optional. Enable managed exports where your FinOps hub instance will create and run Cost Management exports on your behalf. Not supported for Microsoft Customer Agreement (MCA) billing profiles. Requires the ability to grant User Access Administrator role to FinOps hubs, which is required to create Cost Management exports. | True | +| **enableManagedExports** | Bool | Optional. Enable managed exports where your FinOps hub instance will create and run Cost Management exports on your behalf. Not supported for Microsoft Customer Agreement (MCA) billing profiles. Requires the ability to grant the Role Based Access Control Administrator role to FinOps hubs, which is required to create Cost Management exports. | True | | **enableRecommendations** | Bool | Optional. Enable recommendations ingested from Azure Resource Graph based on configurable queries. The Data Factory managed identity requires Reader role on management groups or subscriptions to execute Resource Graph queries. | False | | **enableAHBRecommendations** | Bool | Optional. Enable Azure Hybrid Benefit recommendations that flag VMs and SQL VMs without Azure Hybrid Benefit enabled. May generate noise if your organization does not have on-premises licenses. Requires enableRecommendations. | False | | **enableSpotRecommendations** | Bool | Optional. Enable non-Spot AKS cluster recommendations that flag AKS clusters with autoscaling but not using Spot VMs. May generate noise since Spot VMs are only appropriate for interruptible workloads. Requires enableRecommendations. | False | diff --git a/src/powershell/Tests/Unit/HubsAppStorageRoles.Tests.ps1 b/src/powershell/Tests/Unit/HubsAppStorageRoles.Tests.ps1 new file mode 100644 index 000000000..ac6886465 --- /dev/null +++ b/src/powershell/Tests/Unit/HubsAppStorageRoles.Tests.ps1 @@ -0,0 +1,116 @@ +# Copyright (c) Microsoft Corporation. +# Licensed under the MIT License. + +<# + Regression coverage for hub app storage role assignments (issue #2253): + + Hub apps declare the roles their Data Factory identity needs on the publisher storage account + via the storageRoles parameter of fx/hub-app.bicep. Managed exports requests Role Based Access + Control Administrator there, because Cost Management exports are created with a system-assigned + identity that Cost Management must grant access to the export destination. Without it, every + export creation fails with: + {"error":{"code":"Unauthorized","message":"The user does not have authorization to perform + 'Microsoft.Authorization/roleAssignments/write' action on specified storage account, ..."}} + + The role assignment loop used to be gated on the "Storage" feature, which apps only declare when + they create the publisher storage account. Managed exports uses the storage account created by + Microsoft.CostManagement.Exports, so it doesn't declare the feature, and every role it requested + was silently dropped at build time. + + These tests parse the app registrations and fx/hub-app.bicep and verify: + 1. Roles requested via storageRoles always flow into the assignment loop. + 2. The assignment loop isn't gated on the "Storage" feature. + 3. Every app that requests storage roles declares the "DataFactory" feature, since the roles are + assigned to the Data Factory identity and are otherwise dropped. + 4. Managed exports still requests Role Based Access Control Administrator. +#> + +Describe 'HubsAppStorageRoles' { + + BeforeDiscovery { + $repoRoot = (Resolve-Path "$PSScriptRoot/../../../..").Path + $modulesPath = Join-Path $repoRoot 'src/templates/finops-hub/modules' + + # Extract the features and storageRoles passed to fx/hub-app.bicep from every app registration. + $appRegistrations = @(Get-ChildItem -Path $modulesPath -Recurse -Filter 'app.bicep' -File | ForEach-Object { + $content = Get-Content -Path $_.FullName -Raw + if ($content -notmatch 'hub-app\.bicep') { return } + + $getValues = { + param([string]$Name) + $match = [regex]::Match($content, "(?ms)^\s*$Name\s*:\s*\[(.*?)^\s*\]") + if (-not $match.Success) { return @() } + return @([regex]::Matches($match.Groups[1].Value, "'([^']+)'") | ForEach-Object { $_.Groups[1].Value }) + } + + @{ + Name = (Split-Path -Path $_.DirectoryName -Leaf) + Path = $_.FullName + Features = @(& $getValues 'features') + StorageRoles = @(& $getValues 'storageRoles') + } + }) + + $appsWithStorageRoles = @($appRegistrations | Where-Object { $_.StorageRoles.Count -gt 0 }) + } + + BeforeAll { + $repoRoot = (Resolve-Path "$PSScriptRoot/../../../..").Path + $hubAppPath = Join-Path $repoRoot 'src/templates/finops-hub/modules/fx/hub-app.bicep' + $hubAppContent = Get-Content -Path $hubAppPath -Raw + + # Role definition ID for Role Based Access Control Administrator + # https://learn.microsoft.com/azure/role-based-access-control/built-in-roles/privileged#role-based-access-control-administrator + $rbacAdministratorRoleId = 'f58310d9-a9f6-439a-9e8d-f62e7b41a168' + + $managedExportsPath = Join-Path $repoRoot 'src/templates/finops-hub/modules/Microsoft.CostManagement/ManagedExports/app.bicep' + $managedExportsContent = Get-Content -Path $managedExportsPath -Raw + + # The variable the role assignment loop iterates over + $factoryStorageRoles = [regex]::Match($hubAppContent, '(?ms)^var factoryStorageRoles\s*=.*?^\]\)').Value + + # The condition on the storage role assignment loop + $storageRoleAssignmentCondition = [regex]::Match($hubAppContent, "(?ms)resource storageRoleAssignments\s+'Microsoft\.Authorization/roleAssignments@[^']+'\s*=\s*\[\s*for\s+\w+\s+in\s+(?\w+)\s*:\s*if\s*\((?[^)]*(?:\([^)]*\)[^)]*)*)\)") + } + + Context 'fx/hub-app.bicep' { + + It 'Should iterate over a role list that always includes the requested storageRoles' { + $factoryStorageRoles | Should -Match 'union\(\s*storageRoles\s*,' + } + + It 'Should assign storage roles without requiring the Storage feature' { + $storageRoleAssignmentCondition.Success | Should -BeTrue -Because 'the storage role assignment loop should be parseable' + $storageRoleAssignmentCondition.Groups['roles'].Value | Should -Be 'factoryStorageRoles' + $storageRoleAssignmentCondition.Groups['condition'].Value | Should -Not -Match 'usesStorage' -Because 'apps that request storage roles do not necessarily create the publisher storage account (issue #2253)' + } + + It 'Should still require the Data Factory feature to assign storage roles' { + $storageRoleAssignmentCondition.Groups['condition'].Value | Should -Match 'usesDataFactory' -Because 'the roles are assigned to the Data Factory identity' + } + } + + Context 'App registrations' { + + It 'Should find at least one app that requests storage roles' -ForEach @{ Count = 0 } { + # Recomputed here so the assertion runs even if discovery found nothing + $repoRoot = (Resolve-Path "$PSScriptRoot/../../../..").Path + $modulesPath = Join-Path $repoRoot 'src/templates/finops-hub/modules' + $requesting = @(Get-ChildItem -Path $modulesPath -Recurse -Filter 'app.bicep' -File | Where-Object { + (Get-Content -Path $_.FullName -Raw) -match '(?ms)hub-app\.bicep.*^\s*storageRoles\s*:\s*\[\s*\r?\n\s*[^\]]' + }) + $requesting.Count | Should -BeGreaterThan 0 + } + + It 'Should declare the DataFactory feature when requesting storage roles: ' -ForEach $appsWithStorageRoles { + $Features | Should -Contain 'DataFactory' -Because 'storage roles are assigned to the Data Factory identity and are dropped when the app has no Data Factory' + } + } + + Context 'Managed exports' { + + It 'Should request the Role Based Access Control Administrator role' { + $managedExportsContent | Should -Match $rbacAdministratorRoleId -Because 'Cost Management requires the caller to grant the export identity access to the destination storage account' + } + } +} diff --git a/src/templates/finops-hub/createUiDefinition.json b/src/templates/finops-hub/createUiDefinition.json index d7ea79097..e9d7fb851 100644 --- a/src/templates/finops-hub/createUiDefinition.json +++ b/src/templates/finops-hub/createUiDefinition.json @@ -895,7 +895,7 @@ "name": "enableManagedExports", "type": "Microsoft.Common.CheckBox", "label": "Enable managed exports", - "toolTip": "Creating exports in Cost Management requires the User Access Administrator role. You must have access to grant User Access Administrator to your FinOps hub to enable managed exports." + "toolTip": "Creating exports in Cost Management requires the ability to grant access to the export destination. You must have access to grant the Role Based Access Control Administrator role to your FinOps hub to enable managed exports." } ], "visible": true diff --git a/src/templates/finops-hub/main.bicep b/src/templates/finops-hub/main.bicep index 687c26174..7b556ba00 100644 --- a/src/templates/finops-hub/main.bicep +++ b/src/templates/finops-hub/main.bicep @@ -36,7 +36,7 @@ param remoteHubStorageUri string = '' @secure() param remoteHubStorageKey string = '' -@description('Optional. Enable managed exports where your FinOps hub instance will create and run Cost Management exports on your behalf. Not supported for Microsoft Customer Agreement (MCA) billing profiles. Requires the ability to grant User Access Administrator role to FinOps hubs, which is required to create Cost Management exports. Default: true.') +@description('Optional. Enable managed exports where your FinOps hub instance will create and run Cost Management exports on your behalf. Not supported for Microsoft Customer Agreement (MCA) billing profiles. Requires the ability to grant the Role Based Access Control Administrator role to FinOps hubs, which is required to create Cost Management exports. Default: true.') param enableManagedExports bool = true @description('Optional. Enable recommendations ingested from Azure Resource Graph based on configurable queries. The Data Factory managed identity requires Reader role on management groups or subscriptions to execute Resource Graph queries. Default: false.') diff --git a/src/templates/finops-hub/modules/fx/hub-app.bicep b/src/templates/finops-hub/modules/fx/hub-app.bicep index 6bdd0cf21..b176c05a5 100644 --- a/src/templates/finops-hub/modules/fx/hub-app.bicep +++ b/src/templates/finops-hub/modules/fx/hub-app.bicep @@ -23,7 +23,7 @@ param version string @description('Optional. Indicate which features the app requires. Allowed values: "DataFactory", "KeyVault", "Storage". Default: [] (none).') param features HubAppFeature[] = [] -@description('Optional. Indicate which RBAC roles the Data Factory identity needs on the storage account, if created. This is in addition to Storage Blob Data Contributor for reading and managing content. Default: [] (none).') +@description('Optional. Indicate which RBAC roles the Data Factory identity needs on the publisher storage account. This is in addition to Storage Blob Data Contributor for reading and managing content, which is granted to apps that use the "Storage" feature. Roles are assigned whether or not the app uses the "Storage" feature since the storage account is shared across all apps from the same publisher. Default: [] (none).') param storageRoles string[] = [] @description('Optional. Custom string with additional metadata to log. Must an alphanumeric string without spaces or special characters except for underscores and dashes. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed.') @@ -65,7 +65,8 @@ var factoryManagementRoles = [ // Roles for ADF to manage data in storage // Does not include roles assignments needed against the export scope -var factoryStorageRoles = union(storageRoles, [ +// The data management roles are only granted to apps that use the "Storage" feature; roles requested via storageRoles are always granted +var factoryStorageRoles = union(storageRoles, !usesStorage ? [] : [ // Storage Account Contributor -- https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#storage-account-contributor // Used to move files from the msexports to ingestion container '17d1049b-9a84-46fb-8f53-869881c3d3ab' @@ -284,8 +285,9 @@ module approveStoragePrivateEndpointConnections 'storageEndpoints.bicep' = if (u //------------------------------------------------------------------------------ // Grant ADF identity access to storage +// The storage account is shared across all apps from the same publisher, so apps that request roles via storageRoles are granted them even when they do not create the storage account themselves resource storageRoleAssignments 'Microsoft.Authorization/roleAssignments@2022-04-01' = [ - for role in factoryStorageRoles: if (usesDataFactory && usesStorage) { + for role in factoryStorageRoles: if (usesDataFactory) { name: guid(storageAccount.id, role, dataFactory.id) scope: storageAccount properties: { diff --git a/src/templates/finops-hub/modules/hub.bicep b/src/templates/finops-hub/modules/hub.bicep index 0c9e6b9e5..f682a1829 100644 --- a/src/templates/finops-hub/modules/hub.bicep +++ b/src/templates/finops-hub/modules/hub.bicep @@ -44,7 +44,7 @@ param remoteHubStorageUri string = '' @secure() param remoteHubStorageKey string = '' -@description('Optional. Enable managed exports where your FinOps hub instance will create and run Cost Management exports on your behalf. Not supported for Microsoft Customer Agreement (MCA) billing profiles. Requires the ability to grant User Access Administrator role to FinOps hubs, which is required to create Cost Management exports. Default: true.') +@description('Optional. Enable managed exports where your FinOps hub instance will create and run Cost Management exports on your behalf. Not supported for Microsoft Customer Agreement (MCA) billing profiles. Requires the ability to grant the Role Based Access Control Administrator role to FinOps hubs, which is required to create Cost Management exports. Default: true.') param enableManagedExports bool = true @description('Optional. Enable recommendations ingested from Azure Resource Graph based on configurable queries. The Data Factory managed identity requires Reader role on management groups or subscriptions to execute Resource Graph queries. Default: false.')