diff --git a/docs/scim-provisioning.md b/docs/scim-provisioning.md new file mode 100644 index 00000000..f38af208 --- /dev/null +++ b/docs/scim-provisioning.md @@ -0,0 +1,113 @@ +# SCIM user and group provisioning + +SCIM 2.0 provisions non-administrator gateway accounts and organisation teams. The +API is disabled by default. Unset, empty, `false`, `0`, `off` and `no` values for +`SCIM_ENABLED` return 404 for every `/scim/v2/*` route. An invalid value also disables +the API and emits one warning without the value. Existing account creation is unchanged. + +## Configuration and authentication + +Set `SCIM_ENABLED=true` only after applying the SCIM storage migration and configuring +the account and tenant authorities. `AUTH_STORAGE_BACKEND` and +`INTELLIGENCE_STORAGE_BACKEND` must both be `d1`, and +`CONFIG_REVISION_SYNC_ENABLED=true` must be enabled for the Worker and Container. +Security revisions expire within `CONFIG_SECURITY_TTL_SECONDS` (default and maximum: +five seconds); an unavailable or stale security copy denies generation. + +Configure token references, never token values: + +```json +{"tenants":{"org:example":{"token_ref":"SCIM_PROVISIONING_EXAMPLE"}}} +``` + +This is the value of `SCIM_TRUST_CONFIG_JSON`. The named environment variable or +Worker secret binding supplies the provisioning token privately. Supply each reference +to the Container through the deployment's reviewed secret-binding configuration. +An empty trust config uses `{}` and authorises nobody. Each tenant needs a distinct +token. Missing, unknown, ambiguous or invalid trust entries return a SCIM 401 error. +The server compares SHA-256 digests in constant time, fixes the organisation from the +matched reference, and ignores caller-supplied organisation headers. Provisioning +tokens and account credentials are never returned or logged. + +Send the token in `Authorization: Bearer `. Browser sessions do +not authorise this API. It is exempt from dashboard CSRF checks and accepts +`application/scim+json` or `application/json` request bodies. Responses use +`application/scim+json` and `Cache-Control: no-store`. + +## Endpoints and supported attributes + +The discovery endpoints are `/scim/v2/ServiceProviderConfig`, `/scim/v2/Schemas` +and `/scim/v2/ResourceTypes`; schema and resource-type entries can also be read by ID. +`Users` and `Groups` support list, get, POST, PUT, PATCH and DELETE. PATCH uses the +SCIM PatchOp schema and `Operations` array. Unsupported schemas, attributes, paths, +operations and list parameters return a SCIM 400 error. + +Users accept `userName`, `externalId`, `active`, `displayName`, `name` and `emails`. +Names support `formatted`, `familyName`, `givenName`, `middleName`, `honorificPrefix` +and `honorificSuffix`. Emails support `value`, `type`, `display` and `primary`, with +at most one primary entry. `userName` is immutable and globally unique among gateway +accounts. IDs and external identities remain stable across deactivation. +Administrator roles, permission escalation and environment-managed administrator +names cannot be provisioned. New accounts receive only the default `chat` and +`models` scopes. Their generated key is discarded; an authorised operator must issue +a new key through the existing key-rotation flow when key access is needed. + +Groups accept `displayName`, `externalId` and `members` (`value`, optional `display` +and `$ref`). Member values are SCIM user IDs in the same organisation. Membership +must also pass the tenant authority. Teams and members cannot grant administrator +roles. Group writes require an atomic team authority; missing authorities return 503. + +PATCH supports `add`, `replace` and `remove` for mutable user attributes and group +attributes. A missing path with an object value applies its named mutable attributes. +Adding a name merges its subattributes; adding emails appends entries; adding members +merges by member ID. Removing `active` deactivates the account. A member can be removed +with `members[value eq ""]`. Other selectors and nested PATCH paths are unsupported. +PUT replaces the supported attributes; omitted user `active` defaults to true and +omitted group members defaults to an empty list. + +For example, create a user using the User schema: + +```json +{"schemas":["urn:ietf:params:scim:schemas:core:2.0:User"],"userName":"alice","externalId":"directory-subject-1","active":true} +``` + +POST returns 201 with `Location` and `ETag`. Repeating an external ID in the same +organisation and resource type returns its existing resource with 200, including +after deactivation. A conflicting username returns 409 `uniqueness`. + +Each resource's `meta.version` is its weak ETag. Send the current ETag in `If-Match` +on PUT, PATCH or DELETE. A stale ETag returns 412; omitting the header still uses +an atomic stored-version check, so concurrent writers cannot silently overwrite. +`If-Match: *` requires the resource to exist. + +## Limits, deactivation and retention + +List responses use `startIndex` (one-based, default 1) and `count` (default 100, +maximum 100; zero returns no rows). The only filter expressions are +`userName eq "..."`, `externalId eq "..."` and `displayName eq "..."`. Other expressions +return 400 `invalidFilter`. Sorting, projections, bulk operations and password changes +are unsupported. Requests are limited to 128 KiB, 100 PATCH operations, 100 email +entries and 1,000 group members. Page positions are not snapshot cursors. + +Setting `active=false` or deleting a user deactivates it. The account, external identity +and historical usage remain. The server rotates and discards its key, marks it revoked, +and advances the existing key-control and model-grant security revisions in the same +transaction. Dashboard sessions fail persisted revocation and key-prefix checks. +Reactivation clears the revocation marker without restoring the old key or sessions. +DELETE returns 204. Deleting a group removes its current memberships and deactivates +its mapped team while retaining the group identity and audit history. + +SCIM storage contains directory identity attributes, external mappings, token digests, +membership state and content-free operation audits. It does not record prompts, +generated responses or provisioning bearer values. These records have no automatic +retention expiry; deactivation does not purge them. Operators must account for that +retention when enrolling a directory. SCIM has no provider calls, inference cost, +payment processing or automatic billing; D1 operations incur the deployment's normal +storage costs. Responses do not assert directory-provider certification. + +Errors use `urn:ietf:params:scim:api:messages:2.0:Error` with string `status`, `scimType` +and a bounded detail. With SCIM enabled, missing tables, account columns, security +revision storage, atomic collaborators or durable storage return JSON 503 before +account changes. Storage outages never fall back to local account writes or retry +uncertain mutations. The Worker forwards the public path, body, bearer and `If-Match` +to the Container; the private SCIM domain only accepts its fixed internal POST target. diff --git a/intelligence-migrations/0036_scim_provisioning.sql b/intelligence-migrations/0036_scim_provisioning.sql new file mode 100644 index 00000000..6001f7a1 --- /dev/null +++ b/intelligence-migrations/0036_scim_provisioning.sql @@ -0,0 +1,37 @@ +-- Tenant-scoped external identities, retained account mappings and SCIM revisions. +CREATE TABLE IF NOT EXISTS scim_resources ( + org_id TEXT NOT NULL, + kind TEXT NOT NULL CHECK(kind IN ('Users', 'Groups')), + id TEXT NOT NULL, + external_id TEXT, + user_name TEXT, + display_name TEXT, + deactivated INTEGER NOT NULL DEFAULT 0 CHECK(deactivated IN (0, 1)), + revision INTEGER NOT NULL CONSTRAINT scim_cas_confirmed CHECK(revision BETWEEN 1 AND 9007199254740991), + document TEXT NOT NULL, + PRIMARY KEY(org_id, kind, id), + UNIQUE(org_id, kind, external_id), + UNIQUE(org_id, kind, user_name) +); +CREATE UNIQUE INDEX IF NOT EXISTS idx_scim_account_identity ON scim_resources(user_name) WHERE kind='Users'; +CREATE TABLE IF NOT EXISTS scim_group_members ( + org_id TEXT NOT NULL, + group_id TEXT NOT NULL, + user_id TEXT NOT NULL, + PRIMARY KEY(org_id, group_id, user_id) +); +CREATE TABLE IF NOT EXISTS scim_token_digests ( + org_id TEXT PRIMARY KEY, + digest_sha256 TEXT NOT NULL CHECK(length(digest_sha256)=64), + updated_at TEXT NOT NULL +); +CREATE TABLE IF NOT EXISTS scim_audit ( + sequence INTEGER PRIMARY KEY AUTOINCREMENT, + org_id TEXT NOT NULL, + kind TEXT NOT NULL, + resource_id TEXT NOT NULL, + revision INTEGER NOT NULL, + operation TEXT NOT NULL, + at TEXT NOT NULL +); +CREATE INDEX IF NOT EXISTS idx_scim_audit_scope ON scim_audit(org_id, sequence); diff --git a/package.json b/package.json index ab0e00fa..ecfe77f2 100644 --- a/package.json +++ b/package.json @@ -9,7 +9,7 @@ "lint": "ruff check . --select E9,F63,F7,F82", "typecheck": "mypy --ignore-missing-imports --follow-imports=skip --disable-error-code import-untyped --disable-error-code var-annotated --disable-error-code arg-type --disable-error-code assignment --disable-error-code operator app.py routes services providers streaming", "test": "python -m pytest -q", - "test:worker": "node --test tests/test_cloudflare_worker.mjs tests/test_cascades_worker.mjs tests/test_cascades_dashboard.mjs tests/test_image_quality_worker.mjs tests/test_secret_scan_worker.mjs tests/test_secret_firewall_worker.mjs tests/test_secret_firewall_coverage_worker.mjs tests/test_roleplay_compaction_budget_worker.mjs tests/test_roleplay_compaction_model_worker.mjs tests/test_roleplay_stream_validation_worker.mjs tests/test_aihubmix_worker.mjs tests/test_gpt_image_moderation_worker.mjs tests/test_dashboard_scripts.mjs tests/test_operations_dashboard.mjs tests/test_opencode_reasoning_worker.mjs tests/test_opencode_thinking_worker.mjs tests/test_nanogpt_reasoning_worker.mjs tests/test_nanogpt_speed_routing_worker.mjs tests/test_sse_heartbeat_worker.mjs tests/test_roleplay_worker.mjs tests/test_roleplay_config_worker.mjs tests/test_roleplay_glm_failover_worker.mjs tests/test_roleplay_intelligence_worker.mjs tests/test_roleplay_memory_worker.mjs tests/test_roleplay_compatibility_worker.mjs tests/test_roleplay_history_reconciliation_worker.mjs tests/test_roleplay_output_contract_analysis.mjs tests/test_roleplay_output_contract_cleanup_worker.mjs tests/test_roleplay_output_contract_nonstream_worker.mjs tests/test_roleplay_output_contract_worker.mjs tests/test_roleplay_performance_worker.mjs tests/test_roleplay_quality_routing_worker.mjs tests/test_roleplay_reasoning_worker.mjs tests/test_roleplay_session_storage_worker.mjs tests/test_roleplay_transport_failure_worker.mjs tests/test_roleplay_unlimited_worker.mjs tests/test_roleplay_workbench_worker.mjs tests/test_roleplay_candidate_context_worker.mjs tests/test_roleplay_operator_worker.mjs tests/test_workbench_stream.mjs tests/test_intelligence_worker.mjs tests/test_intelligence_d1_worker.mjs tests/test_intelligence_auth_worker.mjs tests/test_intelligence_outbound_worker.mjs tests/test_intelligence_operator.mjs tests/test_control_users_worker.mjs tests/test_control_audit_worker.mjs tests/test_access_sso_worker.mjs tests/test_d1_schema_worker.mjs tests/test_shadow_eval_worker.mjs tests/test_auto_routes_worker.mjs tests/test_ai_outbound_worker.mjs tests/test_knowledge_*.mjs tests/test_control_state_worker.mjs tests/test_route_health_worker.mjs tests/test_gateway_mcp_worker.mjs tests/test_usage_ledger_worker.mjs tests/test_media_outbound_worker.mjs tests/test_media_files_worker.mjs tests/test_media_jobs_worker.mjs tests/test_tool_repair_worker.mjs tests/test_prometheus_worker.mjs tests/test_evaluation_noise_floor_worker.mjs tests/test_prompt_templates_worker.mjs tests/test_config_snapshots_worker.mjs tests/test_managed_idempotency_worker.mjs tests/test_wave3_hub_worker.mjs tests/test_wave3_transport_worker.mjs tests/test_admission_leases_worker.mjs tests/test_edge_request_metrics_worker.mjs tests/test_request_cancellation_worker.mjs tests/test_mcp_contract_drift_worker.mjs tests/test_zero_content_retention_worker.mjs tests/test_prompt_cache_cost_worker.mjs tests/test_config_revision_sync_worker.mjs tests/test_gateway_alerts_worker.mjs tests/test_session_tiers_worker.mjs tests/test_usage_reservations_worker.mjs tests/test_shared_generation_cache_worker.mjs tests/test_deferred_tools_worker.mjs tests/test_generation_deadline_worker.mjs tests/test_wave4_worker.mjs tests/test_responses_state_worker.mjs tests/test_gateway_batches_worker.mjs tests/test_semantic_generation_cache_worker.mjs tests/test_canary_traffic_worker.mjs tests/test_context_pages_worker.mjs tests/test_observability_exports_worker.mjs tests/test_prompt_injection_worker.mjs tests/test_pii_rehydration_worker.mjs tests/test_wave5_worker.mjs tests/test_usage_receipts_worker.mjs tests/test_learned_cooldown_worker.mjs tests/test_stream_cost_breaker_worker.mjs tests/test_realtime_worker.mjs tests/test_latency_slo_worker.mjs tests/test_context_canary_worker.mjs tests/test_enterprise_contracts_worker.mjs tests/test_wave6_worker.mjs tests/test_saml_federation_worker.mjs tests/test_tenant_governance_worker.mjs tests/test_tenant_hierarchy_worker.mjs", + "test:worker": "node --test tests/test_cloudflare_worker.mjs tests/test_cascades_worker.mjs tests/test_cascades_dashboard.mjs tests/test_image_quality_worker.mjs tests/test_secret_scan_worker.mjs tests/test_secret_firewall_worker.mjs tests/test_secret_firewall_coverage_worker.mjs tests/test_roleplay_compaction_budget_worker.mjs tests/test_roleplay_compaction_model_worker.mjs tests/test_roleplay_stream_validation_worker.mjs tests/test_aihubmix_worker.mjs tests/test_gpt_image_moderation_worker.mjs tests/test_dashboard_scripts.mjs tests/test_operations_dashboard.mjs tests/test_opencode_reasoning_worker.mjs tests/test_opencode_thinking_worker.mjs tests/test_nanogpt_reasoning_worker.mjs tests/test_nanogpt_speed_routing_worker.mjs tests/test_sse_heartbeat_worker.mjs tests/test_roleplay_worker.mjs tests/test_roleplay_config_worker.mjs tests/test_roleplay_glm_failover_worker.mjs tests/test_roleplay_intelligence_worker.mjs tests/test_roleplay_memory_worker.mjs tests/test_roleplay_compatibility_worker.mjs tests/test_roleplay_history_reconciliation_worker.mjs tests/test_roleplay_output_contract_analysis.mjs tests/test_roleplay_output_contract_cleanup_worker.mjs tests/test_roleplay_output_contract_nonstream_worker.mjs tests/test_roleplay_output_contract_worker.mjs tests/test_roleplay_performance_worker.mjs tests/test_roleplay_quality_routing_worker.mjs tests/test_roleplay_reasoning_worker.mjs tests/test_roleplay_session_storage_worker.mjs tests/test_roleplay_transport_failure_worker.mjs tests/test_roleplay_unlimited_worker.mjs tests/test_roleplay_workbench_worker.mjs tests/test_roleplay_candidate_context_worker.mjs tests/test_roleplay_operator_worker.mjs tests/test_workbench_stream.mjs tests/test_intelligence_worker.mjs tests/test_intelligence_d1_worker.mjs tests/test_intelligence_auth_worker.mjs tests/test_intelligence_outbound_worker.mjs tests/test_intelligence_operator.mjs tests/test_control_users_worker.mjs tests/test_control_audit_worker.mjs tests/test_access_sso_worker.mjs tests/test_d1_schema_worker.mjs tests/test_shadow_eval_worker.mjs tests/test_auto_routes_worker.mjs tests/test_ai_outbound_worker.mjs tests/test_knowledge_*.mjs tests/test_control_state_worker.mjs tests/test_route_health_worker.mjs tests/test_gateway_mcp_worker.mjs tests/test_usage_ledger_worker.mjs tests/test_media_outbound_worker.mjs tests/test_media_files_worker.mjs tests/test_media_jobs_worker.mjs tests/test_tool_repair_worker.mjs tests/test_prometheus_worker.mjs tests/test_evaluation_noise_floor_worker.mjs tests/test_prompt_templates_worker.mjs tests/test_config_snapshots_worker.mjs tests/test_managed_idempotency_worker.mjs tests/test_wave3_hub_worker.mjs tests/test_wave3_transport_worker.mjs tests/test_admission_leases_worker.mjs tests/test_edge_request_metrics_worker.mjs tests/test_request_cancellation_worker.mjs tests/test_mcp_contract_drift_worker.mjs tests/test_zero_content_retention_worker.mjs tests/test_prompt_cache_cost_worker.mjs tests/test_config_revision_sync_worker.mjs tests/test_gateway_alerts_worker.mjs tests/test_session_tiers_worker.mjs tests/test_usage_reservations_worker.mjs tests/test_shared_generation_cache_worker.mjs tests/test_deferred_tools_worker.mjs tests/test_generation_deadline_worker.mjs tests/test_wave4_worker.mjs tests/test_responses_state_worker.mjs tests/test_gateway_batches_worker.mjs tests/test_semantic_generation_cache_worker.mjs tests/test_canary_traffic_worker.mjs tests/test_context_pages_worker.mjs tests/test_observability_exports_worker.mjs tests/test_prompt_injection_worker.mjs tests/test_pii_rehydration_worker.mjs tests/test_wave5_worker.mjs tests/test_usage_receipts_worker.mjs tests/test_learned_cooldown_worker.mjs tests/test_stream_cost_breaker_worker.mjs tests/test_realtime_worker.mjs tests/test_latency_slo_worker.mjs tests/test_context_canary_worker.mjs tests/test_enterprise_contracts_worker.mjs tests/test_wave6_worker.mjs tests/test_scim_provisioning_worker.mjs tests/test_saml_federation_worker.mjs tests/test_tenant_governance_worker.mjs tests/test_tenant_hierarchy_worker.mjs", "db:migrate": "python scripts/validate_sqlite_schema.py --apply", "db:validate": "python scripts/validate_sqlite_schema.py", "security:secrets": "python scripts/check_static_secrets.py", diff --git a/routes/scim.py b/routes/scim.py new file mode 100644 index 00000000..9fada8c5 --- /dev/null +++ b/routes/scim.py @@ -0,0 +1,145 @@ +"""SCIM bearer API with no dashboard or cookie-based authorisation.""" +import json + +from flask import Blueprint, Response, current_app, g, request + +from services import scim_provisioning as scim + + +def response(body=None, status=200, resource=None): + result = Response("" if body is None else json.dumps(body, ensure_ascii=True), status=status, + content_type="application/scim+json") + result.headers["Cache-Control"] = "no-store" + if resource: + result.headers["ETag"] = resource["meta"]["version"] + if status == 201: + result.headers["Location"] = resource["meta"]["location"] + return result + + +def discovery(name, resource_id=None): + definitions = [] + for kind, schema, attributes in (("User", scim.USER_SCHEMA, scim.USER_ATTRIBUTES), + ("Group", scim.GROUP_SCHEMA, scim.GROUP_ATTRIBUTES)): + if name == "Schemas": + definitions.append({"schemas": ["urn:ietf:params:scim:schemas:core:2.0:Schema"], + "id": schema, "name": kind, "attributes": _attributes(kind, attributes)}) + else: + definitions.append({"schemas": ["urn:ietf:params:scim:schemas:core:2.0:ResourceType"], + "id": kind, "name": kind, "endpoint": "/" + kind + "s", "schema": schema, + "schemaExtensions": []}) + if name == "ServiceProviderConfig": + if resource_id: + raise scim.ScimError(404, "Resource not found") + return {"schemas": ["urn:ietf:params:scim:schemas:core:2.0:ServiceProviderConfig"], + "patch": {"supported": True}, "bulk": {"supported": False, "maxOperations": 0, "maxPayloadSize": 0}, + "filter": {"supported": True, "maxResults": 100}, "changePassword": {"supported": False}, + "sort": {"supported": False}, "etag": {"supported": True}, + "authenticationSchemes": [{"type": "oauthbearertoken", "name": "Bearer token", + "description": "Tenant-bound provisioning bearer token", "primary": True}]} + if resource_id: + for definition in definitions: + if definition["id"] == resource_id: + return definition + raise scim.ScimError(404, "Resource not found") + return {"schemas": [scim.LIST_SCHEMA], "totalResults": len(definitions), + "startIndex": 1, "itemsPerPage": len(definitions), "Resources": definitions} + + +def _attributes(kind, attributes): + values = [] + for name in sorted(attributes): + complex_attribute = name in {"name", "emails", "members"} + value = {"name": name, "type": "complex" if complex_attribute else "boolean" if name == "active" else "string", + "multiValued": name in {"emails", "members"}, "required": name == ("userName" if kind == "User" else "displayName"), + "mutability": "immutable" if name == "userName" else "readWrite", "returned": "default", + "uniqueness": "server" if name in {"externalId", "userName"} else "none"} + if complex_attribute: + names = {"name": ["formatted", "givenName", "familyName", "middleName", "honorificPrefix", "honorificSuffix"], + "emails": ["value", "type", "display", "primary"], "members": ["value", "display", "$ref"]}[name] + value["subAttributes"] = [{"name": key, "type": "boolean" if key == "primary" else "string", + "multiValued": False, "required": key == "value", "mutability": "readWrite", + "returned": "default", "uniqueness": "none"} for key in names] + values.append(value) + return values + + +def register_scim_routes(app, *, service=None): + """Mount explicit collaborators; an enabled unconfigured authority fails closed.""" + if service is not None: + app.extensions["scim_service"] = service + blueprint = Blueprint("scim", __name__, url_prefix="/scim/v2") + + def guard(): + if not request.path.startswith("/scim/v2/"): + return None + if not scim.enabled(): + return response(scim.ScimError(404, "Resource not found").body(), 404) + try: + g.scim_scope = scim.authenticate(request.headers.get("Authorization")) + if request.method not in {"GET", "HEAD", "POST", "PUT", "PATCH", "DELETE"}: + raise scim.ScimError(405, "Method not allowed") + if not request.endpoint or not request.endpoint.startswith("scim."): + raise scim.ScimError(404, "Resource not found") + except scim.ScimError as error: + return response(error.body(), error.status) + return None + + app.before_request_funcs.setdefault(None, []).insert(0, guard) + + @blueprint.route("/", methods=["GET", "POST", "PUT", "PATCH", "DELETE"], provide_automatic_options=False) + @blueprint.route("//", methods=["GET", "POST", "PUT", "PATCH", "DELETE"], provide_automatic_options=False) + def scim_resource(kind, resource_id=None): + try: + org, digest = g.scim_scope + authority = current_app.extensions.get("scim_service") + if authority is None: + raise scim.ScimError(503, "SCIM authority unavailable") + authority.ready() + if kind in {"Schemas", "ResourceTypes", "ServiceProviderConfig"}: + if request.method not in {"GET", "HEAD"}: + raise scim.ScimError(405, "Method not allowed") + return response(discovery(kind, resource_id)) + if kind not in {"Users", "Groups"}: + raise scim.ScimError(404, "Resource not found") + return _resource_request(authority, org, digest, kind, resource_id) + except scim.ScimError as error: + return response(error.body(), error.status) + except Exception: + return response(scim.ScimError(503, "SCIM authority unavailable").body(), 503) + + csrf = app.extensions.get("csrf") + if csrf is not None: + csrf.exempt(blueprint) + app.register_blueprint(blueprint) + + +def _resource_request(authority, org, digest, kind, resource_id): + if request.method in {"GET", "HEAD"}: + if resource_id is None: + return response(authority.list(org, kind, request.args)) + resource = authority.get(org, kind, resource_id) + return response(resource, resource=resource) + body = None + if request.method != "DELETE": + if request.mimetype not in {"application/json", "application/scim+json"}: + raise scim.ScimError(400, "JSON content type required", "invalidSyntax") + if request.content_length is not None and request.content_length > scim.MAX_BODY: + raise scim.ScimError(413, "SCIM body limit exceeded", "tooMany") + raw = request.stream.read(scim.MAX_BODY + 1) + if len(raw) > scim.MAX_BODY: + raise scim.ScimError(413, "SCIM body limit exceeded", "tooMany") + try: + body = json.loads(raw) + except (ValueError, UnicodeError): + raise scim.ScimError(400, "Invalid JSON", "invalidSyntax") from None + if request.method == "POST" and resource_id is None: + resource, created = authority.create(org, kind, body, digest) + return response(resource, 201 if created else 200, resource) + if request.method in {"PUT", "PATCH", "DELETE"} and resource_id is not None: + resource = authority.update(org, kind, resource_id, body, patch=request.method == "PATCH", + delete=request.method == "DELETE", if_match=request.headers.get("If-Match"), + token_digest=digest) + return response(None if request.method == "DELETE" else resource, + 204 if request.method == "DELETE" else 200, resource) + raise scim.ScimError(405, "Method not allowed") diff --git a/services/scim_provisioning.py b/services/scim_provisioning.py new file mode 100644 index 00000000..416f22c2 --- /dev/null +++ b/services/scim_provisioning.py @@ -0,0 +1,444 @@ +"""Strict SCIM resources and tenant-bound atomic persistence collaborators.""" +from __future__ import annotations + +import copy +import hashlib +import hmac +import json +import logging +import os +import re +import uuid +from datetime import datetime, timezone +from functools import lru_cache + +from error_handlers import APIError +from services.auth_primitives import build_api_key_prefix, hash_api_key, serialize_datetime, serialize_scopes +from services.enterprise_contract import AuthorityDenied, AuthorityOperation, TenantContext +from services.user_provisioning import provision_scim_user +from services.user_store import USER_FIELDS + +logger = logging.getLogger(__name__) +USER_SCHEMA = "urn:ietf:params:scim:schemas:core:2.0:User" +GROUP_SCHEMA = "urn:ietf:params:scim:schemas:core:2.0:Group" +PATCH_SCHEMA = "urn:ietf:params:scim:api:messages:2.0:PatchOp" +ERROR_SCHEMA = "urn:ietf:params:scim:api:messages:2.0:Error" +LIST_SCHEMA = "urn:ietf:params:scim:api:messages:2.0:ListResponse" +MAX_BODY = 131072 +USER_ATTRIBUTES = {"userName", "active", "displayName", "name", "emails", "externalId"} +GROUP_ATTRIBUTES = {"displayName", "externalId", "members"} +FILTER = re.compile(r'(userName|externalId|displayName) eq ("(?:[^"\\]|\\.)*")\Z') +MEMBER_PATH = re.compile(r'members\[value eq ("(?:[^"\\]|\\.)*")\]\Z') +REF = re.compile(r"[A-Za-z_][A-Za-z0-9_]{0,127}\Z") + + +class ScimError(Exception): + def __init__(self, status, detail, scim_type="invalidValue"): + super().__init__(detail) + self.status, self.detail, self.scim_type = status, detail, scim_type + + def body(self): + return {"schemas": [ERROR_SCHEMA], "status": str(self.status), + "scimType": self.scim_type, "detail": self.detail} + + +@lru_cache(maxsize=1) +def _warn_once(): + logger.warning("Invalid SCIM_ENABLED; SCIM provisioning disabled") + + +def enabled(env=None): + raw = (os.environ if env is None else env).get("SCIM_ENABLED", "") + flag = raw.strip().lower() if isinstance(raw, str) else "invalid" + if flag in {"", "0", "false", "off", "no"}: + return False + if flag in {"1", "true", "on", "yes"}: + return True + _warn_once() + return False + + +def authenticate(authorization, env=None): + """Only an unambiguous constant-time digest match establishes tenant scope.""" + env = os.environ if env is None else env + if not isinstance(authorization, str) or not re.fullmatch(r"Bearer [^\s]{1,4096}", authorization, re.I): + raise ScimError(401, "Provisioning bearer token required") + presented = hashlib.sha256(authorization.split(" ", 1)[1].encode()).digest() + try: + config = json.loads(env.get("SCIM_TRUST_CONFIG_JSON", "") or "{}") + if not isinstance(config, dict) or set(config) != {"tenants"} or not isinstance(config["tenants"], dict): + raise ValueError() + if len(config["tenants"]) > 1000: + raise ValueError() + matches = [] + for org, entry in config["tenants"].items(): + TenantContext("scim", org) + if not isinstance(entry, dict) or set(entry) != {"token_ref"} or not isinstance(entry["token_ref"], str): + raise ValueError() + if not REF.fullmatch(entry["token_ref"]): + raise ValueError() + token = env.get(entry["token_ref"]) + if not isinstance(token, str) or not token or len(token) > 4096: + continue + if hmac.compare_digest(presented, hashlib.sha256(token.encode()).digest()): + matches.append(org) + if len(matches) != 1: + raise ValueError() + return matches[0], presented.hex() + except (ValueError, TypeError, AttributeError): + raise ScimError(401, "Provisioning bearer token not recognised") from None + + +def text(value, maximum=1024): + if (not isinstance(value, str) or not value or len(value) > maximum + or any(ord(c) < 32 or ord(c) == 127 or 0xD800 <= ord(c) <= 0xDFFF for c in value)): + raise ScimError(400, "Invalid string attribute") + return value + + +def validate_resource(body, kind): + if not isinstance(body, dict): + raise ScimError(400, "Resource must be an object", "invalidSyntax") + attributes = USER_ATTRIBUTES if kind == "Users" else GROUP_ATTRIBUTES + if set(body) - attributes - {"schemas", "id", "meta"}: + raise ScimError(400, "Unsupported resource attribute") + schema = USER_SCHEMA if kind == "Users" else GROUP_SCHEMA + if "schemas" in body and body["schemas"] != [schema]: + raise ScimError(400, "Unsupported resource schema") + value = copy.deepcopy({key: item for key, item in body.items() if key in attributes}) + if "id" in body and not re.fullmatch(r"[a-f0-9]{32}", text(body["id"], 128)): + raise ScimError(400, "Invalid resource id") + if "meta" in body: + meta = body["meta"] + if not isinstance(meta, dict) or set(meta) - {"resourceType", "created", "lastModified", "location", "version"}: + raise ScimError(400, "Unsupported metadata attribute") + for item in meta.values(): + text(item) + for key in ("externalId", "displayName"): + if key in value: + text(value[key]) + if kind == "Groups": + text(value.get("displayName")) + validate_members(value.setdefault("members", [])) + return value + username = text(value.get("userName"), 128) + if username.strip() != username: + raise ScimError(400, "userName must not have surrounding whitespace") + value.setdefault("active", True) + if type(value["active"]) is not bool: + raise ScimError(400, "active must be a boolean") + if "name" in value: + name = value["name"] + if not isinstance(name, dict) or set(name) - {"formatted", "familyName", "givenName", "middleName", "honorificPrefix", "honorificSuffix"}: + raise ScimError(400, "Unsupported name attribute") + for item in name.values(): + text(item) + if "emails" in value: + emails = value["emails"] + if not isinstance(emails, list) or len(emails) > 100: + raise ScimError(400, "Invalid emails") + primary = 0 + for email in emails: + if not isinstance(email, dict) or set(email) - {"value", "type", "primary", "display"}: + raise ScimError(400, "Unsupported email attribute") + text(email.get("value")) + for key in ("type", "display"): + if key in email: + text(email[key]) + if "primary" in email and type(email["primary"]) is not bool: + raise ScimError(400, "Invalid primary email flag") + primary += email.get("primary", False) + if primary > 1: + raise ScimError(400, "Only one primary email is supported") + return value + + +def validate_members(members): + if not isinstance(members, list) or len(members) > 1000: + raise ScimError(400, "Group membership limit is 1000") + seen = set() + for member in members: + if not isinstance(member, dict) or set(member) - {"value", "display", "$ref"}: + raise ScimError(400, "Unsupported member attribute") + member_id = text(member.get("value"), 128) + if member_id in seen: + raise ScimError(400, "Duplicate group member") + seen.add(member_id) + for key in ("display", "$ref"): + if key in member: + text(member[key]) + + +def pagination(args): + try: + start, count = args.get("startIndex", "1"), args.get("count", "100") + if not re.fullmatch(r"[0-9]{1,9}", start) or not re.fullmatch(r"[0-9]{1,9}", count) or int(start) < 1: + raise ValueError() + return int(start), min(int(count), 100) + except (TypeError, ValueError): + raise ScimError(400, "Invalid pagination") from None + + +def parse_filter(expression): + if expression is None: + return None, None + match = FILTER.fullmatch(expression) if len(expression) <= 2048 else None + if not match: + raise ScimError(400, "Only equality on userName, externalId or displayName is supported", "invalidFilter") + try: + return match[1], text(json.loads(match[2])) + except (ValueError, TypeError): + raise ScimError(400, "Invalid filter string", "invalidFilter") from None + + +def revision(resource): + return int(resource["meta"]["version"][3:-1]) + + +def match_version(resource, if_match): + if if_match is not None and if_match not in {"*", resource["meta"]["version"]}: + raise ScimError(412, "Resource version does not match") + + +def patch_resource(resource, body, kind): + if (not isinstance(body, dict) or set(body) != {"schemas", "Operations"} + or body["schemas"] != [PATCH_SCHEMA] or not isinstance(body["Operations"], list) + or not 1 <= len(body["Operations"]) <= 100): + raise ScimError(400, "Invalid PATCH document", "invalidSyntax") + result = copy.deepcopy(resource) + for operation in body["Operations"]: + _patch_operation(result, operation, kind) + return validate_resource(result, kind) + + +def _patch_operation(result, operation, kind): + if not isinstance(operation, dict) or set(operation) - {"op", "path", "value"}: + raise ScimError(400, "Invalid PATCH operation", "invalidSyntax") + op, path = operation.get("op"), operation.get("path") + if not isinstance(op, str) or op.lower() not in {"add", "replace", "remove"}: + raise ScimError(400, "Unsupported PATCH operation", "invalidSyntax") + op = op.lower() + if path is None and op in {"add", "replace"} and isinstance(operation.get("value"), dict): + for key, value in operation["value"].items(): + _patch_operation(result, {"op": op, "path": key, "value": value}, kind) + return + allowed = (USER_ATTRIBUTES - {"userName"}) if kind == "Users" else GROUP_ATTRIBUTES + selector = MEMBER_PATH.fullmatch(path) if isinstance(path, str) and kind == "Groups" else None + if selector and op == "remove": + if "value" in operation: + raise ScimError(400, "remove does not accept value", "invalidSyntax") + try: + member_id = text(json.loads(selector[1]), 128) + except (TypeError, ValueError): + raise ScimError(400, "Invalid member selector", "invalidSyntax") from None + result["members"] = [member for member in result.get("members", []) if member["value"] != member_id] + return + if not isinstance(path, str) or path not in allowed: + raise ScimError(400, "Unsupported PATCH path", "invalidSyntax") + if op == "remove": + if "value" in operation: + raise ScimError(400, "remove does not accept value", "invalidSyntax") + result.pop(path, None) + if path == "active": + result["active"] = False + return + if "value" not in operation: + raise ScimError(400, "PATCH value required", "invalidSyntax") + value = copy.deepcopy(operation["value"]) + if op == "add" and path in {"members", "emails"}: + if not isinstance(value, list): + raise ScimError(400, "Multi-valued attribute requires an array") + value = result.get(path, []) + value + if path == "members": + unique = {} + for member in value: + validate_members([member]) + unique[member["value"]] = member + value = list(unique.values()) + elif op == "add" and path == "name" and isinstance(value, dict): + value = {**result.get(path, {}), **value} + result[path] = value + + +def _account_row(**values): + row = {key: None for key in USER_FIELDS} + for key, value in values.items(): + if key == "controls": + row.update(value or {}) + elif key == "scopes": + row[key] = serialize_scopes(value) + elif key == "is_admin": + row[key] = int(value) + elif key in USER_FIELDS: + row[key] = serialize_datetime(value) if isinstance(value, datetime) else value + return row + + +class D1ScimStore: + """One-shot private RPC; storage failure never falls back to local accounts.""" + def __init__(self, call): + self.call = call + + def _call(self, operation, **values): + try: + body = self.call({"version": 1, "operation": operation, **values}) + except Exception as error: + status = getattr(error, "status", 503) + if status in {400, 409, 412}: + raise ScimError(status, "SCIM storage rejected the operation", + "uniqueness" if status == 409 else "invalidValue") from None + raise ScimError(503, "SCIM storage is unavailable") from None + if not isinstance(body, dict) or body.get("version") != 1 or "error" in body: + raise ScimError(503, "Invalid SCIM storage response") + return body + + def probe(self): + if (os.environ.get("AUTH_STORAGE_BACKEND", "").strip().lower() != "d1" + or os.environ.get("INTELLIGENCE_STORAGE_BACKEND", "").strip().lower() != "d1" + or os.environ.get("CONFIG_REVISION_SYNC_ENABLED", "").strip().lower() not in {"true", "1", "yes", "on"}): + raise ScimError(503, "SCIM requires durable accounts and security revision sync") + self._call("probe") + + def get(self, org, kind, resource_id): + return self._call("get", org_id=org, kind=kind, id=resource_id)["resource"] + + def account(self, username): + return self._call("account", username=username)["account"] + + def list(self, org, kind, attribute=None, value=None, start=1, count=100): + body = self._call("list", org_id=org, kind=kind, attribute=attribute, value=value, start=start, count=count) + return body["resources"], body["total"] + + def put(self, org, kind, resource, expected, account=None, token_digest=None, deactivate=False): + body = self._call("put", org_id=org, kind=kind, resource=resource, expected=expected, + account=account, token_digest=token_digest, deactivate=deactivate) + return body["resource"], body["created"] + + +class ScimService: + def __init__(self, store, auth, *, tenant_authority=None): + self.store, self.auth, self.tenant_authority = store, auth, tenant_authority + + def ready(self): + self.store.probe() + + def _group_scope(self, org, resource): + if self.tenant_authority is None: + raise ScimError(503, "Tenant authority unavailable") + context = TenantContext("scim", org, resource["id"]) + operation = AuthorityOperation(context, resource["id"], revision(resource), uuid.uuid4().hex) + try: + granted = self.tenant_authority(operation) + except AuthorityDenied: + raise ScimError(503, "Tenant authority unavailable") from None + if type(granted) is not TenantContext or granted != context: + raise ScimError(403, "Tenant authority scope mismatch") + for member in resource.get("members", []): + user = self.store.get(org, "Users", member["value"]) + if user is None: + raise ScimError(400, "Member must belong to this organisation") + member_context = TenantContext(user["userName"], org) + try: + decision = self.tenant_authority(AuthorityOperation(member_context, member["value"], 0, uuid.uuid4().hex)) + except AuthorityDenied: + raise ScimError(400, "Member must belong to this organisation") from None + if type(decision) is not TenantContext or decision != member_context: + raise ScimError(400, "Member must belong to this organisation") + + def get(self, org, kind, resource_id): + self.ready() + resource = self.store.get(org, kind, resource_id) + if resource is None: + raise ScimError(404, "Resource not found") + return resource + + def list(self, org, kind, args): + self.ready() + if set(args) - {"filter", "startIndex", "count"}: + raise ScimError(400, "Unsupported list parameter", "invalidSyntax") + start, count = pagination(args) + attribute, value = parse_filter(args.get("filter")) + rows, total = self.store.list(org, kind, attribute, value, start, count) + return {"schemas": [LIST_SCHEMA], "totalResults": total, "startIndex": start, + "itemsPerPage": len(rows), "Resources": rows} + + def create(self, org, kind, body, token_digest=None): + self.ready() + value = validate_resource(body, kind) + if value.get("externalId"): + rows, _ = self.store.list(org, kind, "externalId", value["externalId"], 1, 1) + if rows: + if kind == "Groups": + self._group_scope(org, rows[0]) + return rows[0], False + resource = self._resource(value, kind) + account = None + if kind == "Users": + prepared = [] + try: + provision_scim_user(self.auth, value["userName"], active=value["active"], + persist=lambda **data: prepared.append(_account_row(**data))) + except APIError as error: + if error.status_code == 409 and value.get("externalId"): + rows, _ = self.store.list(org, kind, "externalId", value["externalId"], 1, 1) + if rows: + return rows[0], False + raise ScimError(error.status_code, "Account provisioning rejected", + "uniqueness" if error.status_code == 409 else "invalidValue") from None + account = prepared[0] + else: + self._group_scope(org, resource) + result = self.store.put(org, kind, resource, 0, account, token_digest) + self.auth._forget_verified_keys() + return result + + @staticmethod + def _resource(value, kind, old=None): + now = datetime.now(timezone.utc).isoformat() + resource_id = old["id"] if old else uuid.uuid4().hex + return {"schemas": [USER_SCHEMA if kind == "Users" else GROUP_SCHEMA], "id": resource_id, **value, + "meta": {"resourceType": "User" if kind == "Users" else "Group", + "version": f'W/"{revision(old) + 1 if old else 1}"', + "created": old["meta"]["created"] if old else now, "lastModified": now, + "location": f"/scim/v2/{kind}/{resource_id}"}} + + def update(self, org, kind, resource_id, body, *, patch=False, delete=False, if_match=None, token_digest=None): + old = self.get(org, kind, resource_id) + match_version(old, if_match) + if delete: + value = validate_resource(old, kind) + if kind == "Users": + value["active"] = False + else: + value["members"] = [] + else: + if isinstance(body, dict) and "id" in body and body["id"] != resource_id: + raise ScimError(400, "id is immutable", "mutability") + value = patch_resource(old, body, kind) if patch else validate_resource(body, kind) + if kind == "Users" and value["userName"] != old["userName"]: + raise ScimError(400, "userName is immutable", "mutability") + resource = self._resource(value, kind, old) + account = self._activation_account(old, resource) if kind == "Users" else None + if kind == "Groups": + self._group_scope(org, resource) + saved, _ = self.store.put(org, kind, resource, revision(old), account, token_digest, + deactivate=delete and kind == "Groups") + self.auth._forget_verified_keys() + return saved + + def _activation_account(self, old, resource): + if old["active"] == resource["active"]: + return None + account = self.store.account(old["userName"]) + if not account or account["is_admin"] != 0: + raise ScimError(503, "Provisioned account unavailable") + if resource["active"] is False: + key = self.auth._generate_api_key() + # Sessions compare the short prefix; a collision must not resurrect them. + if build_api_key_prefix(key) == account["api_key_prefix"]: + key = ("A" if key[0] != "A" else "B") + key[1:] + account.update(api_key_hash=hash_api_key(key), api_key_prefix=build_api_key_prefix(key), + revoked_at=datetime.now(timezone.utc).isoformat()) + else: + account["revoked_at"] = None + return account diff --git a/services/user_provisioning.py b/services/user_provisioning.py index 7c6e71ec..98f8887b 100644 --- a/services/user_provisioning.py +++ b/services/user_provisioning.py @@ -1,10 +1,11 @@ """Account provisioning policy shared by the existing Access console.""" +import os from datetime import datetime, timezone from error_handlers import APIError from services.auth_primitives import ( - build_api_key_prefix, default_scopes, require_valid_username, serialize_datetime, + build_api_key_prefix, default_scopes, hash_api_key, require_valid_username, serialize_datetime, ) from services.intelligence_auth import reject_local_integration_management @@ -43,3 +44,33 @@ def create_user(auth, username, is_admin=False, scopes=None): "api_key_prefix": build_api_key_prefix(api_key), "scopes": list(granted_scopes), "is_admin": is_admin, "created_at": serialize_datetime(created_at), "last_login": None, } + + +def provision_scim_user(auth, username, *, is_admin=False, scopes=None, active=True, persist=None): + """Prepare a least-privilege account for an atomic provisioning authority. + + The generated credential is discarded. A trusted persistence callback can commit + the account and its external identity together without an interactive session. + """ + if is_admin is not False or (scopes is not None and scopes != list(default_scopes(False))): + raise APIError("SCIM cannot grant administrator or elevated scopes", 400) + if type(active) is not bool: + raise APIError("SCIM active must be a boolean", 400) + username = require_valid_username(username) + reject_local_integration_management(username) + reserved = {os.environ.get("ADMIN_USERNAME", "admin").strip(), + *(name.strip() for name in os.environ.get("ADMIN_USERNAMES", "").split(","))} + if username in reserved: + raise APIError("SCIM cannot provision an environment-managed administrator", 400) + if auth._load_user_by_username(username) is not None: + raise APIError("User already exists", 409) + api_key = auth._generate_api_key() + now = datetime.now(timezone.utc) + (persist or auth._persist_user)( + username=username, api_key_hash=hash_api_key(api_key), + api_key_prefix=build_api_key_prefix(api_key), is_admin=False, + created_at=now, last_login=None, scopes=default_scopes(False), + created_by="scim", revoked_at=None if active else now, + ) + return {"id": username, "username": username, "is_admin": False, + "scopes": list(default_scopes(False))} diff --git a/tests/fixtures/secret_firewall_coverage.json b/tests/fixtures/secret_firewall_coverage.json index a4b13a9f..63220171 100644 --- a/tests/fixtures/secret_firewall_coverage.json +++ b/tests/fixtures/secret_firewall_coverage.json @@ -169,6 +169,10 @@ "app.route('/auth/saml/acs', methods=['GET', 'POST'])", "app.route('/auth/saml/callback', methods=['GET', 'POST'])" ], + "routes/scim.py": [ + "blueprint.route('/', methods=['GET', 'POST', 'PUT', 'PATCH', 'DELETE'], provide_automatic_options=False)", + "blueprint.route('//', methods=['GET', 'POST', 'PUT', 'PATCH', 'DELETE'], provide_automatic_options=False)" + ], "routes/shadow_eval.py": [ "app.get('/admin/workbench/shadow/league')", "app.get('/admin/workbench/shadow/samples')", @@ -420,6 +424,7 @@ "worker/route-health-d1.mjs", "worker/saml-identity.mjs", "worker/scheduled-maintenance.mjs", + "worker/scim-d1.mjs", "worker/secret-digest.mjs", "worker/secret-firewall.mjs", "worker/secret-scan.mjs", diff --git a/tests/test_scim_provisioning.py b/tests/test_scim_provisioning.py new file mode 100644 index 00000000..848b1590 --- /dev/null +++ b/tests/test_scim_provisioning.py @@ -0,0 +1,376 @@ +"""SCIM lifecycle through registered bearer routes with isolated authorities.""" +import copy +import json +import threading +from concurrent.futures import ThreadPoolExecutor +from unittest.mock import Mock + +import pytest +from flask import Flask, Response +from flask_wtf.csrf import CSRFProtect + +from services import scim_provisioning as scim +from services import user_provisioning as provisioning +from routes.scim import register_scim_routes + + +class Store: + def __init__(self): + self.rows = {} + self.accounts = {} + self.audit = [] + self.lock = threading.RLock() + self.available = True + + def probe(self): + if not self.available: + raise scim.ScimError(503, "Storage unavailable") + + def list(self, org, kind, attribute=None, value=None, start=1, count=100): + rows = [copy.deepcopy(row) for (tenant, resource_kind, _), row in self.rows.items() + if tenant == org and resource_kind == kind] + rows.sort(key=lambda row: row["id"]) + if attribute: + rows = [row for row in rows if row.get(attribute) == value] + return rows[start - 1:start - 1 + count], len(rows) + + def get(self, org, kind, resource_id): + return copy.deepcopy(self.rows.get((org, kind, resource_id))) + + def account(self, username): + return copy.deepcopy(self.accounts.get(username)) + + def put(self, org, kind, resource, expected, account=None, token_digest=None, deactivate=False): + with self.lock: + self.probe() + key = (org, kind, resource["id"]) + old = self.rows.get(key) + if expected == 0: + for (tenant, resource_kind, _), row in self.rows.items(): + if tenant == org and resource_kind == kind: + if resource.get("externalId") and row.get("externalId") == resource["externalId"]: + return copy.deepcopy(row), False + if kind == "Users" and row["userName"] == resource["userName"]: + raise scim.ScimError(409, "Duplicate userName", "uniqueness") + if account and account["username"] in self.accounts: + raise scim.ScimError(409, "Duplicate account", "uniqueness") + elif not old or scim.revision(old) != expected: + raise scim.ScimError(412, "Stale version", "invalidVers") + self.rows[key] = copy.deepcopy(resource) + if account: + self.accounts[account["username"]] = copy.deepcopy(account) + self.audit.append((org, kind, resource["id"], expected + 1)) + return copy.deepcopy(resource), expected == 0 + + +@pytest.fixture +def setup(monkeypatch): + monkeypatch.setenv("ADMIN_USERNAME", "admin") + monkeypatch.setenv("ADMIN_USERNAMES", "") + monkeypatch.setenv("SCIM_ENABLED", "true") + monkeypatch.setenv("SCIM_TRUST_CONFIG_JSON", json.dumps({"tenants": { + "org:a": {"token_ref": "SCIM_TEST_A"}, "org:b": {"token_ref": "SCIM_TEST_B"}}})) + monkeypatch.setenv("SCIM_TEST_A", "synthetic-scim-a") + monkeypatch.setenv("SCIM_TEST_B", "synthetic-scim-b") + auth = Mock() + auth._load_user_by_username.return_value = None + auth._generate_api_key.return_value = "synthetic-discarded-key" + store = Store() + authority = Mock(side_effect=lambda operation: operation.context) + service = scim.ScimService(store, auth, tenant_authority=authority) + app = Flask(__name__) + app.secret_key = "synthetic-csrf" + CSRFProtect(app) + app.add_url_rule("/raw", view_func=lambda: Response(b"unchanged", headers={"X-Raw": "same"})) + register_scim_routes(app, service=service) + return app.test_client(), store, auth, authority + + +def call(client, method, path="Users", body=None, tenant="a", **headers): + return client.open("/scim/v2/" + path, method=method, json=body, + headers={"Authorization": "Bearer synthetic-scim-" + tenant, **headers}) + + +def user(client, username="alice", external="subject-a", **changes): + return call(client, "POST", body={"schemas": [scim.USER_SCHEMA], "userName": username, + "externalId": external, **changes}) + + +@pytest.mark.parametrize("flag", ["", "false", "0", "off", "bad-private-value"]) +def test_disabled_all_paths_precede_auth_and_storage(setup, monkeypatch, flag): + client, store, auth, _ = setup + monkeypatch.setenv("SCIM_ENABLED", flag) + store.available = False + for method in ("GET", "POST", "PUT", "PATCH", "DELETE", "OPTIONS"): + assert client.open("/scim/v2/unknown", method=method).status_code == 404 + assert client.get("/raw").data == b"unchanged" + auth._load_user_by_username.assert_not_called() + + +def test_flag_warns_once_without_value(caplog): + scim._warn_once.cache_clear() + assert not scim.enabled({}) + for _ in range(2): + assert not scim.enabled({"SCIM_ENABLED": "private-invalid"}) + assert caplog.text.count("Invalid SCIM_ENABLED") == 1 + assert "private-invalid" not in caplog.text + + +def test_auth_scope_discovery_and_csrf_exemption(setup): + client, _, _, _ = setup + for headers in ({}, {"Authorization": "Bearer unknown"}, {"Authorization": "Basic x"}): + response = client.get("/scim/v2/Users", headers=headers) + assert response.status_code == 401 + assert response.json["schemas"] == [scim.ERROR_SCHEMA] + assert response.json["status"] == "401" + assert response.mimetype == "application/scim+json" + for endpoint in ("ServiceProviderConfig", "Schemas", "ResourceTypes"): + assert call(client, "GET", endpoint).status_code == 200 + assert call(client, "GET", "Schemas/" + scim.USER_SCHEMA).status_code == 200 + response = user(client) + assert response.status_code == 201 + assert response.headers["ETag"] == response.json["meta"]["version"] + assert response.headers["Location"].endswith("/Users/" + response.json["id"]) + assert "synthetic-discarded-key" not in response.get_data(as_text=True) + assert call(client, "GET", "Users/" + response.json["id"], tenant="b").status_code == 404 + + +def test_idempotency_uniqueness_default_scopes_and_no_admin(setup): + client, store, auth, _ = setup + first = user(client) + repeat = user(client, username="renamed") + assert repeat.status_code == 200 and repeat.json == first.json + assert len(store.accounts) == len(store.audit) == 1 + account = store.accounts["alice"] + assert account["is_admin"] == 0 and account["scopes"] == "chat,models" + auth._require_admin.assert_not_called() + assert user(client, external="other").status_code == 409 + assert user(client, username="admin", external="admin").status_code == 400 + for changes in ({"is_admin": True}, {"roles": [{"value": "admin"}]}, {"scopes": ["admin"]}): + assert user(client, username="evil", external="evil", **changes).status_code == 400 + for kwargs in ({"is_admin": True}, {"scopes": ["admin"]}, {"scopes": ["knowledge:manage"]}): + with pytest.raises(Exception): + provisioning.provision_scim_user(auth, "evil", **kwargs) + + +def test_patch_etag_deactivate_reactivate_preserves_history(setup): + client, store, auth, _ = setup + created = user(client) + path = "Users/" + created.json["id"] + old_account = copy.deepcopy(store.accounts["alice"]) + patch = {"schemas": [scim.PATCH_SCHEMA], "Operations": [ + {"op": "replace", "path": "displayName", "value": "Alice"}, + {"op": "add", "path": "emails", "value": [{"value": "a@example.test", "primary": True}]}, + {"op": "replace", "path": "active", "value": False}]} + auth._generate_api_key.return_value = "changed-synthetic-discarded-key" + changed = call(client, "PATCH", path, patch, **{"If-Match": created.headers["ETag"]}) + assert changed.status_code == 200 and changed.json["active"] is False + assert store.accounts["alice"]["revoked_at"] + assert store.accounts["alice"]["api_key_prefix"] != old_account["api_key_prefix"] + assert store.accounts["alice"]["created_at"] == old_account["created_at"] + for method in ("PUT", "PATCH", "DELETE"): + assert call(client, method, path, patch, **{"If-Match": created.headers["ETag"]}).status_code == 412 + prefix = store.accounts["alice"]["api_key_prefix"] + on = {"schemas": [scim.PATCH_SCHEMA], "Operations": [{"op": "replace", "path": "active", "value": True}]} + assert call(client, "PATCH", path, on).status_code == 200 + assert store.accounts["alice"]["api_key_prefix"] == prefix + assert store.accounts["alice"]["revoked_at"] is None + assert call(client, "DELETE", path).status_code == 204 + assert call(client, "GET", path).json["externalId"] == "subject-a" + assert len(store.accounts) == 1 + + +def test_filter_pagination_and_strict_validation(setup): + client, _, _, _ = setup + user(client) + for expression in ('userName eq "alice"', 'externalId eq "subject-a"', 'displayName eq "missing"'): + response = call(client, "GET", "Users?filter=" + expression + "&count=101") + assert response.status_code == 200 and response.json["itemsPerPage"] <= 100 + for query in ("filter=active eq true", "filter=userName co alice", "startIndex=0", "count=-1", "count=bad"): + assert call(client, "GET", "Users?" + query).status_code == 400 + assert call(client, "GET", "Users?count=0").json["Resources"] == [] + for changes in ({"schemas": ["unknown"]}, {"active": "false"}, {"name": {"unknown": "x"}}, + {"emails": [{"value": "x", "unknown": True}]}): + assert user(client, **changes).status_code == 400 + path = "Users/" + user(client).json["id"] + for operation in ({"op": "move", "path": "active", "value": True}, + {"op": "replace", "path": "roles", "value": []}, + {"op": "replace", "path": "userName", "value": "bob"}): + assert call(client, "PATCH", path, {"schemas": [scim.PATCH_SCHEMA], "Operations": [operation]}).status_code == 400 + + +def test_groups_authority_members_and_patch_are_scoped(setup): + client, store, _, authority = setup + member = user(client).json["id"] + foreign = call(client, "POST", body={"userName": "bob", "externalId": "foreign"}, tenant="b").json["id"] + payload = {"schemas": [scim.GROUP_SCHEMA], "displayName": "engineering", "externalId": "team-a", + "members": [{"value": member}]} + created = call(client, "POST", "Groups", payload) + assert created.status_code == 201 + assert authority.call_args[0][0].context.org_id == "org:a" + path = "Groups/" + created.json["id"] + for value in (foreign, "unknown"): + bad = {"schemas": [scim.PATCH_SCHEMA], "Operations": [{"op": "add", "path": "members", "value": [{"value": value}]}]} + assert call(client, "PATCH", path, bad).status_code == 400 + removed = {"schemas": [scim.PATCH_SCHEMA], "Operations": [{"op": "remove", "path": 'members[value eq "' + member + '"]'}]} + assert call(client, "PATCH", path, removed).json["members"] == [] + assert call(client, "PUT", path, {"displayName": "renamed", "members": []}).status_code == 200 + assert call(client, "DELETE", path).status_code == 204 + assert len(store.rows) == 3 + authority.side_effect = lambda operation: scim.TenantContext("foreign", "org:b") + assert call(client, "POST", "Groups", payload).status_code == 403 + + +def test_missing_schema_and_authority_precede_account_changes(setup): + client, store, auth, authority = setup + store.available = False + response = user(client) + assert response.status_code == 503 and response.json["status"] == "503" + auth._generate_api_key.assert_not_called() + assert not store.accounts + store.available = True + authority.side_effect = scim.AuthorityDenied("unavailable") + assert call(client, "POST", "Groups", {"displayName": "team"}).status_code == 503 + assert not store.rows + + +def test_atomic_duplicate_create_and_stale_updates(setup): + _, store, auth, _ = setup + service = scim.ScimService(store, auth, tenant_authority=lambda op: op.context) + with ThreadPoolExecutor(max_workers=2) as pool: + results = list(pool.map(lambda _: service.create("org:a", "Users", {"userName": "alice", "externalId": "a"}), range(2))) + assert results[0][0]["id"] == results[1][0]["id"] + assert len(store.accounts) == 1 + + +def test_trust_config_ambiguity_and_invalid_values_fail_closed(monkeypatch): + env = {"SCIM_TRUST_CONFIG_JSON": json.dumps({"tenants": {"org:a": {"token_ref": "A"}, "org:b": {"token_ref": "B"}}}), + "A": "synthetic-shared", "B": "synthetic-shared"} + with pytest.raises(scim.ScimError) as error: + scim.authenticate("Bearer synthetic-shared", env) + assert error.value.status == 401 + for config in ("bad", "[]", '{"tenants":{"org:a":{"token":"raw"}}}'): + with pytest.raises(scim.ScimError): + scim.authenticate("Bearer synthetic-shared", {**env, "SCIM_TRUST_CONFIG_JSON": config}) + + +def test_real_auth_session_and_old_key_cannot_return_after_reactivation(setup, monkeypatch): + from flask import session + from services.auth_service import AuthService + from werkzeug.security import check_password_hash + + client, store, _, _ = setup + app = client.application + + class IsolatedAuth(AuthService): + _users = {} + _verified_keys = {} + _api_key_prefix_index = {} + + def lookup(username): + row = store.accounts.get(username) + return IsolatedAuth._row_to_user(row) if row else None + + monkeypatch.setattr(IsolatedAuth, "_load_user_by_username", staticmethod(lookup)) + monkeypatch.setattr(IsolatedAuth, "_generate_api_key", staticmethod(lambda: "original-synthetic-key")) + app.extensions["scim_service"].auth = IsolatedAuth + app.add_url_rule("/who", endpoint="session_identity", view_func=lambda: {"authenticated": IsolatedAuth.get_current_user() is not None}) + created = user(client) + assert created.status_code == 201 + original = copy.deepcopy(store.accounts["alice"]) + assert check_password_hash(original["api_key_hash"], "original-synthetic-key") + with client.session_transaction() as signed: + signed["authenticated"] = True + signed["user"] = {"username": "alice", "api_key_prefix": original["api_key_prefix"]} + assert client.get("/who").json["authenticated"] + monkeypatch.setattr(IsolatedAuth, "_generate_api_key", staticmethod(lambda: "original-synthetic-rotated-key")) + path = "Users/" + created.json["id"] + assert call(client, "DELETE", path).status_code == 204 + assert store.accounts["alice"]["api_key_prefix"] != original["api_key_prefix"] + assert client.get("/who").json["authenticated"] is False + patch = {"schemas": [scim.PATCH_SCHEMA], "Operations": [{"op": "replace", "path": "active", "value": True}]} + assert call(client, "PATCH", path, patch).status_code == 200 + assert not check_password_hash(store.accounts["alice"]["api_key_hash"], "original-synthetic-key") + with app.test_request_context("/who"): + session["authenticated"] = True + session["user"] = {"username": "alice", "api_key_prefix": original["api_key_prefix"]} + assert IsolatedAuth.get_current_user() is None + + +def test_transport_failure_has_no_retry_and_no_account_fallback(monkeypatch): + for name in ("AUTH_STORAGE_BACKEND", "INTELLIGENCE_STORAGE_BACKEND"): + monkeypatch.setenv(name, "d1") + monkeypatch.setenv("CONFIG_REVISION_SYNC_ENABLED", "true") + transport = Mock(side_effect=RuntimeError("private database details")) + store = scim.D1ScimStore(transport) + auth = Mock() + service = scim.ScimService(store, auth) + with pytest.raises(scim.ScimError) as error: + service.create("org:a", "Users", {"userName": "alice"}) + assert error.value.status == 503 + assert "private database details" not in json.dumps(error.value.body()) + transport.assert_called_once_with({"version": 1, "operation": "probe"}) + auth._persist_user.assert_not_called() + auth._generate_api_key.assert_not_called() + transport.reset_mock() + monkeypatch.setenv("AUTH_STORAGE_BACKEND", "sql") + with pytest.raises(scim.ScimError) as error: + store.probe() + assert error.value.status == 503 + transport.assert_not_called() + + +def test_body_limit_unknown_metadata_and_partial_patch_do_not_mutate(setup): + client, store, _, _ = setup + created = user(client) + path = "Users/" + created.json["id"] + before = copy.deepcopy(store.rows) + for operations in ([{"op": "replace", "path": "displayName", "value": "changed"}, + {"op": "replace", "path": "roles", "value": ["admin"]}], + [{"op": "remove", "path": "members", "value": []}]): + assert call(client, "PATCH", path, {"schemas": [scim.PATCH_SCHEMA], "Operations": operations}).status_code == 400 + assert store.rows == before + assert user(client, meta={"unknown": "x"}).status_code == 400 + assert user(client, username=" alice ").status_code == 400 + raw = client.post("/scim/v2/Users", data="x" * (scim.MAX_BODY + 1), + headers={"Authorization": "Bearer synthetic-scim-a", "Content-Type": "application/scim+json"}) + assert raw.status_code == 413 and raw.json["status"] == "413" + invalid = client.post("/scim/v2/Users", data="invalid", headers={ + "Authorization": "Bearer synthetic-scim-a", "Content-Type": "application/scim+json"}) + assert invalid.status_code == 400 and invalid.json["scimType"] == "invalidSyntax" + for path in ("Users?sortBy=userName", "missing/nested/path"): + result = call(client, "GET", path) + assert result.status_code in {400, 404} and result.mimetype == "application/scim+json" + assert call(client, "HEAD", "Users").status_code == 200 + + +def test_successful_put_and_patch_all_mutable_attributes(setup): + client, _, _, _ = setup + created = user(client) + path = "Users/" + created.json["id"] + changed = call(client, "PUT", path, {"schemas": [scim.USER_SCHEMA], "userName": "alice", "externalId": "new-subject", + "displayName": "Alice", "name": {"givenName": "Alice"}, "emails": []}) + assert changed.status_code == 200 + patch = {"schemas": [scim.PATCH_SCHEMA], "Operations": [ + {"op": "add", "path": "name", "value": {"familyName": "Example"}}, + {"op": "replace", "path": "externalId", "value": "final-subject"}, + {"op": "remove", "path": "displayName"}, + {"op": "remove", "path": "emails"}]} + result = call(client, "PATCH", path, patch) + assert result.status_code == 200 and result.json["name"] == {"givenName": "Alice", "familyName": "Example"} + assert result.json["externalId"] == "final-subject" + assert "emails" not in result.json and "displayName" not in result.json + assert call(client, "PUT", path, {"userName": "bob"}).status_code == 400 + + +def test_registration_default_keeps_existing_bytes_and_headers(monkeypatch): + monkeypatch.setenv("SCIM_ENABLED", "") + def baseline(): + app = Flask(__name__) + app.add_url_rule("/raw", view_func=lambda: Response(b"opaque bytes", headers={"X-Raw": "identical"})) + return app + before = baseline().test_client().get("/raw") + app = baseline() + register_scim_routes(app) + after = app.test_client().get("/raw") + assert (before.data, before.status_code, list(before.headers)) == (after.data, after.status_code, list(after.headers)) diff --git a/tests/test_scim_provisioning_worker.mjs b/tests/test_scim_provisioning_worker.mjs new file mode 100644 index 00000000..7073e786 --- /dev/null +++ b/tests/test_scim_provisioning_worker.mjs @@ -0,0 +1,208 @@ +import assert from "node:assert/strict"; +import test from "node:test"; +import { readFileSync } from "node:fs"; +import { DatabaseSync } from "node:sqlite"; +import { handleScimStateRequest } from "../worker/scim-d1.mjs"; +import { USER_FIELDS } from "../worker/control-users-d1.mjs"; +import { RevisionConsumer } from "../worker/config-revision.mjs"; +import { loadWorkerModule } from "./helpers/load_cloudflare_worker.mjs"; + +const schema = name => readFileSync(new URL(`../intelligence-migrations/${name}`, import.meta.url), "utf8"); +const userSchema = "urn:ietf:params:scim:schemas:core:2.0:User"; +const groupSchema = "urn:ietf:params:scim:schemas:core:2.0:Group"; +const id = n => String(n).padStart(32, "0"); +function resource(n = 1, changes = {}, kind = "Users") { + return { schemas: [kind === "Users" ? userSchema : groupSchema], id: id(n), externalId: `subject-${n}`, + ...(kind === "Users" ? { userName: `user-${n}`, active: true } : { displayName: `team-${n}`, members: [] }), + meta: { resourceType: kind === "Users" ? "User" : "Group", version: 'W/"1"', + created: "2026-10-09T00:00:00Z", lastModified: "2026-10-09T00:00:00Z", location: `/scim/v2/${kind}/${id(n)}` }, ...changes }; +} +function account(n = 1, changes = {}) { + return { ...Object.fromEntries(USER_FIELDS.map(key => [key, null])), username: `user-${n}`, + api_key_hash: "synthetic-hash", api_key_prefix: "mllm_synthetic", is_admin: 0, + scopes: "chat,models", created_at: "2026-10-09T00:00:00Z", ...changes }; +} +function database(t, migrated = true) { + const sql = new DatabaseSync(":memory:"); + t.after(() => sql.close()); + for (const file of ["0003_control_users.sql", "0007_usage_ledger.sql", "0011_secret_firewall.sql", "0013_shadow_eval.sql", "0017_control_revisions.sql"]) + sql.exec(schema(file)); + sql.exec("CREATE TABLE retained_usage (username TEXT, amount INTEGER); INSERT INTO retained_usage VALUES ('user-1', 7)"); + sql.exec("CREATE TABLE fake_teams (org TEXT, id TEXT, name TEXT, active INTEGER, PRIMARY KEY(org,id))"); + sql.exec("CREATE TABLE fake_principals (org TEXT, username TEXT UNIQUE)"); + if (migrated) sql.exec(schema("0036_scim_provisioning.sql")); + const db = { prepare(query) { + let args = []; + const statement = { bind(...values) { args = values; return statement; }, + first() { return sql.prepare(query).get(...args) ?? null; }, + all() { const p = sql.prepare(query); return p.columns().length ? { results: p.all(...args), meta: { changes: 0 } } + : { results: [], meta: { changes: Number(p.run(...args).changes) } }; } }; + return statement; + }, async batch(statements) { + sql.exec("BEGIN IMMEDIATE"); + try { const result = statements.map(s => s.all()); sql.exec("COMMIT"); return result; } + catch (error) { sql.exec("ROLLBACK"); throw error; } + } }; + const env = { SCIM_ENABLED: "true", CONFIG_REVISION_SYNC_ENABLED: "true", INTELLIGENCE_DB: db }; + const collaborators = { principalStatements: ({ org_id, resource: row }) => [ + db.prepare("INSERT INTO fake_principals VALUES (?,?)").bind(org_id, row.userName)], + teamStatements: ({ org_id, resource: row, deactivated }) => [ + db.prepare("INSERT INTO fake_teams VALUES (?,?,?,?) ON CONFLICT(org,id) DO UPDATE SET name=excluded.name,active=excluded.active") + .bind(org_id, row.id, row.displayName, Number(!deactivated))] }; + const send = async (body, extras = {}) => { + const response = await handleScimStateRequest(new Request("http://intelligence.internal/v1/managed-state/scim", { + method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ version: 1, org_id: "org:a", ...body }), + }), env, { ...collaborators, ...extras }); + return { status: response.status, ...await response.json() }; + }; + const put = (row, expected = 0, accountRow = null, kind = "Users", extras) => send({ operation: "put", kind, + resource: row, expected, account: accountRow, token_digest: "a".repeat(64), deactivate: false }, extras); + return { sql, db, env, send, put }; +} + +test("atomic account, identity, externalId replay, scoped uniqueness and retained usage", async t => { + const { sql, send, put } = database(t); + assert.equal((await put(resource(), 0, account())).status, 200); + const duplicate = await put(resource(2, { externalId: "subject-1" }), 0, account(2)); + assert.equal(duplicate.resource.id, id(1)); + assert.equal(duplicate.created, false); + assert.equal((await put(resource(2, { userName: "user-1" }), 0, account(2, { username: "user-1" }))).status, 409); + assert.equal((await send({ operation: "get", kind: "Users", id: id(1), org_id: "org:b" })).resource, null); + assert.equal(sql.prepare("SELECT count(*) n FROM control_users").get().n, 1); + assert.equal(sql.prepare("SELECT count(*) n FROM scim_audit").get().n, 1); + assert.equal(sql.prepare("SELECT org FROM fake_principals").get().org, "org:a"); + assert.equal(sql.prepare("SELECT amount FROM retained_usage").get().amount, 7); + for (const changes of [{ is_admin: 1 }, { scopes: "chat,admin" }]) + assert.equal((await put(resource(3), 0, account(3, changes))).status, 400); +}); + +test("CAS concurrent updates roll back account, audit, membership and revision changes", async t => { + const { sql, put } = database(t); + await put(resource(), 0, account()); + const next = resource(1, { active: false, meta: { ...resource().meta, version: 'W/"2"' } }); + const outcomes = await Promise.all([put(next, 1, account(1, { revoked_at: "2026-10-09T01:00:00Z", api_key_prefix: "mllm_changed", api_key_hash: "synthetic-changed-hash" })), + put(next, 1, account(1, { revoked_at: "2026-10-09T01:00:00Z", api_key_prefix: "mllm_other", api_key_hash: "synthetic-other-hash" }))]); + assert.deepEqual(outcomes.map(r => r.status).sort(), [200, 412]); + assert.equal(sql.prepare("SELECT count(*) n FROM scim_audit").get().n, 2); + assert.equal(sql.prepare("SELECT revision FROM control_revisions WHERE domain='key_controls'").get().revision, 2); + assert.equal(sql.prepare("SELECT api_key_prefix FROM control_users").get().api_key_prefix, "mllm_changed"); +}); + +test("deactivation revision invalidates runtime within security TTL", async t => { + const { db, env, put, sql } = database(t); + let now = 0, refreshed = []; + const consumer = new RevisionConsumer(env, { clock: () => now, jitter: () => 1, + refreshers: Object.fromEntries(["model_overrides", "key_controls", "model_grants"].map(name => [name, () => { + refreshed.push(name); return sql.prepare("SELECT revoked_at FROM control_users").get(); }])) }); + await put(resource(), 0, account()); + await consumer.tick(); + assert.equal(consumer.requireFreshSecurity(), null); + const next = resource(1, { active: false, meta: { ...resource().meta, version: 'W/"2"' } }); + await put(next, 1, account(1, { revoked_at: "2026-10-09T01:00:00Z", api_key_prefix: "mllm_changed", api_key_hash: "synthetic-changed-hash" })); + refreshed = []; now = 5; + await consumer.tick(); + assert.ok(refreshed.includes("key_controls") && refreshed.includes("model_grants")); + assert.equal(consumer.requireFreshSecurity(), null); + db.prepare = () => { throw new Error("synthetic outage"); }; + now = 10.01; await consumer.tick(); + assert.equal(consumer.requireFreshSecurity().status, 503); +}); + +test("teams and membership stay tenant bound; collaborator failure rolls back", async t => { + const { sql, put, send } = database(t); + await put(resource(), 0, account()); + const group = resource(10, { members: [{ value: id(1) }] }, "Groups"); + assert.equal((await put(group, 0, null, "Groups")).status, 200); + assert.equal(sql.prepare("SELECT name FROM fake_teams").get().name, "team-10"); + assert.equal(sql.prepare("SELECT count(*) n FROM scim_group_members").get().n, 1); + const foreign = resource(11, { members: [{ value: id(999) }] }, "Groups"); + assert.equal((await put(foreign, 0, null, "Groups")).status, 400); + const failed = await put(resource(12, {}, "Groups"), 0, null, "Groups", { teamStatements: () => { + throw new Error("private details"); } }); + assert.equal(failed.status, 503); + assert.equal(sql.prepare("SELECT count(*) n FROM scim_resources WHERE kind='Groups'").get().n, 1); + assert.equal((await put(resource(13, {}, "Groups"), 0, null, "Groups", { teamStatements: undefined })).status, 503); + const deactivated = { ...group, members: [], meta: { ...group.meta, version: 'W/"2"' } }; + assert.equal((await send({ operation: "put", kind: "Groups", resource: deactivated, expected: 1, + account: null, token_digest: null, deactivate: true })).status, 200); + assert.equal(sql.prepare("SELECT deactivated FROM scim_resources WHERE kind='Groups'").get().deactivated, 1); + assert.equal(sql.prepare("SELECT active FROM fake_teams").get().active, 0); + assert.equal(sql.prepare("SELECT count(*) n FROM scim_group_members").get().n, 0); + const outcomes = await Promise.all([put({ ...group, displayName: "winning", meta: { ...group.meta, version: 'W/"3"' } }, 2, null, "Groups"), + put({ ...group, displayName: "losing", meta: { ...group.meta, version: 'W/"3"' } }, 2, null, "Groups")]); + assert.deepEqual(outcomes.map(result => result.status).sort(), [200, 412]); + assert.equal(sql.prepare("SELECT name FROM fake_teams").get().name, "winning"); + assert.equal(sql.prepare("SELECT count(*) n FROM scim_group_members").get().n, 1); +}); + +test("missing schema, disabled flag, private target and invalid bodies fail before changes", async t => { + const { sql, env, put } = database(t, false); + assert.equal((await put(resource(), 0, account())).status, 503); + assert.equal(sql.prepare("SELECT count(*) n FROM control_users").get().n, 0); + const req = (url = "http://intelligence.internal/v1/managed-state/scim", body = "{}") => new Request(url, { + method: "POST", headers: { "content-type": "application/json" }, body }); + for (const flag of [undefined, "", "false", "invalid-private"]) + assert.equal((await handleScimStateRequest(req(), { SCIM_ENABLED: flag, INTELLIGENCE_DB: { + prepare() { assert.fail("off touched storage"); } } })).status, 404); + assert.equal((await handleScimStateRequest(req("https://external.test/v1/managed-state/scim"), env)).status, 404); + assert.equal((await handleScimStateRequest(req(undefined, "invalid"), env)).status, 400); + assert.equal((await handleScimStateRequest(req(undefined, "a".repeat(140000)), env)).status, 400); +}); + +test("migration is additive and repeats without destroying existing rows; list count is capped", async t => { + const { sql, send, put } = database(t); + await put(resource(), 0, account()); + sql.exec(schema("0036_scim_provisioning.sql")); + const listed = await send({ operation: "list", kind: "Users", attribute: "userName", value: "user-1", start: 1, count: 100 }); + assert.equal(listed.total, 1); + assert.equal(listed.resources.length, 1); + assert.equal((await send({ operation: "list", kind: "Users", attribute: "roles", value: "admin", start: 1, count: 100 })).status, 400); + assert.equal((await send({ operation: "list", kind: "Users", attribute: null, value: null, start: 1, count: 101 })).status, 400); +}); + +test("mutations require atomic principal authority and fail closed on transaction errors", async t => { + const { sql, put, env, send } = database(t); + assert.equal((await put(resource(), 0, account(), "Users", { principalStatements: undefined })).status, 503); + assert.equal(sql.prepare("SELECT count(*) n FROM control_users").get().n, 0); + assert.equal((await put(resource(), 0, account(), "Users", { principalStatements: () => [ + env.INTELLIGENCE_DB.prepare("INSERT INTO nonexistent_authority VALUES (1)")] })).status, 503); + assert.equal(sql.prepare("SELECT count(*) n FROM scim_resources").get().n, 0); + assert.equal(sql.prepare("SELECT count(*) n FROM control_revisions").get().n, 0); + env.CONFIG_REVISION_SYNC_ENABLED = "false"; + assert.equal((await send({ operation: "probe" })).status, 400); + const response = await handleScimStateRequest(new Request("http://intelligence.internal/v1/managed-state/scim", { + method: "POST", headers: { "content-type": "application/json" }, body: '{"version":1,"operation":"probe"}', + }), env); + assert.equal(response.status, 503); +}); + +test("missing owned columns and security revision schema precede account mutations", async t => { + const { sql, put } = database(t, false); + sql.exec("CREATE TABLE scim_resources (org_id TEXT); INSERT INTO scim_resources VALUES ('retained')"); + sql.exec(schema("0036_scim_provisioning.sql").split("CREATE UNIQUE INDEX", 1)[0]); + assert.equal((await put(resource(), 0, account())).status, 503); + assert.equal(sql.prepare("SELECT org_id FROM scim_resources").get().org_id, "retained"); + assert.equal(sql.prepare("SELECT count(*) n FROM control_users").get().n, 0); + const second = database(t); + second.sql.exec("DROP TABLE control_revisions"); + assert.equal((await second.put(resource(), 0, account())).status, 503); + assert.equal(second.sql.prepare("SELECT count(*) n FROM control_users").get().n, 0); +}); + +test("public SCIM routes forward body, bearer and If-Match unchanged to Container", async () => { + const { default: worker } = await loadWorkerModule(); + const captured = []; + const env = { MULTILLM_PROXY_CONTAINER: { getByName() { return { async fetch(request) { + captured.push({ path: new URL(request.url).pathname, body: await request.text(), headers: request.headers }); + return new Response("{}", { headers: { "content-type": "application/scim+json" } }); + } }; } } }; + const body = '{"Operations":[{"op":"replace","path":"active","value":false}]}'; + const response = await worker.fetch(new Request("https://gateway.test/scim/v2/Users/test", { method: "PATCH", body, + headers: { "Authorization": "Bearer synthetic-scim-token", "Content-Type": "application/scim+json", "If-Match": 'W/"1"' } }), env); + assert.equal(response.status, 200); + assert.equal(captured.length, 1); + assert.equal(captured[0].path, "/scim/v2/Users/test"); + assert.equal(captured[0].body, body); + assert.equal(captured[0].headers.get("Authorization"), "Bearer synthetic-scim-token"); + assert.equal(captured[0].headers.get("If-Match"), 'W/"1"'); +}); diff --git a/worker/container-env.mjs b/worker/container-env.mjs index d461dde9..017e9d73 100644 --- a/worker/container-env.mjs +++ b/worker/container-env.mjs @@ -45,6 +45,8 @@ const DIRECT_ENV_KEYS = [ "SAML_CALLBACK_URL", "SAML_ENABLED", "SAML_TRUST_CONFIG_JSON", + "SCIM_ENABLED", + "SCIM_TRUST_CONFIG_JSON", "TENANT_GOVERNANCE_ENABLED", "CONTEXT_CANARY_MODE", "CONTEXT_CANARY_POLICY_JSON", diff --git a/worker/d1-schema.mjs b/worker/d1-schema.mjs index bf6cb625..0d77910f 100644 --- a/worker/d1-schema.mjs +++ b/worker/d1-schema.mjs @@ -10,7 +10,7 @@ export const REQUIRED_D1_TABLES = Object.freeze(["intelligence_policy", "intelli "usage_events", "usage_daily", "usage_batches", "media_jobs", "media_job_items", "shadow_eval_samples", "shadow_eval_results", "shadow_eval_config", "shadow_eval_policy_backups", "prompt_templates", - "config_snapshot_revisions", "config_snapshots", "config_snapshot_applications", "control_revisions", "tool_grants", "generation_cache", "usage_reservation_budgets", "usage_reservations", "usage_reservation_transitions", "session_tiers", "gateway_alert_rules", "gateway_alert_events", "managed_idempotency", "context_pages", "canary_traffic", "semantic_generation_cache", "gateway_batch_files", "gateway_batches", "gateway_batch_items", "hosted_responses", "realtime_tickets", "realtime_sessions", "learned_cooldown", "usage_receipt_keys", "usage_receipt_heads", "usage_receipts", "tenant_organisations", "tenant_teams", "tenant_memberships", "tenant_bindings", "tenant_audit", "tenant_governance_policies", "tenant_governance_reservations", "tenant_governance_components", "tenant_governance_baselines", "tenant_governance_audit", "saml_requests", "saml_subject_links", "saml_audit"]); + "config_snapshot_revisions", "config_snapshots", "config_snapshot_applications", "control_revisions", "tool_grants", "generation_cache", "usage_reservation_budgets", "usage_reservations", "usage_reservation_transitions", "session_tiers", "gateway_alert_rules", "gateway_alert_events", "managed_idempotency", "context_pages", "canary_traffic", "semantic_generation_cache", "gateway_batch_files", "gateway_batches", "gateway_batch_items", "hosted_responses", "realtime_tickets", "realtime_sessions", "learned_cooldown", "usage_receipt_keys", "usage_receipt_heads", "usage_receipts", "tenant_organisations", "tenant_teams", "tenant_memberships", "tenant_bindings", "tenant_audit", "tenant_governance_policies", "tenant_governance_reservations", "tenant_governance_components", "tenant_governance_baselines", "tenant_governance_audit", "saml_requests", "saml_subject_links", "saml_audit", "scim_resources", "scim_group_members", "scim_token_digests", "scim_audit"]); /** * Readiness for the D1 schema. A deploy that skipped `wrangler d1 migrations apply` reports diff --git a/worker/scim-d1.mjs b/worker/scim-d1.mjs new file mode 100644 index 00000000..b1379d7f --- /dev/null +++ b/worker/scim-d1.mjs @@ -0,0 +1,248 @@ +/** Atomic SCIM identity, account, team and security-revision storage. */ +import { boundedBody, USER_FIELDS, validUser } from "./control-users-d1.mjs"; +import { commitRevision, requireRevisionSchema, revisionSyncSettings } from "./config-revision.mjs"; + +const PATH = "/v1/managed-state/scim"; +const ID = /^[a-f0-9]{32}$/; +const ORG = /^[A-Za-z0-9_:.\-]{1,128}$/; +const HASH = /^[a-f0-9]{64}$/; +const VERSION = /^W\/"([1-9][0-9]{0,15})"$/; +const USER_SCHEMA = "urn:ietf:params:scim:schemas:core:2.0:User"; +const GROUP_SCHEMA = "urn:ietf:params:scim:schemas:core:2.0:Group"; +const TABLE_COLUMNS = { + scim_resources: "org_id,kind,id,external_id,user_name,display_name,deactivated,revision,document", + scim_group_members: "org_id,group_id,user_id", + scim_token_digests: "org_id,digest_sha256,updated_at", + scim_audit: "sequence,org_id,kind,resource_id,revision,operation,at", +}; +const object = value => value !== null && typeof value === "object" && !Array.isArray(value); +const exact = (value, names) => object(value) && Object.keys(value).length === names.length && names.every(key => Object.hasOwn(value, key)); +const string = (value, max = 1024) => typeof value === "string" && value.length > 0 && value.length <= max && !/[\x00-\x1f\x7f]/.test(value); +const version = resource => Number(VERSION.exec(resource.meta.version)?.[1]); +let warned = false; + +export function scimEnabled(env) { + const raw = env.SCIM_ENABLED ?? ""; + const flag = typeof raw === "string" ? raw.trim().toLowerCase() : "invalid"; + if (["", "false", "0", "no", "off"].includes(flag)) return false; + if (["true", "1", "yes", "on"].includes(flag)) return true; + if (!warned) { warned = true; console.warn("Invalid SCIM_ENABLED; SCIM provisioning disabled"); } + return false; +} + +const reply = (value, status = 200) => Response.json(status === 200 ? { version: 1, ...value } + : { version: 1, error: { code: value, message: "SCIM storage could not accept the operation" } }, + { status, headers: { "cache-control": "no-store" } }); + +function validResource(resource, kind, expected) { + if (!object(resource) || !ID.test(resource.id ?? "") || !object(resource.meta)) return false; + const attrs = kind === "Users" ? ["userName", "active", "displayName", "externalId", "name", "emails"] + : ["displayName", "externalId", "members"]; + if (Object.keys(resource).some(key => !["schemas", "id", "meta", ...attrs].includes(key)) + || JSON.stringify(resource.schemas) !== JSON.stringify([kind === "Users" ? USER_SCHEMA : GROUP_SCHEMA]) + || Object.keys(resource.meta).some(key => !["resourceType", "version", "created", "lastModified", "location"].includes(key)) + || resource.meta.resourceType !== (kind === "Users" ? "User" : "Group") + || !VERSION.test(resource.meta.version ?? "") || version(resource) !== expected + 1 + || resource.meta.location !== `/scim/v2/${kind}/${resource.id}` + || !["created", "lastModified"].every(key => string(resource.meta[key], 64) && Number.isFinite(Date.parse(resource.meta[key]))) + || ["displayName", "externalId"].some(key => key in resource && !string(resource[key]))) return false; + if (kind === "Groups") return string(resource.displayName) && Array.isArray(resource.members) && resource.members.length <= 1000 + && new Set(resource.members.map(m => m?.value)).size === resource.members.length + && resource.members.every(m => object(m) && ID.test(m.value ?? "") + && Object.keys(m).every(key => ["value", "display", "$ref"].includes(key)) + && ["display", "$ref"].every(key => !(key in m) || string(m[key]))); + if (!string(resource.userName, 128) || resource.userName.trim() !== resource.userName || typeof resource.active !== "boolean") return false; + if ("name" in resource && (!object(resource.name) || Object.keys(resource.name).some(key => + !["formatted", "familyName", "givenName", "middleName", "honorificPrefix", "honorificSuffix"].includes(key)) + || !Object.values(resource.name).every(value => string(value)))) return false; + if ("emails" in resource && (!Array.isArray(resource.emails) || resource.emails.length > 100 + || resource.emails.filter(e => e?.primary === true).length > 1 || !resource.emails.every(e => object(e) && string(e.value) + && Object.keys(e).every(key => ["value", "type", "display", "primary"].includes(key)) + && (!Object.hasOwn(e, "primary") || typeof e.primary === "boolean") + && ["type", "display"].every(key => !(key in e) || string(e[key]))))) return false; + return true; +} + +function validBody(body) { + if (!object(body) || body.version !== 1) return false; + if (body.operation === "probe") return exact(body, ["version", "operation"]); + if (body.operation === "account") return exact(body, ["version", "operation", "username"]) && string(body.username, 128); + if (!ORG.test(body.org_id ?? "") || !["Users", "Groups"].includes(body.kind)) return false; + const base = ["version", "operation", "org_id", "kind"]; + if (body.operation === "get") return exact(body, [...base, "id"]) && ID.test(body.id ?? ""); + if (body.operation === "list") return exact(body, [...base, "attribute", "value", "start", "count"]) + && Number.isInteger(body.start) && body.start >= 1 && body.start <= 999999999 + && Number.isInteger(body.count) && body.count >= 0 && body.count <= 100 + && (body.attribute === null && body.value === null || ["userName", "externalId", "displayName"].includes(body.attribute) && string(body.value)); + return body.operation === "put" && exact(body, [...base, "resource", "expected", "account", "token_digest", "deactivate"]) + && Number.isSafeInteger(body.expected) && body.expected >= 0 && body.expected < 9007199254740991 + && validResource(body.resource, body.kind, body.expected) + && (body.token_digest === null || HASH.test(body.token_digest ?? "")) + && typeof body.deactivate === "boolean" && (!body.deactivate || body.kind === "Groups" && body.resource.members.length === 0) + && (body.account === null || body.kind === "Users" && validUser(body.account) && body.account.is_admin === 0 + && (body.expected !== 0 || body.account.scopes === "chat,models") && body.account.username === body.resource.userName + && (body.resource.active ? body.account.revoked_at === null : body.account.revoked_at !== null)); +} + +async function requireSchema(db) { + if (!db) throw new Error("Storage unavailable"); + // Probe every owned table and all account columns before preparing any account change. + for (const [table, columns] of Object.entries(TABLE_COLUMNS)) await db.prepare(`SELECT ${columns} FROM ${table} LIMIT 0`).all(); + await db.prepare(`SELECT ${USER_FIELDS.join(",")} FROM control_users LIMIT 0`).all(); + await requireRevisionSchema(db); +} + +async function get(db, org, kind, id) { + const row = await db.prepare("SELECT document FROM scim_resources WHERE org_id=? AND kind=? AND id=?") + .bind(org, kind, id).first(); + return row ? JSON.parse(row.document) : null; +} + +async function list(db, body) { + const column = { userName: "user_name", externalId: "external_id", displayName: "display_name" }[body.attribute]; + const where = `org_id=? AND kind=?${column ? ` AND ${column}=?` : ""}`; + const args = [body.org_id, body.kind, ...(column ? [body.value] : [])]; + const count = await db.prepare(`SELECT count(*) AS total FROM scim_resources WHERE ${where}`).bind(...args).first(); + const rows = await db.prepare(`SELECT document FROM scim_resources WHERE ${where} ORDER BY id LIMIT ? OFFSET ?`) + .bind(...args, body.count, body.start - 1).all(); + return reply({ resources: rows.results.map(row => JSON.parse(row.document)), total: count.total }); +} + +function casGuard(db, body) { + return db.prepare(`INSERT INTO scim_resources (org_id,kind,id,revision,document) + SELECT ?,?,?,-1,'' WHERE changes() != 1`).bind(body.org_id, body.kind, body.resource.id); +} + +function resourceStatements(db, body) { + const { org_id: org, kind, resource: row, expected } = body; + const values = [row.externalId ?? null, row.userName ?? null, row.displayName ?? null, expected + 1, JSON.stringify(row), Number(body.deactivate)]; + const statement = expected === 0 + ? db.prepare("INSERT INTO scim_resources (external_id,user_name,display_name,revision,document,deactivated,org_id,kind,id) VALUES (?,?,?,?,?,?,?,?,?)") + .bind(...values, org, kind, row.id) + : db.prepare("UPDATE scim_resources SET external_id=?,user_name=?,display_name=?,revision=?,document=?,deactivated=? WHERE org_id=? AND kind=? AND id=? AND revision=?") + .bind(...values, org, kind, row.id, expected); + return [statement, casGuard(db, body)]; +} + +function accountStatements(db, body, priorAccount) { + if (!body.account) return []; + const { account, expected } = body; + if (expected === 0) return [db.prepare(`INSERT INTO control_users (${USER_FIELDS.join(",")}) VALUES (${USER_FIELDS.map(() => "?").join(",")})`) + .bind(...USER_FIELDS.map(key => account[key]))]; + return [db.prepare(`UPDATE control_users SET api_key_hash=?,api_key_prefix=?,revoked_at=? + WHERE username=? AND is_admin=0 AND api_key_hash=? AND api_key_prefix=? AND revoked_at IS ?`) + .bind(account.api_key_hash, account.api_key_prefix, account.revoked_at, account.username, + priorAccount.api_key_hash, priorAccount.api_key_prefix, priorAccount.revoked_at), casGuard(db, body)]; +} + +async function prepareAccount(db, body, old) { + if (body.kind !== "Users") return null; + const existing = await db.prepare(`SELECT ${USER_FIELDS.join(",")} FROM control_users WHERE username=?`).bind(body.resource.userName).first(); + if (body.expected === 0) { + if (!body.account) throw Object.assign(new Error(), { status: 400 }); + if (existing) throw Object.assign(new Error(), { status: 409 }); + return null; + } + if (!existing || existing.is_admin !== 0 || body.resource.userName !== old.userName) throw Object.assign(new Error(), { status: 400 }); + const changed = body.resource.active !== old.active; + if (changed !== (body.account !== null)) throw Object.assign(new Error(), { status: 400 }); + if (body.account) { + for (const field of USER_FIELDS.filter(key => !["api_key_hash", "api_key_prefix", "revoked_at"].includes(key))) { + if (body.account[field] !== existing[field]) throw Object.assign(new Error(), { status: 412 }); + } + if (!body.resource.active && (body.account.api_key_hash === existing.api_key_hash || body.account.api_key_prefix === existing.api_key_prefix)) + throw Object.assign(new Error(), { status: 400 }); + if (body.resource.active && (body.account.api_key_hash !== existing.api_key_hash || body.account.api_key_prefix !== existing.api_key_prefix)) + throw Object.assign(new Error(), { status: 400 }); + } + return existing; +} + +function collaboratorStatements(callback, data) { + if (typeof callback !== "function") throw new Error("Authority unavailable"); + const result = callback(data); + if (!Array.isArray(result)) throw new Error("Atomic authority statements required"); + return result; +} + +async function put(db, body, collaborators) { + const { org_id: org, kind, resource: row, expected } = body; + const old = await get(db, org, kind, row.id); + if (expected === 0 && row.externalId) { + const existing = await db.prepare("SELECT document FROM scim_resources WHERE org_id=? AND kind=? AND external_id=?") + .bind(org, kind, row.externalId).first(); + if (existing) return reply({ resource: JSON.parse(existing.document), created: false }); + } + if (expected !== 0 && (!old || version(old) !== expected)) return reply("scim_version_conflict", 412); + if (kind === "Users") { + const names = new Set([String(collaborators.adminUsername ?? "admin"), ...(collaborators.adminUsernames ?? [])]); + if (names.has(row.userName)) return reply("scim_admin_denied", 400); + } + const priorAccount = await prepareAccount(db, body, old); + const statements = resourceStatements(db, body); + statements.push(...accountStatements(db, body, priorAccount)); + if (kind === "Users" && expected === 0) { + statements.push(...collaboratorStatements(collaborators.principalStatements, { org_id: org, resource: row })); + } + if (kind === "Groups") { + statements.push(...collaboratorStatements(collaborators.teamStatements, { org_id: org, resource: row, prior: old, deactivated: body.deactivate })); + statements.push(db.prepare("DELETE FROM scim_group_members WHERE org_id=? AND group_id=?").bind(org, row.id)); + for (const member of row.members) { + const user = await get(db, org, "Users", member.value); + if (!user) return reply("scim_foreign_member", 400); + statements.push(db.prepare(`INSERT INTO scim_group_members (org_id,group_id,user_id) + SELECT ?,?,id FROM scim_resources WHERE org_id=? AND kind='Users' AND id=?`) + .bind(org, row.id, org, member.value), casGuard(db, body)); + } + } + const now = new Date().toISOString(); + if (body.token_digest !== null) statements.push(db.prepare(`INSERT INTO scim_token_digests VALUES (?,?,?) + ON CONFLICT(org_id) DO UPDATE SET digest_sha256=excluded.digest_sha256,updated_at=excluded.updated_at`) + .bind(org, body.token_digest, now)); + statements.push(db.prepare("INSERT INTO scim_audit (org_id,kind,resource_id,revision,operation,at) VALUES (?,?,?,?,?,?)") + .bind(org, kind, row.id, expected + 1, row.active === false || body.deactivate ? "deactivate" : expected === 0 ? "create" : "update", now)); + try { + if (!await commitRevision(db, ["key_controls", "model_grants"], statements)) return reply("scim_revision_conflict", 412); + } catch (error) { + const message = String(error?.message ?? error); + if (message.includes("scim_cas_confirmed")) return reply("scim_version_conflict", 412); + if (/UNIQUE constraint/i.test(message)) { + if (expected === 0 && row.externalId) { + const existing = await db.prepare("SELECT document FROM scim_resources WHERE org_id=? AND kind=? AND external_id=?") + .bind(org, kind, row.externalId).first(); + if (existing) return reply({ resource: JSON.parse(existing.document), created: false }); + } + return reply("scim_uniqueness", 409); + } + throw error; + } + return reply({ resource: row, created: expected === 0 }); +} + +export async function handleScimStateRequest(request, env, collaborators = {}) { + if (!scimEnabled(env)) return reply("not_found", 404); + const url = new URL(request.url); + if (url.origin !== "http://intelligence.internal" || url.pathname !== PATH || url.search || url.hash + || url.username || url.password || request.method !== "POST") return reply("not_found", 404); + let body; + try { + if (!/^application\/json(?:;|$)/i.test(request.headers.get("content-type") ?? "")) throw new Error(); + body = JSON.parse(await boundedBody(request, 131072)); + if (!validBody(body)) throw new Error(); + } catch { return reply("scim_invalid_request", 400); } + try { + if (!revisionSyncSettings(env).enabled) return reply("scim_security_sync_required", 503); + const db = env.INTELLIGENCE_DB; + await requireSchema(db); + if (body.operation === "probe") return reply({ ready: true }); + if (body.operation === "get") return reply({ resource: await get(db, body.org_id, body.kind, body.id) }); + if (body.operation === "list") return await list(db, body); + if (body.operation === "account") return reply({ account: await db.prepare(`SELECT ${USER_FIELDS.join(",")} FROM control_users WHERE username=?`) + .bind(body.username).first() }); + return await put(db, body, { ...collaborators, adminUsername: String(env.ADMIN_USERNAME || "admin").trim(), + adminUsernames: String(env.ADMIN_USERNAMES || "").split(",").map(name => name.trim()) }); + } catch (error) { + const status = [400, 409, 412].includes(error?.status) ? error.status : 503; + return reply(status === 503 ? "scim_storage_unavailable" : "scim_operation_rejected", status); + } +}