Skip to content

feat(google): regenerate SDK 585 options with credential masking - #5347

Closed
thomhurst wants to merge 1 commit into
mainfrom
maintenance/gcloud-options-embedded-key-masking-20260922
Closed

thomhurst wants to merge 1 commit into
mainfrom
maintenance/gcloud-options-embedded-key-masking-20260922

Conversation

@thomhurst

@thomhurst thomhurst commented Sep 22, 2026 •

Copy link
Copy Markdown
Owner

Google Cloud SDK 585.0.0 adds 24 commands. This refresh applies the unchanged latest-main workflow artifact after generator fixes through #5346. Resource constructors, enums, list delimiters, provider requirements, secret metadata, and nested argument choices follow current CLI output. Explicit GKE dependencies and nested Kafka exclusions remain enforced while documented optional members stay optional.

Oracle Goldengate embedded service-account and private-key contents now receive SecretValue metadata, even though the switches end in -file. Compared with rejected #5344, generated output changes only those eight annotations and the generator fingerprint; no secret annotation is removed. All eight affected properties have handwritten runtime attribute checks.

Seven handwritten Google test files adapt four required resource constructor calls and exercise uploads, resource/service contracts, composite values, delimiters, enums, providers, secret metadata, SQL/authentication groups, optional resource selectors, and TLS bundles. The nested-choice suite retains 76 cases for Storage, OAuth, Cloud Build, Cluster Director, Composer, GKE, and Kafka. No compatibility shims or generated API baselines are added.

Replaces closed #5344, #5341, #5338, #5333, #5315, and #5179; retains runtime coverage from #4863, #4869, and #4956. Acceptance issues: #5345, #5339, #5180, #5178, #5166, #5146, #4850, #4849, #4799, #4797, and #4474.

Generation: https://github.com/thomhurst/ModularPipelines/actions/runs/35690354313 from main ca572f7. Artifact 10678711892 was applied unchanged and passes reverse-application and latest-main freshness checks. Coverage contains 7,951 commands and 1,455 groups, adds 24 commands, removes none, and retains all 54 commands reported in #5146 with no exclusions. Generator fingerprint: df509247582dc887b6741d67885abf2b86e7e78eebf77f7e164effbb92e4ee47.

The generated Google library builds in the workflow; its only failures are the four handwritten constructor calls adapted here. Formatting and the documentation build (331 documents) pass. Source prerequisite #5346 passes all 3,402 generator tests and all checks/reviews.

Local broad validation previously reached the mandated 2 GB guard and was not retried. Actual Google integration execution in an explicit branch CI run and PR reviews remain required before merge. Acceptance issues stay open until those pass.

Summary by CodeRabbit

  • Documentation
    • Updated the gcloud CLI command reference with option mappings for:
      • gcloud app-topology
      • gcloud compute image-views
      • Memorystore token-auth user management commands

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-22T06:07:02.763202Z 5a12a85 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉


📝 Walkthrough

Walkthrough

The gcloud command reference adds mappings for App Topology, Compute Image Views, and Memorystore token authentication commands. Each mapping links a CLI command to its corresponding options type.

Changes

gcloud command reference

Layer / File(s) Summary
App Topology command mappings
docs/docs/mp-packages/cli/gcloud.md
Adds eleven App Topology command-to-options mappings.
Compute Image Views command mappings
docs/docs/mp-packages/cli/gcloud.md
Adds three Compute Image Views command-to-options mappings.
Memorystore token authentication mappings
docs/docs/mp-packages/cli/gcloud.md
Adds mappings for Memorystore token authentication commands and their options types.

Priority: ➖ Normal

Estimated code review effort: 1 (Trivial) | ~5 minutes

Merge Risk: ⚪ Minimal · up to 5a12a

This documentation update adds command mappings for newly supported Google Cloud commands without identified runtime or compatibility risk.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Linked Issues check ⚠️ Warning The implementation satisfies the coding objectives in #5344: it adds 24 commands with no removals, applies the generated SDK 585.0.0 artifact, adapts four handwritten constructor calls, and adds the s… Run the explicit Google integration test workflow for the reviewed head and resolve any failures before merge.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: regenerating Google SDK 585 options with credential masking. It is concise and specific.
Out of Scope Changes check ✅ Passed The reviewed changes stay within #5344. The generated SDK refresh, handwritten constructor updates, focused tests, and documentation mappings for the new gcloud command families all support the linked…
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Full details: Linked Issues check

Explanation

The implementation satisfies the coding objectives in #5344: it adds 24 commands with no removals, applies the generated SDK 585.0.0 artifact, adapts four handwritten constructor calls, and adds the stated runtime coverage. The workflow reports successful build, formatting, documentation generation, freshness, and reverse-application checks. The required explicit Google integration execution remains pending, so the automated validation requirement is not complete. The excluded generated files do not provide evidence of a defect because the exclusion is intentional.

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

A rabbit checks the command page,
New mappings hop across each stage,
Topology, images, tokens align,
Options types now clearly shine,
The reference grows one row at a time.

Comment @coderabbitai help to get the list of available commands.

@thomhurst

Copy link
Copy Markdown
Owner Author

Claude's review publication failed validation because its evidence included review-context/pull-request.json, which is not a changed source file. The review payload contained no code findings, but that unpublished result is not being treated as a completed review. I have rerun the failed review job on the unchanged head 5a12a85.

Explicit integration CI continues at https://github.com/thomhurst/ModularPipelines/actions/runs/35692851347. Merge still requires actual Google test execution and completed reviews.

@chatgpt-codex-connector

Copy link
Copy Markdown

💡 Codex Review

https://github.com/thomhurst/ModularPipelines/blob/5a12a85c015a639c916c7356cf40138d6efe0667/src/ModularPipelines.Google/Options/GcloudOracleDatabaseGoldengateConnectionsCreateOptions.Generated.cs#L1451-L1452
P1 Badge Mark inline client private keys as secrets

When a Goldengate MySQL connection uses two-way TLS, this option contains the base64-encoded private key itself, not a file path, but the generated property lacks [SecretValue]. Because SecretProvider discovers option secrets from that attribute, command logging can expose the private key; update the generator’s inline-content detection and regenerate rather than editing this generated file directly.

AGENTS.md reference: AGENTS.md:L33-L34



P2 Badge Stop masking OAuth credential resource IDs

When gcloud iam oauth-clients credentials describe is given its required credential resource ID, this annotation registers that ordinary identifier as secret material even though the generated help explicitly describes it as an ID or fully qualified resource identifier. It will consequently be redacted from command and subsequent pipeline logs, obscuring which resource was operated on; teach the generator’s resource-identifier detection to handle multi-word resource names and regenerate the options.

AGENTS.md reference: AGENTS.md:L33-L34

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Review

This PR applies an unchanged, already-approved generator artifact refresh for Google Cloud SDK 585 (24 new gcloud commands, enum/record renames, nested-choice validation) plus doc-table additions and seven adapted/added handwritten test files; it also carries forward eight new [SecretValue] annotations for Oracle GoldenGate embedded service-account/private-key file contents. I verified the secret-masking additions target genuinely embedded credential content (not paths), confirmed related [SecretValue] removals only affect Secret Manager resource-path references rather than raw secret values, checked that adapted test constructor calls match the newly required positional arguments in the regenerated records, and confirmed the diff stays fully scoped to docs/src/test paths for the Google package with no PublicAPI baseline files touched.

Review evidence

docs/docs/mp-packages/cli/gcloud.md: Verified the 24 new command-mapping rows (App Topology, Compute Image Views, Memorystore token-auth users) are alphabetically placed and each maps to an options type that exists in the corresponding new Generated.cs file added under src/ModularPipelines.Google/Options.

src/ModularPipelines.Google/Options/GcloudOracleDatabaseGoldengateConnectionsCreateOptions.Generated.cs: Confirmed exactly the eight embedded credential-file properties (e.g. GoogleBigQueryConnectionPropertiesServiceAccountKeyFile, OciObjectStorageConnectionPropertiesPrivateKeyFile) gained [SecretValue], and their doc comments explicitly describe 'base64 encoded content of the ... key file', justifying masking despite the misleading '-file' suffix.

src/ModularPipelines.Google/Options/GcloudDeveloperConnectConnectionsCreateOptions.Generated.cs: Checked the removed [SecretValue] annotations (e.g. BitbucketDataCenterConfigWebhookSecretVersion, GithubEnterpriseConfigPrivateKeySecretVersion) and confirmed these properties hold Secret Manager resource-path references (projects//secrets//versions/*), not raw secret content, so dropping the mask is a reasonable, non-regressive refinement rather than a leak.

test/ModularPipelines.Google.UnitTests/GcloudRegenerationTests.cs: Verified this new test file asserts, via reflection, that exactly the eight embedded-credential properties carry SecretValueAttribute while secret-version/resource-identifier properties (e.g. DatabaseAdminPasswordSecretVersion, AuthToken) do not, matching the PR's stated scope of the masking change.

test/ModularPipelines.Google.UnitTests/GcloudCompositeValueTests.cs: Confirmed the adapted constructor calls (e.g. GcloudBmsNfsSharesUpdateOptions("nfs-share")) and updated expected-argument lists correctly reflect the new required positional resource arguments introduced by the regenerated records.

test/ModularPipelines.Google.UnitTests/GcloudResourceArgumentTests.cs: Checked reflection-based tests confirming resource-scoped service methods require the expected options type/non-optional parameter and that fully-qualified resource strings render without extra selector options, and that null resource arguments throw via constructor guard clauses.

No actionable findings.

Optional follow-up notes

This diff touches 7,722 files, all confined to docs/docs/mp-packages/cli/gcloud.md, src/ModularPipelines.Google/, and test/ModularPipelines.Google*/ — consistent with the PR description's claim of a pure regenerated-artifact refresh plus adapted tests, with no scope creep into unrelated packages or PublicAPI baseline files.

Given the size of the diff, verification was done by sampling representative generated files (new commands, renamed enums, the eight secret-masking annotations) and all seven new/modified handwritten test files rather than exhaustively diffing every one of the ~7,700 changed files; broader correctness relies on the CI build/test/format/freshness results already cited in the PR description.

Design observation (non-blocking): the generator now treats 'SecretVersion' fields (Secret Manager resource-path references) as non-secret while treating embedded '-file' fields containing base64 key material as secret. This is a sensible content-vs-reference distinction, but since it is a generator classification policy rather than a defect in this PR, any future disagreement about masking resource paths that name specific secrets should be raised against the generator in tools/ModularPipelines.OptionsGenerator per the repo's PR-maintenance rules, not patched here.

A prior review attempt for this PR was rejected because its evidence cited review-context/pull-request.json, a non-source file; this review's evidence list only cites files present in the actual diff.

@thomhurst

Copy link
Copy Markdown
Owner Author

Closing defective generated output for the two confirmed classifier findings in #5347 (comment). Inline TLS key contents are tracked in #5345; multi-word credential resource IDs are tracked in #5180. Source fixes and broader captured regressions will precede fresh latest-main regeneration. All pending CI, including the manual full pipeline, is being cancelled.

@thomhurst

Copy link
Copy Markdown
Owner Author

Source fix: #5348 (0f31d5b623). It covers both findings from the Codex review, plus the same inline-secret classification gap in PostgreSQL private keys, four KeyStore bodies, and Oracle wallet contents. The focused generator suite passes all 303 tests, including non-secret certificates, TrustStore bodies, configuration bodies, and actual paths.

This generated PR remains closed. Its queued/running work is now cancelled; all five workflow runs are completed. A fresh Google generation will follow after the source fix passes review and merges into main.

@thomhurst

Copy link
Copy Markdown
Owner Author

Source fix #5348 is merged as a94af5a6379aa097b0828d9584323787c9fc6449. All 3,434 generator tests pass, with clear Claude, Codex, and CodeRabbit reviews and successful CodeQL checks.

Fresh Google generation from that main commit: https://github.com/thomhurst/ModularPipelines/actions/runs/35696052122. Acceptance remains open until the generated replacement passes its artifact audit and actual Google integration tests.

@thomhurst

Copy link
Copy Markdown
Owner Author

Fresh replacement: #5349, generated by https://github.com/thomhurst/ModularPipelines/actions/runs/35696052122 from main a94af5a6379aa097b0828d9584323787c9fc6449 after source fix #5348.

The unchanged artifact preserves all 7,951 commands, including all 54 reported commands, with no removals or exclusions. Compared with #5347, exactly seven inline-secret annotations are added and eight OAuth/SCIM resource-ID annotations are removed. All 15 embedded credential fields are now marked as secrets. Documentation builds successfully (331 documents).

The generated library compiles. The replacement adapts the four existing handwritten test constructor calls and adds focused Google regressions. Actual Google integration validation is running in https://github.com/thomhurst/ModularPipelines/actions/runs/35698901283. Acceptance remains open until CI and review finish.

This branch had an error being deployed

1 failed deployment
Pull Requests — 5a12a85c Deployed Sep 22, 2026 by thomhurst via full pipeline (ubuntu-latest) #17032
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant