Skip to content

REGBACKLOG-336: manage DEV storage and eventing in application workflows - #1790

Open
Jacky-Pham wants to merge 7 commits into
mainfrom
Jacky/regbacklog-336-terraform
Open

REGBACKLOG-336: manage DEV storage and eventing in application workflows#1790
Jacky-Pham wants to merge 7 commits into
mainfrom
Jacky/regbacklog-336-terraform

Conversation

@Jacky-Pham

@Jacky-Pham Jacky-Pham commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

STRR's DEV buckets and eventing resources currently sit outside the application deployment workflows. This change gives uploads, email, and bulk validation separate Terraform roots and state prefixes, and runs the relevant root automatically before protected-main DEV deployments.

The roots adopt 15 existing resources: the three core buckets, email messaging/DLQ resources, and the validation trigger and callback/DLQ resources. Both validation workflows share one queued execution group and state. Existing bucket access protections, recovery settings, endpoints, and delivery policies are preserved; Eventarc retains ownership of its generated transport resources.

The first plan for each root must contain only unchanged imports (API 1, email 6, validation 8). Later automatic plans allow DEV creation/updates but reject deletion/replacement. The workflow applies the exact saved plan checked by the policy. Infrastructure failures block app deployment, and DEV-only file changes cannot trigger TEST/PROD promotion. Other environments remain outside Terraform's scope.

Dependency before merge

bcregistry-sre #399 must be reviewed and applied through the SRE dev workspace first. It grants the infrastructure identity only metadata get/update on the three existing DEV buckets. The original WIF/state bootstrap PRs #386/#387 are already merged. Merging this STRR PR starts automatic DEV adoption.

Security review before merge

Sonar's quality gate remains blocked by six findings on the existing bucket configuration: three for logging (terraform:S6258) and three for Object Versioning (terraform:S6412). These definitions preserve current settings for unchanged imports. No finding or rule has been suppressed or dismissed.

  • All three buckets already retain deleted objects for seven days. Google recommends soft delete for deletion protection; review the versioning findings against that existing control.
  • Bucket access logging is absent. Google recommends Cloud Audit Logs in most cases, but this project's IAM policy has no project-level auditConfigs and the current identity cannot read inherited policy. SRE needs to verify effective Cloud Storage Data Access logging; its presence is not established by these tests.

Jacky and Jimmy should review these existing settings and decide the findings' disposition before merge. Any required hardening needs an explicit rollout that preserves the initial import-only policy. The plan-checker path-handling finding has been fixed by consuming Terraform JSON through standard input.

Validation

  • All three roots passed Terraform formatting, backend-disabled initialization, validation, and mock-provider tests.
  • 24 Python tests passed for actual workflow conditions, path detection against Git histories, the plan policy, and its standard-input CLI.
  • Final-head Terraform CI and CodeQL passed. Sonar confirms all script/test findings resolved; only the six bucket-setting findings above remain.
  • Live read-only email import plan: 6 imports, 0 add/change/destroy. This used real GCP APIs and a temporary local backend; no state was written or infrastructure applied.
  • Google 6.50.0 provider import plans against recorded GCP GET metadata: all 15 imports, 0 add/change/destroy. This also caught and corrected subscription-IAM replacement behavior.
  • Actionlint passed apart from its outdated schema for concurrency.queue; only that diagnostic was suppressed. queue: max is supported by current GitHub documentation.
  • PR CI runs without cloud credentials. No merge or apply was performed.

A full live API/validation plan remains dependent on the new bucket permissions; recorded-metadata tests are not a substitute for that live permission/drift check. The runtime workflow enforces the import-only requirement before its first apply. The state bucket was empty when inspected on September 9; verify no other state has adopted these resources before rollout.

Ticket: REGBACKLOG-336.

Comment thread .github/workflows/strr-terraform.yaml Fixed
Comment thread .github/workflows/strr-terraform.yaml Fixed
@Jacky-Pham
Jacky-Pham marked this pull request as ready for review August 25, 2026 21:02
Comment thread .github/workflows/strr-terraform.yaml Outdated
@Jacky-Pham

Copy link
Copy Markdown
Collaborator Author

hey Jimmy, the SRE IAM changes are merged and applied now. when you have a chance can you review this so I can run the terraform plan?

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds dev-only Terraform management for existing STRR Pub/Sub and Eventarc infrastructure.

Changes:

  • Defines and imports existing dev resources.
  • Configures remote state and keyless authentication.
  • Adds manual Terraform plan/apply automation.
File summaries
File Description
terraform/versions.tf Configures Terraform, provider, and backend.
terraform/variables.tf Defines dev infrastructure inputs.
terraform/README.md Documents scope and rollout.
terraform/pubsub.tf Manages and imports Pub/Sub resources.
terraform/eventarc.tf Manages and imports the Eventarc trigger.
terraform/dev.tfvars Supplies dev environment values.
terraform/data.tf Resolves the Pub/Sub service agent.
terraform/.terraform.lock.hcl Locks the Google provider.
.gitignore Excludes Terraform working directories.
.github/workflows/strr-terraform.yaml Adds manual plan/apply automation.
Review details

Files not reviewed (1)

  • terraform/.terraform.lock.hcl: Generated file
  • Files reviewed: 8/10 changed files
  • Comments generated: 1
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/workflows/strr-terraform.yaml Outdated
Comment on lines +69 to +71
- name: Terraform apply
if: inputs.action == 'apply'
run: terraform apply -input=false -auto-approve tfplan
@Jacky-Pham Jacky-Pham changed the title REGBACKLOG-336: manage STRR dev infrastructure with Terraform REGBACKLOG-336: manage DEV storage and eventing in application workflows Sep 9, 2026
Comment thread terraform/scripts/check_plan.py Fixed
Comment thread terraform/api/storage.tf
@@ -0,0 +1,22 @@
resource "google_storage_bucket" "registration_documents" {
@@ -0,0 +1,45 @@
resource "google_storage_bucket" "bulk_validation_requests" {
}
}

resource "google_storage_bucket" "bulk_validation_responses" {
Comment thread terraform/api/storage.tf
@@ -0,0 +1,22 @@
resource "google_storage_bucket" "registration_documents" {
@@ -0,0 +1,45 @@
resource "google_storage_bucket" "bulk_validation_requests" {
}
}

resource "google_storage_bucket" "bulk_validation_responses" {
@sonarqubecloud

sonarqubecloud Bot commented Sep 9, 2026

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
C Security Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants