-
Notifications
You must be signed in to change notification settings - Fork 0
feat(infra): add Terraform kind cluster and operator deployment module #13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| name: Terraform Lint | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - "infra/**" | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| terraform-lint: | ||
| name: fmt / tflint / trivy | ||
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Terraform | ||
| uses: hashicorp/setup-terraform@v3 | ||
| with: | ||
| terraform_version: "1.9.5" | ||
|
|
||
| - name: Terraform Format Check | ||
| run: terraform fmt -check -recursive infra/ | ||
|
|
||
| - name: Set up TFLint | ||
| uses: terraform-linters/setup-tflint@v4 | ||
| with: | ||
| tflint_version: "v0.53.0" | ||
|
|
||
| - name: TFLint init | ||
| run: tflint --init --chdir=infra/environments/dev | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: TFLint | ||
| run: tflint --chdir=infra/environments/dev | ||
|
|
||
| - name: Trivy IaC Scan | ||
| uses: aquasecurity/trivy-action@master | ||
| with: | ||
| scan-type: config | ||
| scan-ref: infra/ | ||
| exit-code: "1" | ||
| severity: HIGH,CRITICAL | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -28,3 +28,16 @@ go.work | |
|
|
||
| # Temporary design scratch pad | ||
| rough/ | ||
|
|
||
| # Terraform | ||
| **/.terraform/ | ||
| *.tfstate | ||
| *.tfstate.* | ||
| crash.log | ||
| crash.*.log | ||
| override.tf | ||
| override.tf.json | ||
| *_override.tf | ||
| *_override.tf.json | ||
| *-config | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win Narrow the This unanchored pattern ignores every file ending in 🤖 Prompt for AI Agents |
||
| *kubeconfig* | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -168,6 +168,26 @@ check-metrics: ## Diagnostic: check whether custom metrics are available via the | |
| $(KUBECTL) get --raw /apis/custom.metrics.k8s.io/v1beta1 2>/dev/null | python3 -m json.tool || \ | ||
| echo "Custom metrics API not available — ensure Prometheus Adapter is running." | ||
|
|
||
| ##@ Terraform (Infrastructure) | ||
|
|
||
| TF_DIR ?= infra/environments/dev | ||
|
|
||
| .PHONY: terraform-init | ||
| terraform-init: ## Initialise Terraform in the dev environment (downloads providers). | ||
| terraform -chdir=$(TF_DIR) init | ||
|
|
||
| .PHONY: terraform-plan | ||
| terraform-plan: ## Preview Terraform changes for the dev kind cluster. | ||
| terraform -chdir=$(TF_DIR) plan | ||
|
|
||
| .PHONY: terraform-apply | ||
| terraform-apply: ## Provision the dev kind cluster and deploy the full Agentrax stack. | ||
| terraform -chdir=$(TF_DIR) apply -auto-approve | ||
|
|
||
| .PHONY: terraform-destroy | ||
| terraform-destroy: ## Tear down the dev kind cluster and all provisioned resources. | ||
| terraform -chdir=$(TF_DIR) destroy -auto-approve | ||
|
Comment on lines
+183
to
+189
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- Makefile targets ---'
sed -n '160,195p' Makefile
printf '%s\n' '--- TF_DIR references ---'
rg -n -C 3 'TF_DIR|terraform-(init|plan|apply|destroy)' Makefile docs README.md .github 2>/dev/null || true
printf '%s\n' '--- Terraform environment directories ---'
find . -maxdepth 4 -type f \( -name '*.tf' -o -name '*.tfvars' \) -print | sortRepository: gitcommitankit/agentrax Length of output: 3788 🏁 Script executed: #!/bin/bash
set -eu
python3 - <<'PY'
from pathlib import Path
import re
makefile = Path("Makefile").read_text()
default = re.search(r"^TF_DIR\s*\?\=\s*(.+)$", makefile, re.MULTILINE).group(1).strip()
recipes = {}
for target in ("terraform-apply", "terraform-destroy"):
match = re.search(
rf"^{target}:.*\n\t([^\n]+)$",
makefile,
re.MULTILINE,
)
recipes[target] = match.group(1) if match else None
print(f"default TF_DIR: {default}")
print(f"prod directory exists: {Path('infra/environments/prod').is_dir()}")
for target, recipe in recipes.items():
print(f"{target} default command: {recipe}")
print(f"{target} overridden command: {recipe.replace('$(TF_DIR)', 'infra/environments/prod')}")
print(f"{target} auto-approve: {'-auto-approve' in recipe}")
PYRepository: gitcommitankit/agentrax Length of output: 662 Do not auto-approve an overridable Terraform directory.
🤖 Prompt for AI Agents |
||
|
|
||
| ##@ Dependencies | ||
|
|
||
| ## Location to install dependencies to | ||
|
|
||
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| # tflint configuration for the infra/ directory. | ||
| # Enables the official Terraform plugin for provider-level schema validation. | ||
| # Run: tflint --chdir=infra/environments/dev | ||
|
|
||
| plugin "terraform" { | ||
| enabled = true | ||
| preset = "recommended" | ||
| } | ||
|
|
||
| # Enforce consistent code style. | ||
| rule "terraform_naming_convention" { | ||
| enabled = true | ||
| } | ||
|
|
||
| rule "terraform_required_version" { | ||
| enabled = true | ||
| } | ||
|
|
||
| rule "terraform_required_providers" { | ||
| enabled = true | ||
| } | ||
|
|
||
| rule "terraform_documented_variables" { | ||
| enabled = true | ||
| } | ||
|
|
||
| rule "terraform_documented_outputs" { | ||
| enabled = true | ||
| } |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| # dev environment — local kind cluster + agentrax stack | ||
| # This is the primary target for local development, integration testing, and CI. | ||
| # State is stored in a local backend file (terraform.tfstate) — not shared. | ||
|
|
||
| terraform { | ||
| required_version = ">= 1.6" | ||
|
|
||
| # Local backend — intentional for dev. Do not check in terraform.tfstate. | ||
| backend "local" {} | ||
|
|
||
| required_providers { | ||
| kind = { | ||
| source = "tehcyx/kind" | ||
| version = "~> 0.6" | ||
| } | ||
| helm = { | ||
| source = "hashicorp/helm" | ||
| version = "~> 2.14" | ||
| } | ||
| kubernetes = { | ||
| source = "hashicorp/kubernetes" | ||
| version = "~> 2.31" | ||
| } | ||
| } | ||
| } | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # Step 1: Provision the kind cluster | ||
| # --------------------------------------------------------------------------- | ||
| module "kind_cluster" { | ||
| source = "../../modules/kind_cluster" | ||
| cluster_name = var.cluster_name | ||
| } | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # Step 2: Configure the Helm and Kubernetes providers to target the new cluster. | ||
| # Both providers read credentials from the kind_cluster module outputs so no | ||
| # local kubeconfig file needs to exist before `terraform apply`. | ||
| # --------------------------------------------------------------------------- | ||
| provider "helm" { | ||
| kubernetes { | ||
| host = module.kind_cluster.endpoint | ||
| client_certificate = module.kind_cluster.client_certificate | ||
| client_key = module.kind_cluster.client_key | ||
| cluster_ca_certificate = module.kind_cluster.cluster_ca_certificate | ||
| } | ||
| } | ||
|
|
||
| provider "kubernetes" { | ||
| host = module.kind_cluster.endpoint | ||
| client_certificate = module.kind_cluster.client_certificate | ||
| client_key = module.kind_cluster.client_key | ||
| cluster_ca_certificate = module.kind_cluster.cluster_ca_certificate | ||
| } | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # Step 3: Install cert-manager → kube-prometheus-stack → agentrax | ||
| # --------------------------------------------------------------------------- | ||
| module "agentrax_stack" { | ||
| source = "../../modules/agentrax_stack" | ||
|
|
||
| cert_manager_version = var.cert_manager_version | ||
| prometheus_stack_version = var.prometheus_stack_version | ||
| agentrax_chart_path = var.agentrax_chart_path | ||
| agentrax_leader_elect = var.agentrax_leader_elect | ||
| agentrax_extra_values = var.agentrax_extra_values | ||
|
|
||
| # The stack module requires the cluster to exist first. | ||
| # Provider-level dependency is enforced via the shared kubeconfig above. | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| # dev environment — outputs | ||
|
|
||
| output "cluster_endpoint" { | ||
| description = "Kubernetes API server endpoint for the dev kind cluster." | ||
| value = module.kind_cluster.endpoint | ||
| } | ||
|
|
||
| output "agentrax_namespace" { | ||
| description = "Namespace where the agentrax operator was deployed." | ||
| value = module.agentrax_stack.agentrax_namespace | ||
| } | ||
|
|
||
| output "agentrax_release_status" { | ||
| description = "Helm release status for the agentrax chart." | ||
| value = module.agentrax_stack.agentrax_release_status | ||
| } | ||
|
|
||
| output "prometheus_namespace" { | ||
| description = "Namespace where kube-prometheus-stack is deployed." | ||
| value = module.agentrax_stack.prometheus_namespace | ||
| } | ||
|
|
||
| output "kubeconfig" { | ||
| description = "Raw kubeconfig for the kind cluster. Pipe into kubectl or save to a file." | ||
| value = module.kind_cluster.kubeconfig | ||
| sensitive = true | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| # dev environment — variables | ||
|
|
||
| variable "cluster_name" { | ||
| description = "Name of the local kind cluster." | ||
| type = string | ||
| default = "agentrax-dev" | ||
| } | ||
|
|
||
| variable "cert_manager_version" { | ||
| description = "cert-manager Helm chart version." | ||
| type = string | ||
| default = "v1.15.3" | ||
| } | ||
|
|
||
| variable "prometheus_stack_version" { | ||
| description = "kube-prometheus-stack Helm chart version." | ||
| type = string | ||
| default = "61.8.0" | ||
| } | ||
|
|
||
| variable "agentrax_chart_path" { | ||
| description = "Path to the agentrax Helm chart directory, relative to this environment root." | ||
| type = string | ||
| default = "../../../charts/agentrax" | ||
| } | ||
|
|
||
| variable "agentrax_leader_elect" { | ||
| description = "Enable leader election for the Agentrax controller manager." | ||
| type = bool | ||
| default = false | ||
| } | ||
|
|
||
| variable "agentrax_extra_values" { | ||
| description = "Additional Helm set key=value overrides for the agentrax release." | ||
| type = map(string) | ||
| default = {} | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| # Production Environment — Azure AKS (Stub) | ||
|
|
||
| This environment targets an Azure Kubernetes Service (AKS) cluster for production workloads. The `agentrax_stack` module is cloud-agnostic; the production configuration differs from `dev` in four ways: | ||
|
|
||
| 1. **Remote State Backend**: State stored in Azure Blob Storage with state locking. | ||
| 2. **Cloud Provider Authentication**: Uses `azurerm` / `azapi` providers authenticated via Azure OIDC / Workload Identity. | ||
| 3. **High Availability**: `agentrax_leader_elect = true` with $\ge 2$ controller replicas. | ||
| 4. **Workload Identity**: Cloud identity parameters passed via `agentrax_extra_values`. | ||
|
|
||
| ### Activation Runbook | ||
|
|
||
| 1. Provision the target AKS cluster and retrieve its kubeconfig credentials. | ||
| 2. Configure `backend.tf` with the Azure Blob Storage container coordinates. | ||
| 3. Export Azure authentication environment variables (`ARM_CLIENT_ID`, `ARM_TENANT_ID`, `ARM_SUBSCRIPTION_ID`, `ARM_USE_OIDC=true`). | ||
| 4. Execute deployment: | ||
| ```bash | ||
| terraform -chdir=infra/environments/prod init | ||
| terraform -chdir=infra/environments/prod apply | ||
|
Comment on lines
+10
to
+18
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- relevant files ---'
git ls-files \
infra/environments/prod/README.md \
infra/environments/prod \
infra/environments/dev \
docs/ARCHITECTURE.md \
.agents/skills/agentrax-context/SKILL.md \
Makefile
printf '%s\n' '--- production environment tree ---'
find infra/environments/prod -maxdepth 2 -type f -print 2>/dev/null | sort
printf '%s\n' '--- production README ---'
cat -n infra/environments/prod/README.md
printf '%s\n' '--- architecture references ---'
sed -n '375,390p;418,432p' docs/ARCHITECTURE.md
printf '%s\n' '--- provisioning-boundary context ---'
sed -n '1,45p' .agents/skills/agentrax-context/SKILL.md
printf '%s\n' '--- Terraform roots and legacy provisioning entry points ---'
find infra -type f \( -name '*.tf' -o -name '*.tf.json' \) -print | sort
rg -n --glob '!**/.git/**' \
'terraform-(apply|init)|TF_DIR|provision|provisioning|legacy|environments/(dev|prod)' \
Makefile infra docs .agents 2>/dev/null | head -250Repository: gitcommitankit/agentrax Length of output: 11040 🏁 Script executed: #!/bin/bash
set -eu
python3 - <<'PY'
from pathlib import Path
import re
prod = Path("infra/environments/prod")
prod_tf = sorted(p.name for p in prod.glob("*.tf"))
prod_tf_json = sorted(p.name for p in prod.glob("*.tf.json"))
readme = Path("infra/environments/prod/README.md").read_text()
architecture = Path("docs/ARCHITECTURE.md").read_text()
makefile = Path("Makefile").read_text()
print({
"prod_exists": prod.is_dir(),
"prod_root_tf_files": prod_tf,
"prod_root_tf_json_files": prod_tf_json,
"readme_marks_stub": "Stub" in readme.splitlines()[0],
"readme_runs_prod_init": "terraform -chdir=infra/environments/prod init" in readme,
"readme_runs_prod_apply": "terraform -chdir=infra/environments/prod apply" in readme,
"architecture_advertises_prod_override": "TF_DIR=infra/environments/prod" in architecture,
})
tf_dir = re.search(r"^TF_DIR \?= (.+)$", makefile, re.MULTILINE)
apply = re.search(
r"^terraform-apply:.*\n\t(terraform -chdir=)\$\(TF_DIR\) apply",
makefile,
re.MULTILINE,
)
print({
"make_default_tf_dir": tf_dir.group(1) if tf_dir else None,
"make_apply_uses_tf_dir": bool(apply),
"prod_apply_target_resolves_to": (
f"{apply.group(1)}infra/environments/prod apply"
if apply else None
),
})
PYRepository: gitcommitankit/agentrax Length of output: 542 Keep production Terraform documentation consistent with the stub implementation.
🧰 Tools🪛 markdownlint-cli2 (0.23.2)[warning] 10-10: Heading levels should only increment by one level at a time (MD001, heading-increment) [warning] 16-16: Fenced code blocks should be surrounded by blank lines (MD031, blanks-around-fences) 📍 Affects 2 files
🤖 Prompt for AI Agents |
||
| ``` | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
Complete and harden the Terraform CI workflow.
--config="$GITHUB_WORKSPACE/infra/.tflint.hcl"to both TFLint commands because changing intoinfra/environments/devprevents discovery of the parent configuration.make deploy-depscurrently installs only cert-manager, the Prometheus Operator bundle, and Gateway API CRDs.aquasecurity/trivy-action@masterwith a reviewed release tag or full commit SHA.📍 Affects 1 file
.github/workflows/terraform-lint.yml#L11-L47(this comment).github/workflows/terraform-lint.yml#L33-L39.github/workflows/terraform-lint.yml#L41-L42🤖 Prompt for AI Agents
Source: Path instructions