Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions .github/workflows/automation-control-plane.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: Automation control plane

on:
pull_request:
paths:
- .github/workflows/automation-control-plane.yml
- AUTOMATION.md
- automation/**
push:
branches: [main]
paths:
- .github/workflows/automation-control-plane.yml
- AUTOMATION.md
- automation/**

permissions:
contents: read

concurrency:
group: automation-control-plane-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
contracts:
name: Validate automation contracts
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Check out repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Run control-plane tests
run: automation/scripts/registry.test.sh
19 changes: 17 additions & 2 deletions AUTOMATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,13 @@ evidence location, CI workflow (when present), acceptance statement, privacy
boundary, and explicit coverage gap. These fields guide discovery; the product
repository and immutable run artifacts remain the source of truth.

The desired one-environment/one-repository mapping for Slack-driven Codex work
lives in [`automation/codex-targets.yml`](automation/codex-targets.yml). A
target marked `pending_verification` or `paused` is not runnable. Every request
must pass the fail-closed
[`automation/codex-task-contract.md`](automation/codex-task-contract.md) before
an operator mentions `@Codex`.

## Operator entry points

- Run `automation/scripts/portfolio-health.sh` for a read-only JSON Lines
Expand All @@ -74,8 +81,16 @@ repository and immutable run artifacts remain the source of truth.
shell (including shell process substitution); scan failures and matches are
configuration drift rather than a silent clean result.
- Run `automation/scripts/registry.test.sh` after changing repository
lifecycle or Eval metadata. Every active product must keep a complete,
machine-readable Eval discovery contract.
lifecycle, Eval metadata, or Codex targets. Every active product must keep a
complete, machine-readable Eval discovery contract and exactly one desired
Codex environment mapping.
- Run `automation/scripts/codex-task-lint.sh TASK.md` before dispatching a
Slack task to Codex. It rejects ambiguous, secret-bearing, unapproved, or
unverified targets and requires the source thread, acceptance evidence,
recovery path, and human boundary.
- Run `automation/scripts/codex-review-health.sh` for a read-only JSON Lines
inventory of Codex environment readiness, applicable `AGENTS.md` files, and
repository-owned `## Code Review Rules` coverage.
- Run `automation/scripts/eval-health.sh` before the weekly Eval review. It
proves the declared suite and workflow exist on the default branch, finds a
successful run where the exact Eval job and step actually executed, verifies
Expand Down
57 changes: 57 additions & 0 deletions automation/codex-targets.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
version: 1

slack:
channel: build
task_thread_required: true
completion_posting: link_only_preferred_if_workspace_supports_it

policy:
repository_scope: selected_public_repositories
environment_scope: one_environment_with_one_repository
ambiguous_target: reject
unverified_target: reject
review_rollout: manual_before_automatic
approval_boundary: explicit_human_message

targets:
talent-signal:
repository: getyak/talent-signal
desired_environment: getyak-talent-signal
repo_map: [getyak/talent-signal]
integration_state: pending_verification
review_mode: manual_p0_p1_after_setup
daypage:
repository: getyak/daypage
desired_environment: getyak-daypage
repo_map: [getyak/daypage]
integration_state: pending_verification
review_mode: manual_p0_p1_after_setup
FitCoach:
repository: getyak/FitCoach
desired_environment: getyak-fitcoach
repo_map: [getyak/FitCoach]
integration_state: pending_verification
review_mode: manual_p0_p1_after_setup
apply-agent:
repository: getyak/apply-agent
desired_environment: getyak-apply-agent
repo_map: [getyak/apply-agent]
integration_state: pending_verification
review_mode: manual_p0_p1_after_setup
telepace-next:
repository: getyak/telepace-next
desired_environment: getyak-telepace-next
repo_map: [getyak/telepace-next]
integration_state: pending_verification
review_mode: manual_p0_p1_after_setup
solo-compass:
repository: getyak/solo-compass
desired_environment: getyak-solo-compass
repo_map: [getyak/solo-compass]
integration_state: pending_verification
review_mode: manual_p0_p1_after_setup

excluded:
- private signing repositories
- credential and secret repositories
- repositories outside getyak
60 changes: 60 additions & 0 deletions automation/codex-task-contract.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Slack to Codex Task Contract

This contract makes a Slack request safe to hand to Codex. It is intentionally
fail closed: a task is not ready merely because it is understandable to a
person.

Codex in Slack can use the most recently used environment when a request is
ambiguous, and a chat runs against the default branch of the first repository
in an environment's repository map. Therefore every Getyak Codex environment
has one repository and every request names both values explicitly. The desired
mapping and its verified state live in
[`codex-targets.yml`](codex-targets.yml). This defensive rule follows the
[official Codex in Slack behavior](https://learn.chatgpt.com/docs/third-party/slack).

## Required envelope

Start from [`templates/codex-task.md`](templates/codex-task.md). A task must
contain:

- exactly one approved public Getyak repository;
- that repository's exact desired Codex environment;
- the originating `#build` Slack thread permalink;
- one stable fingerprint for retries and follow-up;
- one observable outcome and one durable GitHub evidence URL;
- explicit constraints, acceptance checks, and rollback or recovery steps;
- the human boundary for deploys, secrets, permissions, OAuth, external
communication, and other consequential actions.

Run the local validator before mentioning `@Codex`:

```bash
automation/scripts/codex-task-lint.sh path/to/task.md
```

The validator rejects an environment whose `integration_state` is not `ready`.
`CODEX_TASK_ALLOW_PENDING_TARGET=true` exists only for testing the contract
before an integration is configured; it must not be used to dispatch a real
Slack task.

The resulting pull request is the durable artifact. Diagnosis, approval, and
completion remain replies in the originating Slack thread. A completion link
does not prove acceptance: verify the named checks and evidence before closing
the task.

## Review rollout

Begin with an explicit `@codex review` on selected pull requests. Add two or
three concise, outcome-focused rules under `## Code Review Rules` in the
repository's applicable `AGENTS.md`. Mechanical formatting and test checks stay
in CI. Enable automatic review only after the manual sample demonstrates useful
signal and an acceptable false-positive rate.

The rollout follows the
[official Codex code-review guidance](https://learn.chatgpt.com/docs/third-party/github):
keep repository-specific review instructions concise, and use CI for
mechanical checks and enforcement.

Use `automation/scripts/codex-review-health.sh` for a read-only inventory of
environment readiness and review-rule coverage. A pending integration or
missing review rule is planned setup debt, not an hourly incident.
27 changes: 17 additions & 10 deletions automation/prompts/daily-ops.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,11 @@ Run once on weekdays in `Asia/Shanghai`. This is a read-mostly control loop.
`automation/slack-routing.md` from `getyak/.github`.
2. Run `automation/scripts/portfolio-health.sh` or perform equivalent read-only
checks with GitHub CLI for every public, non-archived repository.
3. Review `configuration_drift`. Secret scanning, push protection, Dependabot
3. Run `automation/scripts/codex-review-health.sh`. Treat an environment that
changed from `ready` to another state or a newly missing review rule as
configuration drift. Known `pending_verification` setup is planned debt and
must not create repeated hourly or daily incidents.
4. Review `configuration_drift`. Secret scanning, push protection, Dependabot
security updates, read-only default workflow permissions, and immutable
Action SHA enforcement are the expected public-repository baseline. Each
repository must allow only GitHub-owned Actions plus the exact external
Expand All @@ -18,26 +22,29 @@ Run once on weekdays in `Asia/Shanghai`. This is a read-mostly control loop.
`workflow_supply_chain` scan, an unpinned external Action, or any remote
script piped directly into a shell as configuration drift requiring
inspection.
4. Inspect only newly failing or still-unresolved workflow runs, deployment
5. Inspect only newly failing or still-unresolved workflow runs, deployment
failures, high/critical Dependabot alerts, code-scanning alerts, and missing
security coverage. Treat `recent_failed_runs` as default-branch candidates,
not proof of an incident, and inspect failure steps before classifying them.
5. Check the automation host for disk pressure, memory pressure, unhealthy
6. Check the automation host for disk pressure, memory pressure, unhealthy
containers, stopped required services, and expiring certificates. Read the
cached macOS update inventory and reject it as stale when its successful scan
is older than 48 hours. A restart update is maintenance debt, not an incident;
include it only when its version differs from the last version recorded after
a verified digest post. Never include secret values or private data in output.
6. Compare each finding with the latest `#ops` threads. Reuse an unresolved
7. Compare each finding with the latest `#ops` threads. Reuse an unresolved
thread with the same fingerprint; do not create a duplicate root.
7. Post immediately only for new SEV-1/2 findings. Otherwise create at most one
8. Post immediately only for new SEV-1/2 findings. Otherwise create at most one
concise daily digest in `#ops`. If there is no actionable change, post
nothing.
8. Put scoped engineering work in `#build`, preferably as a reply linking the
original `#ops` thread. Do not deploy, rotate secrets, change permissions,
grant OAuth access, install system updates, restart the host, or send external
communications without explicit human approval naming the action and target.
9. When evidence confirms recovery, reply once to the original incident and
9. Put scoped engineering work in `#build`, preferably as a reply linking the
original `#ops` thread. Build the request from
`automation/templates/codex-task.md` and require
`automation/scripts/codex-task-lint.sh` to pass before mentioning `@Codex`.
Do not deploy, rotate secrets, change permissions, grant OAuth access,
install system updates, restart the host, or send external communications
without explicit human approval naming the action and target.
10. When evidence confirms recovery, reply once to the original incident and
mark the state resolved. Never infer recovery from elapsed time.

The final task result must state what changed since the previous run, what was
Expand Down
4 changes: 3 additions & 1 deletion automation/prompts/weekly-eval.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,9 @@ Run once each Friday in `Asia/Shanghai`.
a coverage state of `complete`, `partial`, or `invalid`.
6. Route real regressions or meaningful improvements to one weekly `#signals`
digest. Put implementation work in `#build` only when it can be expressed as
a bounded task with acceptance evidence.
a bounded task with acceptance evidence. Build it from
`automation/templates/codex-task.md` and do not mention `@Codex` unless
`automation/scripts/codex-task-lint.sh` passes against a `ready` target.
7. Escalate to `#hq` only when a result changes a product decision, release
decision, safety boundary, or portfolio priority. Do not copy the full
`#signals` digest.
Expand Down
121 changes: 121 additions & 0 deletions automation/scripts/codex-review-health.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
#!/usr/bin/env bash
set -euo pipefail

for required_command in gh jq ruby base64 grep; do
command -v "$required_command" >/dev/null || {
echo "$required_command is required" >&2
exit 2
}
done

repository_root="$(cd "$(dirname "$BASH_SOURCE")/../.." && pwd)"
targets_path="${CODEX_TARGETS_PATH:-$repository_root/automation/codex-targets.yml}"
retry_delay="${CODEX_REVIEW_API_RETRY_DELAY_SECONDS:-1}"

api_json() {
local endpoint="$1"
local attempt output
for attempt in 1 2 3; do
if output="$(gh api "$endpoint" 2>/dev/null)" && jq -e . >/dev/null 2>&1 <<<"$output"; then
printf '%s\n' "$output"
return 0
fi
if ((attempt < 3)); then
sleep "$retry_delay"
fi
done
return 1
}

target_records="$({
ruby - "$targets_path" <<'RUBY'
require "base64"
require "json"
require "yaml"

YAML.load_file(ARGV.fetch(0)).fetch("targets").each do |name, target|
puts Base64.strict_encode64(JSON.generate(
name: name,
repository: target.fetch("repository"),
desired_environment: target.fetch("desired_environment"),
integration_state: target.fetch("integration_state"),
review_mode: target.fetch("review_mode")
))
end
RUBY
} )"

while IFS= read -r encoded_record; do
[[ -n "$encoded_record" ]] || continue
record="$(printf '%s' "$encoded_record" | base64 --decode)"
repository="$(jq -r '.repository' <<<"$record")"
desired_environment="$(jq -r '.desired_environment' <<<"$record")"
integration_state="$(jq -r '.integration_state' <<<"$record")"
review_mode="$(jq -r '.review_mode' <<<"$record")"
default_branch=""
repository_state="unavailable"
agents_files='[]'
review_rules_files='[]'
review_debt='[]'

if repository_json="$(api_json "repos/$repository")"; then
default_branch="$(jq -r '.default_branch // empty' <<<"$repository_json")"
repository_state="available"
else
review_debt='["repository_unavailable"]'
fi

if [[ -n "$default_branch" ]]; then
if tree_json="$(api_json "repos/$repository/git/trees/$default_branch?recursive=1")" &&
jq -e '.truncated != true and (.tree | type == "array")' >/dev/null 2>&1 <<<"$tree_json"; then
agents_files="$(jq -c '[.tree[]? | select(.type == "blob" and (.path | test("(^|/)AGENTS\\.md$"))) | .path] | sort' <<<"$tree_json")"

while IFS=$'\t' read -r agents_path blob_sha; do
[[ -n "$agents_path" && -n "$blob_sha" ]] || continue
if blob_json="$(api_json "repos/$repository/git/blobs/$blob_sha")"; then
content="$(jq -r '.content // empty' <<<"$blob_json" | tr -d '\n' | base64 --decode 2>/dev/null || true)"
if grep -Eq '^##[[:space:]]+Code Review Rules[[:space:]]*$' <<<"$content"; then
review_rules_files="$(jq -c --arg path "$agents_path" '. + [$path]' <<<"$review_rules_files")"
fi
else
review_debt="$(jq -c '. + ["agents_file_unavailable"] | unique' <<<"$review_debt")"
fi
done < <(jq -r '.tree[]? | select(.type == "blob" and (.path | test("(^|/)AGENTS\\.md$"))) | [.path, .sha] | @tsv' <<<"$tree_json")
else
review_debt="$(jq -c '. + ["default_branch_tree_unavailable"] | unique' <<<"$review_debt")"
fi
fi

if [[ "$integration_state" != "ready" ]]; then
review_debt="$(jq -c '. + ["codex_slack_environment_not_ready"] | unique' <<<"$review_debt")"
fi
if [[ "$(jq 'length' <<<"$agents_files")" == "0" ]]; then
review_debt="$(jq -c '. + ["agents_instructions_missing"] | unique' <<<"$review_debt")"
elif [[ "$(jq 'length' <<<"$review_rules_files")" == "0" ]]; then
review_debt="$(jq -c '. + ["code_review_rules_missing"] | unique' <<<"$review_debt")"
fi

jq -cn \
--arg observed_at "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
--arg repository "$repository" \
--arg repository_state "$repository_state" \
--arg default_branch "$default_branch" \
--arg desired_environment "$desired_environment" \
--arg integration_state "$integration_state" \
--arg review_mode "$review_mode" \
--argjson agents_files "$agents_files" \
--argjson review_rules_files "$review_rules_files" \
--argjson review_debt "$review_debt" \
'{
observed_at: $observed_at,
repository: $repository,
repository_state: $repository_state,
default_branch: ($default_branch | if length > 0 then . else null end),
desired_environment: $desired_environment,
integration_state: $integration_state,
review_mode: $review_mode,
agents_files: $agents_files,
code_review_rules_files: $review_rules_files,
review_debt: $review_debt
}'
done <<<"$target_records"
Loading