Skip to content

fix: scope stale-service reconciliation to the services that failed to resolve - #238

Merged
vigneshrajsb merged 2 commits into
mainfrom
fix/scope-stale-service-reconciliation-veto
Aug 25, 2026
Merged

fix: scope stale-service reconciliation to the services that failed to resolve#238
vigneshrajsb merged 2 commits into
mainfrom
fix/scope-stale-service-reconciliation-veto

Conversation

@vigneshrajsb

Copy link
Copy Markdown
Contributor

Problem

Stale-service reconciliation silently stopped running for any environment whose lifecycle.yaml names a service that cannot be fully resolved — most commonly a dependency repository: that is no longer live in Lifecycle (archived, or removed from the GitHub App installation), or a legacy serviceId: entry.

The failure is invisible: the environment keeps building and deploying services that were renamed or deleted from lifecycle.yaml, and every one of those deploys fails. Renaming a service is the usual way to hit it — the old deployable row is never reaped, so it redeploys forever alongside the new one.

Origin

Reconciliation landed in #198 with a single environment-wide veto: if a referenced YAML config could not be fetched, skip cleanup entirely.

#229 refactored the dependency fetch from a repository name to a repository row:

- fetchLifecycleConfig(yamlEnvService.repository, branchName)
+ fetchLifecycleConfigByRepository(repository, dependencyConfigRef)

That needs a non-null row, so a !repository || repository.deletedAt != null guard was added — and it reached for the same environment-wide veto. The liveness test thereby moved from "can this YAML be fetched?" to "does Lifecycle hold a live repository row?", which a de-onboarded repository now fails permanently even though its config is still readable.

The veto is also checked before the reconcileDeletedServices feature flag, so an operator who enables the flag sees no effect and no log line explaining why.

Fix

Make the veto per-service rather than environment-wide.

upsertDeployables now returns unresolvedServiceNames and unresolvedRepositoryIds. Reaping excludes those names, plus any deployable owned by a repository whose config could not be read — requires: children of such a repository are never enumerated, so they are protected by repository id rather than by name.

canReconcile is kept for the root-level failures where genuinely nothing is known:

  • the source repository or ref is missing
  • the targeted repository has no live row
  • no service could be attributed to the targeted scope

The safety property is unchanged — never reap a service whose config we could not read — it is just enforced at the granularity of the service that actually failed. A service that is absent from a config we read successfully is still reaped, which is what makes renames and deletions work.

The surviving canReconcile early return now logs instead of returning silently.

Risks

  • A genuinely deleted service inside an unresolvable repository is never reaped. It stays protected until that repository resolves again. Conservative, and strictly better than the current behaviour, where nothing in the environment is reaped at all.
  • Fail-closed behaviour for API-created environments is unchanged. The guard those depend on is the targeted-repository check, which this PR leaves in place; its test (deployableSourceSeam.test.ts, "fails closed before YAML import when the targeted repository has no live row") still passes unmodified.
  • Wider reaping is now possible in environments where it was previously vetoed. That is the intent, but it means environments that have accumulated stale deployables will clean them up on their next reconciliation pass. Cleanup is still gated behind the reconcileDeletedServices feature flag.

Tests

  • deployableSourceSeam.test.ts — an unresolvable dependency repository, and a legacy serviceId entry, are recorded as unresolved and no longer flip canReconcile.
  • build.test.ts — unresolved names and unresolved repository ids are excluded from reaping; an unresolved dependency no longer blocks reaping the rest of the environment.

Full suite passes (4198 tests). Lint and Prettier clean.

…o resolve

A service that could not be resolved from YAML raised an environment-wide
veto (`canReconcile = false`), which disabled stale-service cleanup for
every other service in the environment. One unresolvable reference — an
archived dependency repository, or a legacy `serviceId` entry — was enough
to switch reconciliation off permanently for that environment.

Record the unresolved service names, and the repositories whose config
could not be read, and exclude only those from reaping. `canReconcile`
now covers just the root-level failures where nothing is known: the
source repository or ref is missing, the targeted repository is not live,
or no service could be attributed to the targeted scope.

Also log the remaining `canReconcile` early return, which was silent and
indistinguishable from a clean reconciliation pass.
@vigneshrajsb
vigneshrajsb requested a review from a team as a code owner August 24, 2026 22:15
…ution

The `requires:` recursion only runs for a service that resolved, so when
exact-name resolution fails the service's inner dependencies are never
enumerated and fall out of the expected set. Record the repository, not
just the service name, so those children are protected from reaping.
@vigneshrajsb
vigneshrajsb merged commit aee3c46 into main Aug 25, 2026
5 checks passed
@vigneshrajsb
vigneshrajsb deleted the fix/scope-stale-service-reconciliation-veto branch August 25, 2026 15:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants