Skip to content

bug: the remote control plane's wake is not idempotent against a concurrent start #224

Description

@outofcoffee

remote/lambda/start/index.ts's wake() de-duplicates a concurrent start purely by looking up the environment's instance by tag (findManagedInstance, an eventually-consistent DescribeInstances call). Two wake() invocations for the same environment close enough together can each miss the other's not-yet-visible instance and each reach launchAcrossAzs, launching two EC2 instances for one environment — a real double-billed launch, not a cosmetic race.

#222 closes this gap for calls originating from one gateway process (a Go-side singleflight.Group coalesces concurrent wakes before they ever reach the control plane), but the control plane itself is still unprotected against:

  • two separate gateway processes pointed at the same environment
  • a spinloop remote start CLI call racing a gateway wake
  • any other direct caller

Fix

An idempotency lock in wake(), acquired before findManagedInstance and released (try/finally) across every one of its early-return paths (unconfigured, undeployed, the seeding gate, no-capacity, boot-failed, deadline-timeout, ready).

Cheapest option: an SSM parameter (e.g. /cloud-vm-llm/<env>/wake-lock) written with Overwrite: false as a create-if-absent primitive — the existing SSM IAM policy on the start Lambda's role is already namespace-wide (arn:...:parameter/cloud-vm-llm/*), so no new IAM grant is needed. ParameterAlreadyExists means another wake is in flight (or crashed): read the existing lock's expiry and either return the existing 503-retry reply, or overwrite and proceed if it's stale. The TTL needs to exceed the Lambda's own 900s timeout, since a legitimate wake can run that long.

A DynamoDB-backed lock (conditional PutItem, TTL attribute) is the more conventional alternative, at the cost of a new CDK construct and IAM grants — no DynamoDB table exists in the stack today.

Ruled out: reserved concurrency = 1 on the Lambda serializes every environment's wakes through one slot, not just the racing one. An SQS FIFO queue keyed by environment would be correctly scoped, but means converting the Lambda from its current synchronous poll-until-ready HTTP contract to an async one — a bigger redesign than the lock itself.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions