From 719f498cca0574e5ac53bf96b6ecab6015330df9 Mon Sep 17 00:00:00 2001 From: Pete Cornish Date: Sat, 12 Sep 2026 17:07:34 +0100 Subject: [PATCH 1/3] docs(openspec): propose per-environment remote instance type --- .../remote-instance-type/.openspec.yaml | 2 + .../changes/remote-instance-type/design.md | 125 ++++++++++++++ .../changes/remote-instance-type/proposal.md | 71 ++++++++ .../specs/endpoint-lifecycle/spec.md | 161 ++++++++++++++++++ .../specs/environment-deployment/spec.md | 62 +++++++ .../specs/fleet-client/spec.md | 82 +++++++++ .../specs/remote-node/spec.md | 67 ++++++++ .../changes/remote-instance-type/tasks.md | 90 ++++++++++ 8 files changed, 660 insertions(+) create mode 100644 openspec/changes/remote-instance-type/.openspec.yaml create mode 100644 openspec/changes/remote-instance-type/design.md create mode 100644 openspec/changes/remote-instance-type/proposal.md create mode 100644 openspec/changes/remote-instance-type/specs/endpoint-lifecycle/spec.md create mode 100644 openspec/changes/remote-instance-type/specs/environment-deployment/spec.md create mode 100644 openspec/changes/remote-instance-type/specs/fleet-client/spec.md create mode 100644 openspec/changes/remote-instance-type/specs/remote-node/spec.md create mode 100644 openspec/changes/remote-instance-type/tasks.md diff --git a/openspec/changes/remote-instance-type/.openspec.yaml b/openspec/changes/remote-instance-type/.openspec.yaml new file mode 100644 index 00000000..2b596d13 --- /dev/null +++ b/openspec/changes/remote-instance-type/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-09-12 diff --git a/openspec/changes/remote-instance-type/design.md b/openspec/changes/remote-instance-type/design.md new file mode 100644 index 00000000..6c5f5999 --- /dev/null +++ b/openspec/changes/remote-instance-type/design.md @@ -0,0 +1,125 @@ +## Context + +Today the instance type a remote environment launches as is a single value for +the whole control plane: a CDK-stack setting (`remote/lib/config.ts`, default +`g6e.xlarge`) handed to the start Lambda as the `INSTANCE_TYPE` env var, used +verbatim in its `runInstance` call. The only way to change it is to redeploy +the whole control plane, which changes every environment at once. + +The per-environment record that already varies between environments is the +`DeployConfig`: a JSON document stored in an SSM parameter, written by the +deploy Lambda and read by the start Lambda on every wake. It already carries +the model, runner, context, and the spinloop version pin. Adding the instance +type there reuses the existing vehicle rather than inventing a new one. + +See proposal.md for motivation and the specs for the requirements this +implements. + +## Goals / Non-Goals + +**Goals:** +- Let one environment launch as a different instance type than the rest of the + account, set at deploy time and read back on every fresh launch. +- Keep a fleet file and a standalone `remote deploy` of the same source in + agreement about what a node's environment launches as. +- Stay backward compatible: an environment (or a control plane) that carries no + type behaves exactly as it does now. + +**Non-Goals:** +- No Spinloop-file keyword for the type — it stays a machine/environment-local + setting, like the harness and an alias. +- No `remote start` flag. The type is a property of the deployment, not of a + launch. +- No way to resize a running or stopped instance (EC2 does not allow it); a + changed type applies on the next fresh launch. +- No per-AZ or per-request type choice. + +## Decisions + +### The type lives in the per-environment `DeployConfig`, not on the start request + +`DeployConfig` (Go `internal/remote/remote.go`, TS +`remote/lambda/shared/deploy-config.ts`) gains an optional `instanceType`. +The deploy path records it; the start Lambda reads it on every wake and uses it +in the fresh-launch `runInstance` call, falling back to the `INSTANCE_TYPE` +env var when it is absent. + +This matches how the spinloop version pin already works (a deploy-time value, +stored in the same config, applied on the next fresh boot) and keeps the start +request unchanged. A re-wake of a stopped instance never relaunches, so it +naturally keeps the type it was launched with — no extra code is needed for the +fresh-launch-only semantics; the spec states it. + +*Alternative considered:* pass the type on each start request. Rejected — the +type is a property of the environment (the operator sets it once and expects +every launch to honour it), a start is not a natural place for it, and a re-wake +could not apply it anyway. + +### Fallback to the stack default keeps everything backward compatible + +Absent `instanceType` → the start Lambda uses the `INSTANCE_TYPE` env var +exactly as today. A control plane whose `parseDeployConfig` predates the field +ignores an unknown key in the stored config, so it keeps launching on its stack +default. Per-environment types therefore require a control plane deployed with +the updated `remote/` sources; until then the feature degrades to the default +rather than failing. + +### Validation in two places, one pattern + +The value is interpolated into an EC2 `RunInstances` call, so a malformed value +is best named early. It is checked in the Go CLI (so a `--instance-type` typo or +a bad fleet-file value is reported before anything is sent) and in the TS +`parseDeployConfig` (the server-side authority). Both use the same shape: +`^[a-z0-9]+(?:-[a-z0-9]+)*\.[a-z0-9]+$` — a lowercase family (which may be +hyphenated, as in `u7i-6tb` and `mac2-m2`) and a size separated by a single dot. +Permissive enough for every current EC2 family, strict enough to reject obvious +junk. + +The Go pattern lives in `internal/remote` (an exported validator/pattern) so +both `cmd/spinloop` (the flag) and `internal/fleet` (the file field) reuse it — +`internal/fleet` already imports `internal/remote`. + +### The fleet file field is remote-only and parsed eagerly + +`NodeConfig` (internal/fleet) gains `InstanceType` (`yaml:"instance-type"`). +`validate` rejects it on a `kind: daemon` node (a daemon's hardware is not +something the fleet file provisions) and checks its shape on a `kind: remote` +node, so a typo fails at file load with the node named — the file's existing +discipline. `fleet deploy` threads the node's value into the `DeployConfig` it +derives (the same seam that already carries the spinloop version), with no +fleet-level flag: the type is per-node, and a `--all` run has no single value to +apply. + +## Risks / Trade-offs + +- [A changed type does not reach a live instance] → The instance must be + terminated (`remote stop` or the idle sweep) and relaunched for the new type + to apply; a re-wake or `remote restart` keeps the old type. Documented in the + spec and the docs; there is no EC2 API to resize, so this is inherent. +- [An old control plane silently ignores the type] → Deploys still succeed and + launch on the stack default. Mitigation: the docs state that per-environment + types need a re-bootstrapped control plane; the fallback means nothing breaks. +- [A valid-format type has no capacity or no quota in the region] → The + existing per-AZ fallback and quota handling cover it; the no-capacity answer + now names the type it was trying, so the operator sees which type was short. +- [The pattern is wrong (too strict rejects a real type, too loose admits + junk)] → Chosen to cover all current EC2 families including hyphenated ones; + the server-side check is the final guard even if the CLI pattern drifts. + +## Migration Plan + +- The Go and TypeScript changes ship in this repository. The Lambdas take effect + when the operator re-runs `spinloop remote bootstrap` (which drives the + updated `remote/` CDK sources); no SSM data migration is needed because the + deploy-config parameter simply gains an optional key. +- Until a control plane is re-bootstrapped it ignores the key and launches on + its stack default — existing fleets and environments are unaffected. +- Rollback: revert the source. A stored deploy-config carrying `instanceType` is + harmless to an older Lambda (unknown keys are ignored), so no cleanup is + required. + +## Open Questions + +None. (Surfacing the *configured* type alongside the *actual* EC2 type in +`remote status`/`metrics` is a possible later nicety; the actual type is already +reported from EC2 today, so it is out of scope here.) diff --git a/openspec/changes/remote-instance-type/proposal.md b/openspec/changes/remote-instance-type/proposal.md new file mode 100644 index 00000000..4b4955bf --- /dev/null +++ b/openspec/changes/remote-instance-type/proposal.md @@ -0,0 +1,71 @@ +## Why + +A remote environment's instance type is fixed for the whole control plane (one +CDK-stack value, `g6e.xlarge` by default), so every environment in an account +launches the same machine. There is no way to give one environment a bigger or +cheaper GPU — the only lever today is to redeploy the entire control plane with +a different stack value, which changes every environment at once. A fleet that +mixes a small model and a large one, or that wants to trial a cheaper type, +cannot express that. + +## What Changes + +- A `kind: remote` node in `fleet.yaml` MAY declare an optional `instance-type` + naming the EC2 instance type its environment launches as. It is remote-only: + a `kind: daemon` node naming one is a parse error, and the value is + format-checked when the file is read. +- `spinloop remote deploy` gains an optional `--instance-type` flag. When given, + the type is recorded in the environment's stored deploy config so its next + fresh launch uses it; when absent (or empty), the environment keeps the + control plane's default. +- The per-environment deploy config (the SSM state the start Lambda reads on + every wake) gains an optional `instanceType`. The control plane validates and + persists it, and the start Lambda launches with it when present, falling back + to the stack-level default when absent. +- `spinloop fleet deploy` derives each `kind: remote` node's deploy config + including that node's `instance-type`, so a fleet file and a standalone + `remote deploy` of the same source still agree about what a node deploys. +- The deploy plan (including `--dry-run`) states the instance type an + environment will launch as, the way it already states the spinloop version. +- A changed type takes effect on the next **fresh launch** only: EC2 cannot + resize a running or stopped instance, so a re-wake (and `remote restart`) + keeps the original type, and the type applies once the instance has been + terminated (by `remote stop` or the idle sweep) and relaunched. + +No Spinloop-file keyword is introduced, and `remote start` takes no +instance-type flag — the type is a property of the environment, set at deploy +time, not of a launch. + +## Capabilities + +### New Capabilities + + + +### Modified Capabilities + +- `environment-deployment`: the stored per-environment deploy config may carry + an optional instance type; `remote deploy --instance-type` records it; the + deploy plan states it. +- `remote-node`: the fleet file's `kind: remote` node entry may declare an + optional, remote-only, format-checked `instance-type`. +- `fleet-client`: `fleet deploy` includes each remote node's `instance-type` in + the deploy config it derives and applies. +- `endpoint-lifecycle`: the start launches with the environment's stored + instance type when present, else the stack default; the type applies on a + fresh launch, not a re-wake; the no-capacity answer names the actual type. + +## Impact + +- Go: `internal/remote` (`DeployConfig` gains `InstanceType`), + `internal/fleet` (`NodeConfig` gains `InstanceType` + validation), + `cmd/spinloop/remote.go` (flag, plan output, derivation), `cmd/spinloop/fleet.go` + (fleet deploy passes the node's value). +- Control plane (`remote/`, TypeScript): `lambda/shared/deploy-config.ts` + (parse + validate the optional field), `lambda/start/index.ts` (launch with + the stored type, fallback to the env-var default, no-capacity wording). +- Behavioural: existing environments and a control plane that predates the field + are unchanged (absent type falls back to the stack default). Per-environment + types require a control plane deployed with the updated `remote/` sources. +- Docs: the `fleet.yaml` reference, the `remote deploy` reference, and the + fleet/remote examples. diff --git a/openspec/changes/remote-instance-type/specs/endpoint-lifecycle/spec.md b/openspec/changes/remote-instance-type/specs/endpoint-lifecycle/spec.md new file mode 100644 index 00000000..4d35868f --- /dev/null +++ b/openspec/changes/remote-instance-type/specs/endpoint-lifecycle/spec.md @@ -0,0 +1,161 @@ +## MODIFIED Requirements + +### Requirement: Starting on demand + +Each environment SHALL hold no running instance when idle. A start request names +an environment and SHALL launch that environment's instance, trying each +configured availability zone in turn until one has capacity, since GPU capacity +is not guaranteed in any single zone. A launch SHALL provision the instance's +root volume: a gp3 volume of the AMI's own root size, with provisioned +throughput at the volume's ceiling — the size is read from the AMI's own root +mapping, because a launch's block device mapping replaces the AMI's rather +than extending it — and IOPS provisioned at four times that throughput, +which is the minimum EC2 allows for it (gp3 caps throughput at 0.25 MiB/s +per provisioned IOP). The +instance SHALL be given the environment's own stable address (its Elastic IP) +so the environment's URL does not change between launches, and the request +SHALL NOT report success until the model is answering — the caller receives +one "ready", never a URL that is not yet serving. When no capacity can be +found anywhere, the response SHALL say so, SHALL name the instance type it was +trying to launch, and SHALL be retryable rather than +fatal. One shared set of lifecycle Lambdas SHALL serve every environment in +the account, selecting the instance by the environment identifier. + +A launch SHALL use the instance type the environment's deploy config names, +when it names one, and the control plane's default type otherwise. The type is +a property of the environment's deployment, read from the same stored deploy +config the start already reads for what to serve, so changing it is a deploy, +not a start. Because EC2 cannot change the type of an existing instance, a +stored type takes effect on a fresh launch only: a re-wake of a stopped +instance SHALL keep the type it was originally launched with, and a changed +type applies once the instance has been terminated — by an explicit stop or +the idle sweep — and relaunched. + +Before launching or re-waking the instance, a start SHALL check that the +environment's weights are present in shared storage, judged by the same +completeness record the seeding writes rather than by the absence of an error. +While the weights are absent, a start SHALL NOT launch or re-wake the +instance, and SHALL report a retryable state that names the seed producing +the weights, so a caller can wait for the seed or follow it separately rather +than receiving an instance that boots against an incomplete prefix. When no +seed is running for those weights, the start SHALL start one, so that the +weights are produced rather than the start failing; a seed whose compute has +ceased — failed, stopped or reaped — does not count as running, and its +re-run follows the same identity and convergence rules as any other seed +request. A start that would exceed the cap on seeds in flight SHALL NOT start +another seed, and SHALL report the retryable state until a later start can. + +The control plane SHALL request the engine's start on every path — a fresh +launch and a re-wake alike — once the instance's daemon answers its control +API, which on a fresh boot is the signal that the boot has stored the deploy +config; the boot's own user data SHALL NOT start the engine. The start SHALL +carry the deploy config as its body, so it always names the exact config the +daemon runs. + +#### Scenario: A zone without capacity is not the end of it + +- **WHEN** the first availability zone cannot provide the instance type +- **THEN** the remaining zones are tried before reporting failure + +#### Scenario: Ready means serving + +- **WHEN** a start request returns success +- **THEN** the model is answering requests at the environment's reported address + +#### Scenario: No capacity anywhere + +- **WHEN** every configured zone is out of capacity +- **THEN** the response says so, names the instance type it was trying, and + indicates the caller may retry shortly + +#### Scenario: Starting the right environment + +- **WHEN** several environments are deployed and a start names one of them +- **THEN** only that environment's instance is launched, at its own Elastic IP + +#### Scenario: Nothing has been deployed + +- **WHEN** a start is requested for an environment before it has been deployed +- **THEN** it fails saying what to deploy, rather than launching an instance + with nothing to serve + +#### Scenario: A launch provisions the root volume + +- **WHEN** a start launches a fresh instance +- **THEN** its root volume is the AMI's gp3 root, at the AMI's own size, with + provisioned throughput at the volume's ceiling and provisioned IOPS at four + times that throughput + +#### Scenario: A launch uses the environment's stored instance type + +- **WHEN** an environment's deploy config names an instance type and a start + launches a fresh instance for it +- **THEN** the instance is launched as that type + +#### Scenario: A launch with no stored type uses the control plane default + +- **WHEN** an environment's deploy config names no instance type and a start + launches a fresh instance for it +- **THEN** the instance is launched as the control plane's default type + +#### Scenario: A re-wake keeps the instance's original type + +- **WHEN** an environment's instance is stopped, its deploy config's instance + type is changed, and a start re-wakes the stopped instance +- **THEN** the instance comes back as the type it was originally launched + with, because a stopped instance is not resized + +#### Scenario: A changed type applies after the instance is terminated + +- **WHEN** an environment's deploy config instance type is changed, its + instance is terminated, and a later start launches it +- **THEN** the fresh instance is launched as the new type + +#### Scenario: The control plane starts the engine on a fresh boot + +- **WHEN** a fresh instance's daemon first answers its control API +- **THEN** the start request itself issues the engine's start, with the + deploy config as its body, and reports ready only once the model answers — + the boot started no engine + +#### Scenario: A start while the weights are seeding launches nothing + +- **WHEN** a start is requested for an environment whose weights are still + being seeded +- **THEN** no instance of the environment is launched or re-woken, and the + response is retryable and names the running seed + +#### Scenario: A start with absent weights and no running seed starts one + +- **WHEN** a start is requested for an environment whose weights are absent + and no seed is running for them +- **THEN** a seed for those weights is started, no instance is launched, and + the response is the same retryable state naming the seed + +#### Scenario: Two starts for the same weights share one seed + +- **WHEN** starts for two environments naming the same weights arrive while no + seed is running +- **THEN** one seed is started, and both responses name it + +#### Scenario: A ceased seed does not block a start + +- **WHEN** the only seed for the weights has ceased — failed, stopped or + reaped — and a start is requested +- **THEN** the weights are treated as absent: a new seed is started and the + response is the retryable state, never a launch against the partial prefix + the failed seed left behind + +#### Scenario: A start that would exceed the seed cap waits + +- **WHEN** the cap on seeds in flight is reached and a start needs to start a + seed for absent weights +- **THEN** no further seed is started, and the response is retryable until a + later start can start the seed + +#### Scenario: A start proceeds once the weights are present + +- **WHEN** a start has been reported in the seeding state and the seed has + since finished, leaving the weights present +- **THEN** the next start launches the environment's instance and proceeds to + ready as usual diff --git a/openspec/changes/remote-instance-type/specs/environment-deployment/spec.md b/openspec/changes/remote-instance-type/specs/environment-deployment/spec.md new file mode 100644 index 00000000..6e8cbd1c --- /dev/null +++ b/openspec/changes/remote-instance-type/specs/environment-deployment/spec.md @@ -0,0 +1,62 @@ +## ADDED Requirements + +### Requirement: Deploy accepts an optional instance type + +`spinloop remote deploy` SHALL accept an optional `--instance-type` flag naming +the EC2 instance type the environment's instances launch as. When the flag is +given, deploy SHALL record that type in the environment's stored deploy config +so the environment's next fresh launch uses it; when it is absent, deploy SHALL +record no type and the environment launches as the control plane's default. An +empty or whitespace-only value SHALL be treated as if the flag were not given. +A value that is not shaped like an EC2 instance type SHALL be refused before +anything is sent, naming the value. + +The instance type is a property of the environment's deployment, recorded in +the stored deploy config the way the spinloop version pin is — not a property +of a single start, and never derived from the Spinloop. + +#### Scenario: A type is recorded in the deploy config + +- **WHEN** `spinloop remote deploy` runs with `--instance-type g6e.2xlarge` +- **THEN** the environment's stored deploy config carries that type, and the + environment's next fresh launch uses it + +#### Scenario: No type leaves the launch on its default + +- **WHEN** `spinloop remote deploy` runs without `--instance-type` +- **THEN** the stored deploy config carries no instance type, and the + environment's launches use the control plane's default type + +#### Scenario: An empty type value is ignored + +- **WHEN** `spinloop remote deploy` is given an `--instance-type` whose value + is empty or whitespace only +- **THEN** it is treated as if no type were given + +#### Scenario: A malformed type is refused before sending + +- **WHEN** `spinloop remote deploy` is given an `--instance-type` that is not + shaped like an EC2 instance type +- **THEN** the command fails, naming the value, and nothing is sent to the + control plane + +### Requirement: The deploy plan shows the resolved instance type + +The plan `spinloop remote deploy` prints — including under `--dry-run`, before +any AWS work or send — SHALL state the instance type the environment will +launch as: the type named by `--instance-type` when one is given, otherwise a +statement that the environment launches as the control plane's default. It +SHALL appear alongside the runner and model the plan already prints. + +#### Scenario: A typed deploy prints the type + +- **WHEN** `spinloop remote deploy --dry-run` runs with `--instance-type + g6e.2xlarge` +- **THEN** the printed plan names `g6e.2xlarge` as the instance type the + environment will launch as + +#### Scenario: An untyped deploy prints the default + +- **WHEN** `spinloop remote deploy --dry-run` runs without `--instance-type` +- **THEN** the printed plan says the environment launches as the control + plane's default instance type diff --git a/openspec/changes/remote-instance-type/specs/fleet-client/spec.md b/openspec/changes/remote-instance-type/specs/fleet-client/spec.md new file mode 100644 index 00000000..f59dafef --- /dev/null +++ b/openspec/changes/remote-instance-type/specs/fleet-client/spec.md @@ -0,0 +1,82 @@ +## MODIFIED Requirements + +### Requirement: Fleet deploy derives and applies each node's config + +Each targeted node SHALL be deployed from the Spinloop file its deploy +source resolves to (see fleet-config's "Node Spinloop source" and "...falls +back to name-based lookup" requirements: its `file` field, else an alias +registered under its name, else a `/` subdirectory beside the fleet +file), deriving the deploy config and registering the resulting environment +exactly as `spinloop remote deploy ` does for that same file — the two +SHALL NOT be able to disagree about what a given Spinloop file deploys. A +targeted node for which no source resolves SHALL fail for that node alone, +naming all three ways one could have been given, without touching the other +targeted nodes. The resolved source (the path used, or the alias name when +one was used) SHALL be reported alongside that node's plan, so which of the +three supplied it is never left to be inferred. + +Where a node declares an `instance-type` in the fleet file, the deploy config +derived for it SHALL carry that type, so the node's environment launches as +named — the same value a standalone `spinloop remote deploy --instance-type` +would record for the environment — and a node declaring none SHALL deploy an +environment on the control plane's default type. This keeps `fleet deploy` and +a matching standalone deploy in agreement about what a node's environment +launches as. + +Nodes SHALL be deployed independently: one node already registered or live +SHALL require `--overwrite` for that node exactly as a standalone `remote +deploy` does, and refusing it SHALL NOT stop the other targeted nodes from +deploying. A node whose deploy fails for any other reason SHALL likewise be +reported against that node without aborting the rest. The command SHALL exit +non-zero when any targeted node failed to deploy, having still attempted +every other targeted node. + +`--dry-run` SHALL print the plan for every targeted node without deploying +any of them, exactly as a standalone `remote deploy --dry-run` does for one. +`--overwrite` SHALL apply to every targeted node that needs it. + +#### Scenario: A node deploys from its own Spinloop file + +- **WHEN** `fleet deploy` targets a node declaring `file: + ./envs/gpu.Spinloop` +- **THEN** that node's environment is created and registered from that file, + the same as `spinloop remote deploy ./envs/gpu.Spinloop` would produce, and + the resolved path is reported against that node + +#### Scenario: A node's declared instance type is deployed + +- **WHEN** `fleet deploy` targets a `kind: remote` node declaring + `instance-type: g6e.2xlarge` +- **THEN** the environment it deploys launches as `g6e.2xlarge`, the same + value a standalone `spinloop remote deploy --instance-type g6e.2xlarge` of + the node's source would record + +#### Scenario: A node with no instance type deploys the default + +- **WHEN** `fleet deploy` targets a `kind: remote` node declaring no + `instance-type` +- **THEN** the environment it deploys launches as the control plane's default + instance type + +#### Scenario: A node with no resolvable source fails only that node + +- **WHEN** `fleet deploy` targets two remote nodes and one declares no `file` + field, has no alias registered under its name, and has no same-named + subdirectory beside the fleet file +- **THEN** the other node still deploys, and the command reports against the + unresolved node that none of the `file` field, a matching alias, or a + matching subdirectory was found + +#### Scenario: One node's guard does not block the others + +- **WHEN** `fleet deploy` targets two remote nodes and one is already + registered while the other is not, and `--overwrite` is not given +- **THEN** the unregistered node deploys, the registered node is refused with + the same message a standalone `remote deploy` gives, and the command exits + non-zero + +#### Scenario: Dry run previews every targeted node + +- **WHEN** `spinloop fleet deploy --dry-run --all` runs +- **THEN** the plan for every `kind: remote` node in the file is printed and + no environment is created or registered diff --git a/openspec/changes/remote-instance-type/specs/remote-node/spec.md b/openspec/changes/remote-instance-type/specs/remote-node/spec.md new file mode 100644 index 00000000..8c8c7d6b --- /dev/null +++ b/openspec/changes/remote-instance-type/specs/remote-node/spec.md @@ -0,0 +1,67 @@ +## MODIFIED Requirements + +### Requirement: The fleet file declares remote nodes + +The fleet file SHALL be able to list a remote environment as one of its nodes, alongside +daemon nodes. The node's kind SHALL default to daemon. A node of kind `remote` SHALL be +keyed by its name: the name IS the registered environment it drives (there is no separate +address field, because an environment is already user-named at deployment), and such a +node SHALL need no host, because the environment's control URLs come from that +environment's own config rather than the fleet file. Because the name doubles as the +environment key, it SHALL be constrained to an environment shape — no path separator, no +`.json` suffix — so a path-like name is rejected rather than read as a registry directory. +Building the live node for a `remote` entry SHALL load that environment's config keyed by +its name, and an environment that is not registered SHALL fail as a per-node +configuration error — naming the environment — rather than failing the command or blanking +the view. A fleet of remote environments, or of daemons and remote environments mixed, +SHALL be observable and drivable (status, metrics, start, stop) through the same fan-out +as a fleet of daemons alone. + +A node of kind `remote` MAY declare an optional `instance-type` naming the EC2 instance +type its environment launches as. It is a property of the remote environment only: a +`kind: daemon` node naming one SHALL be rejected, because a daemon's hardware is the +operator's to choose, not something the fleet file provisions. When present, the value +SHALL be checked for the shape of an EC2 instance type — a lowercase family and size +separated by a single dot, as in `g6e.xlarge` — when the file is read, so a typo is named +at parse rather than at launch. A `kind: remote` node naming no `instance-type` SHALL +deploy an environment that launches as the control plane's default, unchanged from before +the field existed. + +#### Scenario: A fleet file lists a remote environment as a node + +- **WHEN** a fleet file lists a node of kind `remote` whose name is a registered + environment +- **THEN** the fan-out builds it as a remote node and observes it as one row, alongside any + daemon nodes in the same file + +#### Scenario: A fleet file lists a remote without its environment + +- **WHEN** a fleet file lists a node of kind `remote` whose name is not a registered + environment +- **THEN** that node is reported with a configuration error naming the environment, and + the rest of the fleet is still observed + +#### Scenario: A remote node's name must be env-shaped + +- **WHEN** a fleet file lists a node of kind `remote` whose name contains a path separator + or a `.json` suffix +- **THEN** the fleet file is rejected, naming the node, because the name is the environment + key + +#### Scenario: A remote node names its instance type + +- **WHEN** a fleet file lists a `kind: remote` node declaring `instance-type: g6e.2xlarge` +- **THEN** the file parses, and that node's environment is deployed to launch as + `g6e.2xlarge` + +#### Scenario: A daemon node naming an instance type is rejected + +- **WHEN** a fleet file lists a `kind: daemon` node declaring an `instance-type` +- **THEN** the file is rejected, naming the node, because instance type is a property of + a remote environment only + +#### Scenario: A malformed instance type is named at parse + +- **WHEN** a fleet file lists a `kind: remote` node whose `instance-type` is not shaped + like an EC2 instance type +- **THEN** the file is rejected, naming the node and the value diff --git a/openspec/changes/remote-instance-type/tasks.md b/openspec/changes/remote-instance-type/tasks.md new file mode 100644 index 00000000..278e9422 --- /dev/null +++ b/openspec/changes/remote-instance-type/tasks.md @@ -0,0 +1,90 @@ +## 1. Go: instance type in the deploy config and a shared validator + +- [ ] 1.1 Add `InstanceType string` (JSON key `instanceType`, `omitempty`) to + `DeployConfig` in `internal/remote/remote.go`. Verify with a unit test that a + config carrying the type marshals an `instanceType` key and a config without + it omits the key (matching the `SpinloopVersion` pattern). +- [ ] 1.2 Add an exported EC2 instance-type pattern and validator in + `internal/remote` (shape `family.size`, lowercase, family may be hyphenated). + Verify with a unit test that it accepts `g6e.xlarge`, `u7i-6tb.112xlarge`, + `mac2-m2.2xlarge`, `trn1.2xlarge` and rejects empty, `g6exlarge`, + `G6E.xlarge`, and `g6e.xlarge.extra`. + +## 2. Go: the fleet file field + +- [ ] 2.1 Add `InstanceType string` (YAML key `instance-type`) to `NodeConfig` + in `internal/fleet/config.go`. Verify the field parses from a `fleet.yaml` + listing a `kind: remote` node with `instance-type: g6e.2xlarge`. +- [ ] 2.2 In `validate()`, reject `instance-type` on a `kind: daemon` node and + check its shape (via the `internal/remote` validator) on a `kind: remote` + node. Verify with tests: a daemon node naming one is rejected naming the + node; a remote node with a malformed value is rejected naming the node and + value; a remote node with no value or a valid value parses. + +## 3. Go: `remote deploy --instance-type` + +- [ ] 3.1 Add an `instanceType string` field to `deployOpts` and an + `--instance-type` flag to `remoteDeployCmd`, threading it through + `runRemoteDeploy` into the `deployOpts` passed to `runDeploy`. Verify + `spinloop remote deploy --help` lists the flag. +- [ ] 3.2 In `runDeploy`, when `opts.instanceType` is non-empty, validate it + (naming the value on failure, before any send) and set `dc.InstanceType`; + treat an empty/whitespace value as absent. Verify with a test that a + malformed value fails before the deploy seam is reached and a valid value is + set on the derived config. +- [ ] 3.3 Print the resolved instance type in the deploy plan (the type when + set, otherwise a statement that the environment launches as the control + plane's default), alongside the runner and model. Verify `remote deploy + --dry-run --instance-type g6e.2xlarge` prints `g6e.2xlarge` and a run without + the flag prints the default statement. + +## 4. Go: `fleet deploy` threads the node's value + +- [ ] 4.1 In `deployOneNode`, set the targeted node's `InstanceType` into the + `deployOpts` passed to `runDeploy` (per-node; a node naming none leaves it + empty). Verify with a test that `fleet deploy --dry-run` for a node declaring + `instance-type` shows that type in its plan and a node declaring none shows + the default. + +## 5. TypeScript: parse and validate the field + +- [ ] 5.1 Add `instanceType?: string` to the `DeployConfig` interface and parse + + validate it in `parseDeployConfig` in `remote/lambda/shared/deploy-config.ts` + (absent → `undefined`; present → non-empty string matching the same + `family.size` shape, else a thrown error naming the value). Verify by + extending `remote/test/deploy-config.test.ts`: a config with the type parses + it, one without leaves it `undefined`, and a malformed value throws. +- [ ] 5.2 Confirm `writeDeployConfig`/`readDeployConfig` round-trip the field + (they serialise the whole config). Verify with a test that a config written + with `instanceType` reads it back. + +## 6. TypeScript: the start launches with the stored type + +- [ ] 6.1 In `remote/lambda/start/index.ts`, launch with + `deployConfig.instanceType` when present, else the `INSTANCE_TYPE` env var, + in the fresh-launch path. Verify by extending `remote/test/start-launch.test.ts`: + a deploy config naming a type launches as it, and one naming none launches as + the env-var default. +- [ ] 6.2 Update the no-capacity reply to name the instance type it was trying + (rather than a hard-coded `g6e`). Verify the no-capacity test asserts the + type appears in the message. + +## 7. Whole-repo verification + +- [ ] 7.1 Run `gofmt -l` over the touched Go files and confirm no output; run + `go vet ./...` and confirm it is clean. +- [ ] 7.2 Run `go test ./... -cover` and confirm the suite passes with total + coverage still >= 80%. +- [ ] 7.3 From `remote/`, run `pnpm build` (tsc type-check) and `pnpm test` + (vitest) and confirm both pass. + +## 8. Docs and examples + +- [ ] 8.1 Document `instance-type` in the `fleet.yaml` reference (remote-only, + the shape it must take, that it is deployed by `fleet deploy`) and + `--instance-type` in the `remote deploy` reference (recorded per environment, + applied on the next fresh launch, default when omitted). Verify the docs + build/lint if the repo has a docs check, else re-read for accuracy. +- [ ] 8.2 Add an `instance-type` to a `kind: remote` node in the fleet/remote + example (`examples/fleet-remote/fleet.yaml`) with a short comment, so the + field is discoverable. Verify the example still matches the documented format. From a8aad0b0153b762e383d67296aa9c3709e1ce2b0 Mon Sep 17 00:00:00 2001 From: Pete Cornish Date: Sat, 12 Sep 2026 17:45:50 +0100 Subject: [PATCH 2/3] feat(remote): add per-environment instance type to deploy and fleet nodes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The type is stored on the environment's deploy config and read back on its next fresh launch — a re-wake of a stopped instance keeps the type it launched with. Fed by `remote deploy --instance-type` and a fleet node's `instance-type`; omitted means the control plane's default. --- cmd/spinloop/fleet.go | 8 +- cmd/spinloop/fleet_deploy_test.go | 51 +++++++++++ cmd/spinloop/remote.go | 36 +++++++- cmd/spinloop/remote_deploy_test.go | 85 +++++++++++++++++++ docs/commands/fleet.md | 21 +++++ docs/commands/remote.md | 9 ++ docs/openapi.yaml | 9 ++ examples/fleet-remote/fleet.yaml | 1 + internal/fleet/config.go | 17 ++++ internal/fleet/config_test.go | 65 ++++++++++++-- internal/remote/remote.go | 23 +++++ internal/remote/remote_test.go | 62 ++++++++++++++ .../changes/remote-instance-type/tasks.md | 34 ++++---- remote/lambda/shared/deploy-config.ts | 45 ++++++++++ remote/lambda/start/index.ts | 9 +- remote/test/deploy-config.test.ts | 43 ++++++++++ remote/test/start-launch.test.ts | 66 ++++++++++++++ 17 files changed, 556 insertions(+), 28 deletions(-) diff --git a/cmd/spinloop/fleet.go b/cmd/spinloop/fleet.go index b168b2ab..67d9a865 100644 --- a/cmd/spinloop/fleet.go +++ b/cmd/spinloop/fleet.go @@ -674,7 +674,13 @@ func deployOneNode(cfg *fleet.Config, name string, opts deployOpts) fleetDeployR if err != nil { return fleetDeployResult{node: name, outcome: deployRowFailed, detail: err.Error()} } - outcome, err := runDeploy(spinloopPath, env, dc, opts) + // The node's own instance type is per-node. opts is shared across the + // concurrent deploy, so one node's machine must not leak to another — + // copy it for this call and set the node's type (a node naming none + // leaves it empty, i.e. the control plane's default). + nodeOpts := opts + nodeOpts.instanceType = entry.InstanceType + outcome, err := runDeploy(spinloopPath, env, dc, nodeOpts) if err != nil { var guarded *errDeployGuarded if errors.As(err, &guarded) { diff --git a/cmd/spinloop/fleet_deploy_test.go b/cmd/spinloop/fleet_deploy_test.go index 84327252..84ae3f33 100644 --- a/cmd/spinloop/fleet_deploy_test.go +++ b/cmd/spinloop/fleet_deploy_test.go @@ -328,6 +328,57 @@ func TestCmdFleetDeployDryRunTouchesNothing(t *testing.T) { } } +// A node's instance-type is per-node: `fleet deploy` threads each targeted +// node's own value into its deploy, so one node's machine never leaks to a +// sibling's plan — a node naming a type shows it, one naming none shows the +// control plane's default. +func TestCmdFleetDeployInstanceTypePerNode(t *testing.T) { + isolateConfig(t) + dir := writeFleetFile(t, ` +nodes: + - name: typed + kind: remote + file: ./typed.Spinloop + instance-type: g6e.2xlarge + - name: untyped + kind: remote + file: ./untyped.Spinloop +`) + writeSpinloop := func(name, env string) { + t.Helper() + body := fmt.Sprintf("PROVIDER llamacpp\nMODEL org/m:Q4\nCONTEXT 8192\nREMOTE %s\n", env) + if err := os.WriteFile(filepath.Join(dir, name), []byte(body), 0o600); err != nil { + t.Fatal(err) + } + } + writeSpinloop("typed.Spinloop", "typed") + writeSpinloop("untyped.Spinloop", "untyped") + + deployDiscoverFn = func(context.Context, aws.Config, string) (remote.ControlPlane, error) { + return remote.ControlPlane{}, fmt.Errorf("must not be called") + } + t.Cleanup(func() { deployDiscoverFn = remote.DiscoverControlPlane }) + + // --dry-run touches nothing, so no deploy seams need stubbing. + outTyped := captureStdout(t, func() { + if err := cmdFleet([]string{"deploy", "typed", "--dry-run"}); err != nil { + t.Errorf("deploy typed --dry-run: %v", err) + } + }) + if !strings.Contains(outTyped, "instance: g6e.2xlarge") { + t.Errorf("typed node's plan should name its instance type, got:\n%s", outTyped) + } + + outUntyped := captureStdout(t, func() { + if err := cmdFleet([]string{"deploy", "untyped", "--dry-run"}); err != nil { + t.Errorf("deploy untyped --dry-run: %v", err) + } + }) + if !strings.Contains(outUntyped, "instance: the control plane's default") { + t.Errorf("untyped node's plan should state the default machine, got:\n%s", outUntyped) + } +} + // mustEnvConfigPath resolves where a deployed environment would be // registered, under the isolated config dir this test's HOME points at. func mustEnvConfigPath(t *testing.T, env string) string { diff --git a/cmd/spinloop/remote.go b/cmd/spinloop/remote.go index c8449dea..9ecfbb17 100644 --- a/cmd/spinloop/remote.go +++ b/cmd/spinloop/remote.go @@ -1434,6 +1434,7 @@ func remoteDeployCmd() *cobra.Command { allowedCidr string region string spinloopVersion string + instanceType string apiKeyEnv string ) c := &cobra.Command{ @@ -1443,14 +1444,17 @@ func remoteDeployCmd() *cobra.Command { address, API key and allowed CIDR — and says what it serves (PROVIDER picks the engine, just as it does for serve). --spinloop-version pins the spinloop release a fresh boot of the environment installs; without it, a boot -installs the latest published release.`, +installs the latest published release. --instance-type names the EC2 instance +type the environment's instances launch as; without it, they launch as the +control plane's default. The type is recorded on the environment and applies +from its next fresh launch.`, Args: cobra.ArbitraryArgs, SilenceErrors: true, SilenceUsage: true, ValidArgsFunction: aliasSlot, RunE: func(c *cobra.Command, args []string) error { resolve(c) - return runRemoteDeploy(args, dryRun, overwrite, reseed, allowedCidr, region, spinloopVersion, apiKeyEnv) + return runRemoteDeploy(args, dryRun, overwrite, reseed, allowedCidr, region, spinloopVersion, instanceType, apiKeyEnv) }, } fs := c.Flags() @@ -1460,12 +1464,13 @@ installs the latest published release.`, fs.StringVar(&allowedCidr, "allowed-cidr", "", "who may reach this environment's instance (default: your public IP as a /32, on first deploy)") fs.StringVar(®ion, "region", "", "AWS region of the control plane (default: AWS_REGION or us-east-1)") fs.StringVar(&spinloopVersion, "spinloop-version", "", "spinloop release the environment's instances install at boot (default: latest)") + fs.StringVar(&instanceType, "instance-type", "", "EC2 instance type the environment's instances launch as (e.g. g6e.xlarge; default: the control plane's default type)") fs.StringVar(&apiKeyEnv, "api-key-env", "", "the environment variable holding the API key to store for this environment (a variable name, never the key itself)") return c } // runRemoteDeploy is the body of `spinloop remote deploy`. -func runRemoteDeploy(args []string, dryRun, overwrite, reseed bool, allowedCidr, region, spinloopVersion, apiKeyEnv string) error { +func runRemoteDeploy(args []string, dryRun, overwrite, reseed bool, allowedCidr, region, spinloopVersion, instanceType, apiKeyEnv string) error { _, spinloopPath, dc, env, err := deriveDeployTarget("spinloop remote deploy ", spinloopArg(args)) if err != nil { return err @@ -1477,6 +1482,7 @@ func runRemoteDeploy(args []string, dryRun, overwrite, reseed bool, allowedCidr, allowedCidr: allowedCidr, region: region, spinloopVersion: spinloopVersion, + instanceType: instanceType, apiKeyEnv: apiKeyEnv, }) if err != nil { @@ -1537,6 +1543,11 @@ type deployOpts struct { allowedCidr string region string spinloopVersion string + // instanceType names the EC2 instance type the environment's instances + // launch as. Empty means launch as the control plane's default. It is + // recorded on the environment at deploy time and read back on the next + // fresh launch. + instanceType string // apiKeyEnv names the environment variable holding an externally // supplied key to store as the environment's engine key, never a // literal on the command line. Empty means the control plane manages @@ -1592,6 +1603,18 @@ func runDeploy(spinloopPath, env string, dc remote.DeployConfig, opts deployOpts } dc.SpinloopVersion = pin } + // The EC2 instance type the environment's instances launch as: empty (or + // whitespace) means the control plane's own default, a name means exactly + // that machine. Checked here, so a typo is named now rather than as a + // RunInstances failure inside a deploy nobody is watching. + if name := strings.TrimSpace(opts.instanceType); name != "" { + if !remote.IsInstanceType(name) { + return deployOutcome{}, fmt.Errorf( + "--instance-type must be an EC2 instance type (a family and size separated by a dot, e.g. g6e.xlarge), got %q", + opts.instanceType) + } + dc.InstanceType = name + } // A supplied key arrives the way every other secret does: as a reference // to an environment variable, never a literal on the command line. By // this point deriveDeployTarget has already applied the Spinloop's local @@ -1642,6 +1665,13 @@ func runDeploy(spinloopPath, env string, dc remote.DeployConfig, opts deployOpts spinloopVer = "latest" } fmt.Fprintf(&buf, " spinloop: %s\n", spinloopVer) + // A machine is worth stating too: a name is a promise, the default a + // statement — so the plan always says what the environment launches as. + instanceType := dc.InstanceType + if instanceType == "" { + instanceType = "the control plane's default" + } + fmt.Fprintf(&buf, " instance: %s\n", instanceType) // A key is worth stating too — it rotates — but the value is never // printed, in the dry run or the report. if opts.apiKeyEnv != "" { diff --git a/cmd/spinloop/remote_deploy_test.go b/cmd/spinloop/remote_deploy_test.go index 8d4cb87a..fda450f6 100644 --- a/cmd/spinloop/remote_deploy_test.go +++ b/cmd/spinloop/remote_deploy_test.go @@ -617,6 +617,91 @@ func TestRemoteDeploy_SpinloopVersion(t *testing.T) { }) } +// An instance type is recorded on the environment: it reaches the signed body +// so the deploy Lambda persists it, and the plan names the machine the +// environment will launch as — a promise when named, the default otherwise. +func TestRemoteDeploy_InstanceType(t *testing.T) { + t.Run("the type reaches the deploy body and the plan", func(t *testing.T) { + isolateConfig(t) + stubAWSEnv(t) + + var got remote.DeployConfig + var gotRaw []byte + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + gotRaw, _ = io.ReadAll(r.Body) + _ = json.Unmarshal(gotRaw, &got) + w.Header().Set("Content-Type", "application/json") + w.Write([]byte(`{"deployed":true,"environment":"testenv","base_url":"http://198.51.100.9:8000/v1"}`)) + })) + defer server.Close() + stubDeploySeams(t, server.URL, "undeployed") + writeDeployEnvSpinloop(t, "testenv") + + out := captureStdout(t, func() { + if err := cmdRemoteDeploy([]string{"--instance-type", "g6e.2xlarge"}); err != nil { + t.Errorf("cmdRemoteDeploy: %v", err) + } + }) + + if got.InstanceType != "g6e.2xlarge" { + t.Errorf("posted instanceType = %q, want g6e.2xlarge", got.InstanceType) + } + if !strings.Contains(string(gotRaw), `"instanceType":"g6e.2xlarge"`) { + t.Errorf("instanceType did not reach the body: %s", gotRaw) + } + if !strings.Contains(out, "instance: g6e.2xlarge") { + t.Errorf("plan should name the type, got:\n%s", out) + } + }) + + t.Run("an untyped deploy omits the field entirely", func(t *testing.T) { + isolateConfig(t) + stubAWSEnv(t) + + var gotRaw []byte + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + gotRaw, _ = io.ReadAll(r.Body) + w.Header().Set("Content-Type", "application/json") + w.Write([]byte(`{"deployed":true,"environment":"testenv","base_url":"http://198.51.100.9:8000/v1"}`)) + })) + defer server.Close() + stubDeploySeams(t, server.URL, "undeployed") + writeDeployEnvSpinloop(t, "testenv") + + if err := cmdRemoteDeploy(nil); err != nil { + t.Errorf("cmdRemoteDeploy: %v", err) + } + // Absent, not null and not empty: an untyped deploy sends exactly what + // a control plane predating the field expects. + if strings.Contains(string(gotRaw), "instanceType") { + t.Errorf("instanceType should be omitted when untyped: %s", gotRaw) + } + }) + + t.Run("a dry run names the default machine when untyped", func(t *testing.T) { + isolateConfig(t) + writeDeployEnvSpinloop(t, "testenv") + out := captureStdout(t, func() { + if err := cmdRemoteDeploy([]string{"--dry-run"}); err != nil { + t.Errorf("cmdRemoteDeploy --dry-run: %v", err) + } + }) + if !strings.Contains(out, "instance: the control plane's default") { + t.Errorf("--dry-run should state the default machine, got:\n%s", out) + } + }) + + t.Run("a value outside the shape is refused before anything is sent", func(t *testing.T) { + isolateConfig(t) + stubDeploySeams(t, "https://unused", "undeployed") + writeDeployEnvSpinloop(t, "testenv") + err := cmdRemoteDeploy([]string{"--instance-type", "g6exlarge"}) + if err == nil || !strings.Contains(err.Error(), "--instance-type") { + t.Errorf("want a --instance-type validation error, got %v", err) + } + }) +} + // A supplied key is resolved from the environment the Spinloop's local // environment populated, reaches the signed body as a request-scoped field, // and the report says what happened to it — the action, never the value. diff --git a/docs/commands/fleet.md b/docs/commands/fleet.md index bdf80db0..2b992c45 100644 --- a/docs/commands/fleet.md +++ b/docs/commands/fleet.md @@ -105,6 +105,27 @@ blanking the fleet. See [`examples/fleet-remote`](../../examples/fleet-remote/README.md) and [`examples/fleet-mixed`](../../examples/fleet-mixed/README.md). +A `kind: remote` node may also name the EC2 instance type its environment +launches as, with `instance-type` (a family and size separated by a dot, e.g. +`g6e.xlarge`): + +```yaml +nodes: + - name: qwen + kind: remote + instance-type: g6e.2xlarge +``` + +It is a property of the cloud environment, not of the fleet's view of it: +`fleet deploy` records it on the environment, and the environment's next +**fresh** launch uses it. A re-wake of a stopped instance keeps the type it +was launched with — EC2 cannot resize a running or stopped box — so a changed +value takes effect only after the instance is terminated (an idle sweep or +`spinloop remote stop`) and launched again. Omitted, the environment launches +as its control plane's default type. Naming `instance-type` on a `kind: daemon` +node is a configuration error: a daemon's hardware is the operator's to choose, +not the fleet file's. + ### A node's Spinloop source Both `fleet deploy` (for a `kind: remote` node's environment) and `fleet diff --git a/docs/commands/remote.md b/docs/commands/remote.md index 7f29af9f..26d02c26 100644 --- a/docs/commands/remote.md +++ b/docs/commands/remote.md @@ -377,6 +377,14 @@ pin (`1.26.1`; a leading `v` is fine) it installs exactly that. The pin is environment state, not engine state: it takes effect at the next boot, so a running instance keeps the daemon it was deployed with. +`--instance-type` names the EC2 instance type the environment's instances +launch as (a family and size separated by a dot, e.g. `g6e.2xlarge`). Like the +pin, it is recorded on the environment at deploy time and applies from its next +**fresh** launch: a re-wake of a stopped instance keeps the type it launched +with, so a changed value takes effect only once the instance is terminated and +launched again. Without it the environment launches as the control plane's +default type. + Deploying doesn't start anything. If the shared bucket doesn't have those weights yet it fetches them (about 15–20 minutes, entirely on its side) and says so; wait for that before your first `start`, or the model won't be there. @@ -421,6 +429,7 @@ included) for every `kind: remote` node a fleet file names — or a chosen few | `-n`, `--dry-run` | `deploy` only: print what would be sent, without sending it | | `--reseed` | `deploy` only: re-fetch the weights even if they are already in S3 | | `--spinloop-version` | `deploy` only: the spinloop release fresh boots install (default: the latest published release) | +| `--instance-type` | `deploy` only: the EC2 instance type the environment's instances launch as (e.g. `g6e.xlarge`); recorded on the environment, applied on its next fresh launch (default: the control plane's default type) | | `--api-key-env` | `deploy` only: name the environment variable holding the engine key to create or rotate; with no flag the stored key is kept | `bootstrap` and `bake` have their own too (`--ref`, `--dir`, `--region`, diff --git a/docs/openapi.yaml b/docs/openapi.yaml index fde05cee..2aca8c5c 100644 --- a/docs/openapi.yaml +++ b/docs/openapi.yaml @@ -599,6 +599,15 @@ components: of a release tag is not part of the version (1.26.1, not v1.26.1). The environment's control plane reads it when it renders the boot script; the daemon does not act on it. + instanceType: + type: string + description: | + The EC2 instance type the environment's instances launch as — a + family and size separated by a dot (g6e.xlarge). Empty or absent + means launch as the control plane's default type. A property of + the deployment: the environment's control plane reads it when it + launches a fresh instance; a re-wake of a stopped instance keeps + the type it launched with. The daemon does not act on it. Message: type: object diff --git a/examples/fleet-remote/fleet.yaml b/examples/fleet-remote/fleet.yaml index 084d75e2..2c27efad 100644 --- a/examples/fleet-remote/fleet.yaml +++ b/examples/fleet-remote/fleet.yaml @@ -24,6 +24,7 @@ nodes: - name: qwen kind: remote + instance-type: g6e.2xlarge # EC2 type this environment launches as (default: the control plane's) - name: llama kind: remote diff --git a/internal/fleet/config.go b/internal/fleet/config.go index 1b702ea8..13841ebe 100644 --- a/internal/fleet/config.go +++ b/internal/fleet/config.go @@ -127,6 +127,13 @@ type NodeConfig struct { // beside the fleet file, before either command gives up on it. Not // read by any other fleet command. File string `yaml:"file"` + // InstanceType names the EC2 instance type a kind: remote node's + // environment launches as, read by `spinloop fleet deploy` into the + // deploy config it derives. It is a property of the remote environment + // only — a kind: daemon node's hardware is the operator's to choose, so + // naming one there is a configuration error. Empty means the node's + // environment launches as the control plane's default type. + InstanceType string `yaml:"instance-type"` } // EngineOverride is a node's declared engine endpoint. Each field is optional @@ -213,6 +220,11 @@ func (c *Config) validate() error { if n.Host == "" { return fmt.Errorf("node %q has no host", n.Name) } + if n.InstanceType != "" { + return fmt.Errorf( + "node %q is kind %q: instance-type names the cloud environment's machine, and a daemon's hardware is the operator's to choose, not the fleet file's", + n.Name, KindDaemon) + } case KindRemote: // The node's name *is* the registered environment's key, so it must // be env-shaped; a path-like name would be read as a registry @@ -222,6 +234,11 @@ func (c *Config) validate() error { "node %q is kind %q: its name must be a registered environment name (no /, no .json)", n.Name, KindRemote) } + if n.InstanceType != "" && !remote.IsInstanceType(n.InstanceType) { + return fmt.Errorf( + "node %q has instance-type %q, which is not shaped like an EC2 instance type (a family and size separated by a dot, e.g. g6e.xlarge)", + n.Name, n.InstanceType) + } default: return fmt.Errorf( "node %q has kind %q: supported kinds are %q and %q", diff --git a/internal/fleet/config_test.go b/internal/fleet/config_test.go index d9f220fc..c37b5198 100644 --- a/internal/fleet/config_test.go +++ b/internal/fleet/config_test.go @@ -94,11 +94,13 @@ nodes: func TestLoadRejectsIncompleteNodes(t *testing.T) { for name, body := range map[string]string{ - "no nodes": "nodes: []\n", - "no name": "nodes:\n - host: a.local\n", - "no host": "nodes:\n - name: studio\n", - "remote name is a path": "nodes:\n - name: a/b\n kind: remote\n", - "remote name has .json": "nodes:\n - name: prod.json\n kind: remote\n", + "no nodes": "nodes: []\n", + "no name": "nodes:\n - host: a.local\n", + "no host": "nodes:\n - name: studio\n", + "remote name is a path": "nodes:\n - name: a/b\n kind: remote\n", + "remote name has .json": "nodes:\n - name: prod.json\n kind: remote\n", + "daemon instance-type": "nodes:\n - name: studio\n host: a.local\n instance-type: g6e.xlarge\n", + "remote bad instance-type": "nodes:\n - name: prod\n kind: remote\n instance-type: g6exlarge\n", } { t.Run(name, func(t *testing.T) { if _, err := Load(writeFleet(t, body, "")); err == nil { @@ -137,6 +139,59 @@ nodes: } } +// A kind-remote node may name the instance type its environment launches as; +// the field parses onto the node so `fleet deploy` can read it. +func TestLoadRemoteKindInstanceType(t *testing.T) { + path := writeFleet(t, ` +nodes: + - name: prod + kind: remote + instance-type: g6e.2xlarge +`, "") + cfg, err := Load(path) + if err != nil { + t.Fatal(err) + } + if cfg.Nodes[0].InstanceType != "g6e.2xlarge" { + t.Errorf("instance-type = %q, want g6e.2xlarge", cfg.Nodes[0].InstanceType) + } +} + +// A remote node naming a malformed instance type is a configuration error that +// names both the node and the value, not a silent deploy of junk. +func TestLoadRemoteKindBadInstanceTypeNamesIt(t *testing.T) { + _, err := Load(writeFleet(t, ` +nodes: + - name: prod + kind: remote + instance-type: g6exlarge +`, "")) + if err == nil { + t.Fatal("accepted a remote node with a malformed instance-type") + } + msg := err.Error() + if !strings.Contains(msg, "prod") || !strings.Contains(msg, "g6exlarge") { + t.Errorf("error %q does not name the node and the value", msg) + } +} + +// A daemon node's hardware is the operator's to choose, so naming an instance +// type on one is a configuration error, not silently ignored. +func TestLoadDaemonKindInstanceTypeNamesIt(t *testing.T) { + _, err := Load(writeFleet(t, ` +nodes: + - name: studio + host: a.local + instance-type: g6e.xlarge +`, "")) + if err == nil { + t.Fatal("accepted a daemon node with an instance-type") + } + if !strings.Contains(err.Error(), "studio") { + t.Errorf("error %q does not name the node", err) + } +} + func TestResolveDefaultAndExplicit(t *testing.T) { path := writeFleet(t, "nodes:\n - name: studio\n host: a.local\n", "") diff --git a/internal/remote/remote.go b/internal/remote/remote.go index 2d0cc694..fc8144ac 100644 --- a/internal/remote/remote.go +++ b/internal/remote/remote.go @@ -18,6 +18,7 @@ import ( "net/url" "os" "path/filepath" + "regexp" "strings" "time" @@ -261,6 +262,28 @@ type DeployConfig struct { // Empty means the boot installs the latest published release. Omitted // when empty, so an unpinned deploy sends exactly what it always did. SpinloopVersion string `json:"spinloopVersion,omitempty"` + // InstanceType is the EC2 instance type the environment's instances launch + // as. Empty means launch as the control plane's default type. It is a + // property of the deployment, stored in the deploy config and read back on + // the next fresh launch — a re-wake of a stopped instance keeps the type it + // was launched with. Omitted when empty, so an untyped deploy sends exactly + // what it always did. + InstanceType string `json:"instanceType,omitempty"` +} + +// instanceTypePattern is the shape of an EC2 instance type: a lowercase family +// and size separated by a single dot. The family may be hyphenated, as in +// u7i-6tb and mac2-m2; the size is lowercase alphanumerics (xlarge, 112xlarge, +// metal). Deliberately permissive to every current EC2 family while rejecting +// obvious junk before it reaches a RunInstances call. +var instanceTypePattern = regexp.MustCompile(`^[a-z0-9]+(?:-[a-z0-9]+)*\.[a-z0-9]+$`) + +// IsInstanceType reports whether value is shaped like an EC2 instance type. +// It is the guard a deploy (flag or fleet file) runs before sending a type the +// control plane would otherwise reject at launch; a name that fails it is +// reported by its caller, which has the value to name. +func IsInstanceType(value string) bool { + return instanceTypePattern.MatchString(value) } // Deploy creates (or updates) cfg.Environment on the control plane and sets diff --git a/internal/remote/remote_test.go b/internal/remote/remote_test.go index 346460e7..734ccd6e 100644 --- a/internal/remote/remote_test.go +++ b/internal/remote/remote_test.go @@ -1033,6 +1033,68 @@ func TestDeploy_SpinloopVersionOmittedWhenUnpinned(t *testing.T) { } } +func TestDeploy_InstanceTypeReachesTheRequest(t *testing.T) { + stubAWSEnv(t) + var gotBody []byte + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + gotBody, _ = io.ReadAll(r.Body) + w.Write([]byte(`{"state":"deployed","deployed":true}`)) + })) + defer server.Close() + + cfg := Config{DeployURL: server.URL, Region: "eu-west-1"} + dc := DeployConfig{Runner: "vllm", ModelID: "org/model", InstanceType: "g6e.2xlarge"} + if _, err := Deploy(context.Background(), cfg, dc, "", false, ""); err != nil { + t.Fatal(err) + } + if !strings.Contains(string(gotBody), `"instanceType":"g6e.2xlarge"`) { + t.Errorf("instanceType did not reach the request body: %s", gotBody) + } +} + +// An untyped deploy sends exactly the body a control plane predating the field +// expects — the key is absent, not null or empty. +func TestDeploy_InstanceTypeOmittedWhenUntyped(t *testing.T) { + stubAWSEnv(t) + var gotBody []byte + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + gotBody, _ = io.ReadAll(r.Body) + w.Write([]byte(`{"state":"deployed","deployed":true}`)) + })) + defer server.Close() + + cfg := Config{DeployURL: server.URL, Region: "eu-west-1"} + dc := DeployConfig{Runner: "vllm", ModelID: "org/model"} + if _, err := Deploy(context.Background(), cfg, dc, "", false, ""); err != nil { + t.Fatal(err) + } + if strings.Contains(string(gotBody), "instanceType") { + t.Errorf("instanceType should be omitted when empty: %s", gotBody) + } +} + +// TestIsInstanceType pins the shape guard a deploy runs before sending a type +// the control plane would otherwise reject at launch. +func TestIsInstanceType(t *testing.T) { + for _, ok := range []string{ + "g6e.xlarge", "g6e.2xlarge", "g5.2xlarge", "trn1.2xlarge", + "inf2.xlarge", "c7g.large", "m5.xlarge", "u7i-6tb.112xlarge", + "mac2-m2.2xlarge", "p4d.24xlarge", "g6e.metal", + } { + if !IsInstanceType(ok) { + t.Errorf("IsInstanceType(%q) = false, want true", ok) + } + } + for _, bad := range []string{ + "", "g6exlarge", "G6E.xlarge", "g6e.xlarge.extra", "g6e.", ".xlarge", + "g6e.xlarge ", " g6e.xlarge", "g6e_xlarge", "g6e..xlarge", + } { + if IsInstanceType(bad) { + t.Errorf("IsInstanceType(%q) = true, want false", bad) + } + } +} + func TestDeploy_ReseedReachesTheRequest(t *testing.T) { stubAWSEnv(t) var gotBody []byte diff --git a/openspec/changes/remote-instance-type/tasks.md b/openspec/changes/remote-instance-type/tasks.md index 278e9422..e2e12c3b 100644 --- a/openspec/changes/remote-instance-type/tasks.md +++ b/openspec/changes/remote-instance-type/tasks.md @@ -1,10 +1,10 @@ ## 1. Go: instance type in the deploy config and a shared validator -- [ ] 1.1 Add `InstanceType string` (JSON key `instanceType`, `omitempty`) to +- [x] 1.1 Add `InstanceType string` (JSON key `instanceType`, `omitempty`) to `DeployConfig` in `internal/remote/remote.go`. Verify with a unit test that a config carrying the type marshals an `instanceType` key and a config without it omits the key (matching the `SpinloopVersion` pattern). -- [ ] 1.2 Add an exported EC2 instance-type pattern and validator in +- [x] 1.2 Add an exported EC2 instance-type pattern and validator in `internal/remote` (shape `family.size`, lowercase, family may be hyphenated). Verify with a unit test that it accepts `g6e.xlarge`, `u7i-6tb.112xlarge`, `mac2-m2.2xlarge`, `trn1.2xlarge` and rejects empty, `g6exlarge`, @@ -12,10 +12,10 @@ ## 2. Go: the fleet file field -- [ ] 2.1 Add `InstanceType string` (YAML key `instance-type`) to `NodeConfig` +- [x] 2.1 Add `InstanceType string` (YAML key `instance-type`) to `NodeConfig` in `internal/fleet/config.go`. Verify the field parses from a `fleet.yaml` listing a `kind: remote` node with `instance-type: g6e.2xlarge`. -- [ ] 2.2 In `validate()`, reject `instance-type` on a `kind: daemon` node and +- [x] 2.2 In `validate()`, reject `instance-type` on a `kind: daemon` node and check its shape (via the `internal/remote` validator) on a `kind: remote` node. Verify with tests: a daemon node naming one is rejected naming the node; a remote node with a malformed value is rejected naming the node and @@ -23,16 +23,16 @@ ## 3. Go: `remote deploy --instance-type` -- [ ] 3.1 Add an `instanceType string` field to `deployOpts` and an +- [x] 3.1 Add an `instanceType string` field to `deployOpts` and an `--instance-type` flag to `remoteDeployCmd`, threading it through `runRemoteDeploy` into the `deployOpts` passed to `runDeploy`. Verify `spinloop remote deploy --help` lists the flag. -- [ ] 3.2 In `runDeploy`, when `opts.instanceType` is non-empty, validate it +- [x] 3.2 In `runDeploy`, when `opts.instanceType` is non-empty, validate it (naming the value on failure, before any send) and set `dc.InstanceType`; treat an empty/whitespace value as absent. Verify with a test that a malformed value fails before the deploy seam is reached and a valid value is set on the derived config. -- [ ] 3.3 Print the resolved instance type in the deploy plan (the type when +- [x] 3.3 Print the resolved instance type in the deploy plan (the type when set, otherwise a statement that the environment launches as the control plane's default), alongside the runner and model. Verify `remote deploy --dry-run --instance-type g6e.2xlarge` prints `g6e.2xlarge` and a run without @@ -40,7 +40,7 @@ ## 4. Go: `fleet deploy` threads the node's value -- [ ] 4.1 In `deployOneNode`, set the targeted node's `InstanceType` into the +- [x] 4.1 In `deployOneNode`, set the targeted node's `InstanceType` into the `deployOpts` passed to `runDeploy` (per-node; a node naming none leaves it empty). Verify with a test that `fleet deploy --dry-run` for a node declaring `instance-type` shows that type in its plan and a node declaring none shows @@ -48,43 +48,43 @@ ## 5. TypeScript: parse and validate the field -- [ ] 5.1 Add `instanceType?: string` to the `DeployConfig` interface and parse +- [x] 5.1 Add `instanceType?: string` to the `DeployConfig` interface and parse + validate it in `parseDeployConfig` in `remote/lambda/shared/deploy-config.ts` (absent → `undefined`; present → non-empty string matching the same `family.size` shape, else a thrown error naming the value). Verify by extending `remote/test/deploy-config.test.ts`: a config with the type parses it, one without leaves it `undefined`, and a malformed value throws. -- [ ] 5.2 Confirm `writeDeployConfig`/`readDeployConfig` round-trip the field +- [x] 5.2 Confirm `writeDeployConfig`/`readDeployConfig` round-trip the field (they serialise the whole config). Verify with a test that a config written with `instanceType` reads it back. ## 6. TypeScript: the start launches with the stored type -- [ ] 6.1 In `remote/lambda/start/index.ts`, launch with +- [x] 6.1 In `remote/lambda/start/index.ts`, launch with `deployConfig.instanceType` when present, else the `INSTANCE_TYPE` env var, in the fresh-launch path. Verify by extending `remote/test/start-launch.test.ts`: a deploy config naming a type launches as it, and one naming none launches as the env-var default. -- [ ] 6.2 Update the no-capacity reply to name the instance type it was trying +- [x] 6.2 Update the no-capacity reply to name the instance type it was trying (rather than a hard-coded `g6e`). Verify the no-capacity test asserts the type appears in the message. ## 7. Whole-repo verification -- [ ] 7.1 Run `gofmt -l` over the touched Go files and confirm no output; run +- [x] 7.1 Run `gofmt -l` over the touched Go files and confirm no output; run `go vet ./...` and confirm it is clean. -- [ ] 7.2 Run `go test ./... -cover` and confirm the suite passes with total +- [x] 7.2 Run `go test ./... -cover` and confirm the suite passes with total coverage still >= 80%. -- [ ] 7.3 From `remote/`, run `pnpm build` (tsc type-check) and `pnpm test` +- [x] 7.3 From `remote/`, run `pnpm build` (tsc type-check) and `pnpm test` (vitest) and confirm both pass. ## 8. Docs and examples -- [ ] 8.1 Document `instance-type` in the `fleet.yaml` reference (remote-only, +- [x] 8.1 Document `instance-type` in the `fleet.yaml` reference (remote-only, the shape it must take, that it is deployed by `fleet deploy`) and `--instance-type` in the `remote deploy` reference (recorded per environment, applied on the next fresh launch, default when omitted). Verify the docs build/lint if the repo has a docs check, else re-read for accuracy. -- [ ] 8.2 Add an `instance-type` to a `kind: remote` node in the fleet/remote +- [x] 8.2 Add an `instance-type` to a `kind: remote` node in the fleet/remote example (`examples/fleet-remote/fleet.yaml`) with a short comment, so the field is discoverable. Verify the example still matches the documented format. diff --git a/remote/lambda/shared/deploy-config.ts b/remote/lambda/shared/deploy-config.ts index 160827b3..caa0b852 100644 --- a/remote/lambda/shared/deploy-config.ts +++ b/remote/lambda/shared/deploy-config.ts @@ -53,6 +53,14 @@ export const COMPANION_FILENAME = /^[A-Za-z0-9._-]+$/; */ export const SPINLOOP_VERSION_PIN = /^[0-9A-Za-z.-]+$/; +/** + * What an EC2 instance type may look like: a lowercase family and size + * separated by a single dot. Deliberately permissive to every current family + * (hyphenated families like u7i-6tb and mac2-m2 are valid) while rejecting + * obvious junk before it reaches a RunInstances call. + */ +export const INSTANCE_TYPE = /^[a-z0-9]+(?:-[a-z0-9]+)*\.[a-z0-9]+$/; + /** * The boot's spinloop default: the latest published release, resolved by the * boot itself at launch — so "latest" stays latest on every fresh boot rather @@ -140,6 +148,14 @@ export interface DeployConfig { * daemon's stored deploy config. */ spinloopVersion: string; + /** + * The EC2 instance type the environment's instances launch as. Optional; + * absent/undefined means launch as the control plane's default type. A + * property of the deployment — stored here and read back on the next fresh + * launch, so a re-wake of a stopped instance keeps the type it launched + * with (EC2 cannot resize an existing instance). + */ + instanceType?: string; } /** @@ -181,6 +197,7 @@ export function parseDeployConfig(raw: string | undefined): DeployConfig { throw new Error('deploy-config.serveArgs must be an array of strings'); } const quant = typeof obj.quant === 'string' ? obj.quant : ''; + const instanceType = parseInstanceType(obj.instanceType); return { runner: obj.runner, modelId, @@ -193,9 +210,37 @@ export function parseDeployConfig(raw: string | undefined): DeployConfig { serveArgs: serveArgs as string[], companions: parseCompanions(obj.companions), spinloopVersion: parseSpinloopVersion(obj.spinloopVersion), + instanceType, }; } +/** + * Validate the optional EC2 instance type. Absent (or empty/whitespace) means + * the control plane's default, returned as undefined so an untyped config + * round-trips unchanged; present means a non-empty string shaped like + * `family.size`, else a thrown error naming the value. + */ +function parseInstanceType(raw: unknown): string | undefined { + if (raw === undefined || raw === null) { + return undefined; + } + if (typeof raw !== 'string') { + throw new Error(`deploy-config.instanceType must be a string, got ${JSON.stringify(raw)}`); + } + const value = raw.trim(); + if (value === '') { + return undefined; + } + if (!INSTANCE_TYPE.test(value)) { + throw new Error( + `deploy-config.instanceType must be an EC2 instance type (a family and size separated by a dot, e.g. g6e.xlarge), got ${JSON.stringify( + raw, + )}`, + ); + } + return value; +} + /** * Normalise the boot's spinloop version. Absent, empty or `latest` means the * boot resolves the latest published release at launch; a pin is stored minus diff --git a/remote/lambda/start/index.ts b/remote/lambda/start/index.ts index 943e7b5a..5eec1f24 100644 --- a/remote/lambda/start/index.ts +++ b/remote/lambda/start/index.ts @@ -545,12 +545,17 @@ async function launchAcrossAzs( }; } const userData = buildInferenceUserData(env, deployConfig); + // The instance type this environment launches as: the deploy config's own + // type when it names one, else the control plane's default (the env var). + // A re-wake of a stopped instance never reaches this path — it keeps the + // type the box was launched with; only a fresh launch reads the config here. + const instanceType = deployConfig.instanceType ?? INSTANCE_TYPE; const tried: string[] = []; for (const subnetId of SUBNET_IDS) { try { const instanceId = await runInstance({ imageId: ami.imageId, - instanceType: INSTANCE_TYPE, + instanceType, subnetId, securityGroupId, instanceProfileArn: INSTANCE_PROFILE_ARN, @@ -600,7 +605,7 @@ async function launchAcrossAzs( 503, { state: 'no-capacity', - message: `no g6e capacity in any of ${tried.length} availability zone(s); retry shortly`, + message: `no ${instanceType} capacity in any of ${tried.length} availability zone(s); retry shortly`, retry_after_seconds: 120, }, { 'retry-after': '120' }, diff --git a/remote/test/deploy-config.test.ts b/remote/test/deploy-config.test.ts index 52113750..836831ca 100644 --- a/remote/test/deploy-config.test.ts +++ b/remote/test/deploy-config.test.ts @@ -220,3 +220,46 @@ describe('parseDeployConfig spinloopVersion', () => { ); }); }); + +describe('parseDeployConfig instanceType', () => { + it('defaults to undefined, so a config written before the field existed parses unchanged', () => { + expect(parseDeployConfig(JSON.stringify(VLLM)).instanceType).toBeUndefined(); + }); + + it('round-trips a named type, and accepts the hyphenated families EC2 ships', () => { + for (const type of ['g6e.2xlarge', 'u7i-6tb.112xlarge', 'mac2-m2.2xlarge', 'trn1.2xlarge']) { + const withType = { ...VLLM, instanceType: type }; + expect(parseDeployConfig(JSON.stringify(withType))).toEqual(withType); + } + }); + + it('treats an empty or whitespace value as no type', () => { + expect(parseDeployConfig(JSON.stringify({ ...VLLM, instanceType: '' })).instanceType).toBeUndefined(); + expect(parseDeployConfig(JSON.stringify({ ...VLLM, instanceType: ' ' })).instanceType).toBeUndefined(); + }); + + it('rejects a malformed value, naming the field', () => { + for (const bad of ['g6exlarge', 'G6E.xlarge', 'g6e.xlarge.extra', 'g6e.', '.xlarge']) { + expect(() => parseDeployConfig(JSON.stringify({ ...VLLM, instanceType: bad }))).toThrow( + /instanceType/, + ); + } + }); + + it('rejects a non-string value', () => { + expect(() => parseDeployConfig(JSON.stringify({ ...VLLM, instanceType: 7 }))).toThrow( + /instanceType/, + ); + }); + + it('survives the write-then-read path the Lambdas perform (stringify on write, parse on read)', () => { + // writeDeployConfig stores JSON.stringify(config); readDeployConfig runs + // parseDeployConfig over the stored value. instanceType is a plain field, + // so a config written with the type reads it back. + const written: DeployConfig = { ...LLAMACPP, instanceType: 'g6e.2xlarge' }; + const stored = JSON.stringify(written); // what writeDeployConfig puts in SSM + const readBack = parseDeployConfig(stored); // what readDeployConfig returns + expect(readBack.instanceType).toBe('g6e.2xlarge'); + expect(readBack).toEqual(written); + }); +}); diff --git a/remote/test/start-launch.test.ts b/remote/test/start-launch.test.ts index bc319a18..db7d1859 100644 --- a/remote/test/start-launch.test.ts +++ b/remote/test/start-launch.test.ts @@ -170,3 +170,69 @@ describe('fresh launch', () => { expect(body).toContain('"modelId": "/opt/llm/model/model.gguf"'); }); }); + +describe('instance type', () => { + // A fresh launch reads the type from the deploy config; a re-wake never + // reaches this path, so only these tests exercise the choice. + it('launches as the deploy config type when it names one', async () => { + readDeployConfig.mockResolvedValue({ + runner: 'llamacpp', + modelId: 'org/model', + quant: 'Q4_K_M', + weightsPrefix: 'llamacpp/org/model/Q4_K_M', + contextSize: 32768, + servedModelName: 'friendly', + serveArgs: [], + companions: {}, + spinloopVersion: 'latest', + instanceType: 'g6e.2xlarge', + }); + findLatestAmi.mockResolvedValue({ imageId: 'ami-test1', rootVolumeSizeGb: 80 }); + + const result = await handler(wakeEvent, context); + expect(structured(result).statusCode).toBe(200); + + expect(runInstance).toHaveBeenCalledWith( + expect.objectContaining({ instanceType: 'g6e.2xlarge' }), + ); + }); + + it('launches as the env-var default when the config names none', async () => { + // The beforeEach config carries no instanceType, so the control plane's + // default (LAMBDA_ENV.INSTANCE_TYPE) is what the launch uses. + findLatestAmi.mockResolvedValue({ imageId: 'ami-test1', rootVolumeSizeGb: 80 }); + + const result = await handler(wakeEvent, context); + expect(structured(result).statusCode).toBe(200); + + expect(runInstance).toHaveBeenCalledWith( + expect.objectContaining({ instanceType: 'g6e.xlarge' }), + ); + }); + + it('names the type it was trying when no AZ has capacity', async () => { + readDeployConfig.mockResolvedValue({ + runner: 'llamacpp', + modelId: 'org/model', + quant: 'Q4_K_M', + weightsPrefix: 'llamacpp/org/model/Q4_K_M', + contextSize: 32768, + servedModelName: 'friendly', + serveArgs: [], + companions: {}, + spinloopVersion: 'latest', + instanceType: 'g6e.2xlarge', + }); + findLatestAmi.mockResolvedValue({ imageId: 'ami-test1', rootVolumeSizeGb: 80 }); + runInstance.mockRejectedValue( + Object.assign(new Error('no capacity'), { name: 'InsufficientInstanceCapacity' }), + ); + + const result = await handler(wakeEvent, context); + const reply = JSON.parse(structured(result).body); + expect(structured(result).statusCode).toBe(503); + expect(reply.state).toBe('no-capacity'); + // The reply names the machine it was trying to launch, not a hard-coded one. + expect(reply.message).toContain('g6e.2xlarge'); + }); +}); From 11c4bbb8af113e90db5123e23ed5d3b6d2a76146 Mon Sep 17 00:00:00 2001 From: Pete Cornish Date: Sat, 12 Sep 2026 19:57:49 +0100 Subject: [PATCH 3/3] docs: archive remote-instance-type and sync its four specs --- .../.openspec.yaml | 0 .../design.md | 0 .../proposal.md | 0 .../specs/endpoint-lifecycle/spec.md | 0 .../specs/environment-deployment/spec.md | 0 .../specs/fleet-client/spec.md | 0 .../specs/remote-node/spec.md | 0 .../2026-09-12-remote-instance-type}/tasks.md | 0 openspec/specs/endpoint-lifecycle/spec.md | 41 ++++++++++++- openspec/specs/environment-deployment/spec.md | 61 +++++++++++++++++++ openspec/specs/fleet-client/spec.md | 23 +++++++ openspec/specs/remote-node/spec.md | 28 +++++++++ 12 files changed, 151 insertions(+), 2 deletions(-) rename openspec/changes/{remote-instance-type => archive/2026-09-12-remote-instance-type}/.openspec.yaml (100%) rename openspec/changes/{remote-instance-type => archive/2026-09-12-remote-instance-type}/design.md (100%) rename openspec/changes/{remote-instance-type => archive/2026-09-12-remote-instance-type}/proposal.md (100%) rename openspec/changes/{remote-instance-type => archive/2026-09-12-remote-instance-type}/specs/endpoint-lifecycle/spec.md (100%) rename openspec/changes/{remote-instance-type => archive/2026-09-12-remote-instance-type}/specs/environment-deployment/spec.md (100%) rename openspec/changes/{remote-instance-type => archive/2026-09-12-remote-instance-type}/specs/fleet-client/spec.md (100%) rename openspec/changes/{remote-instance-type => archive/2026-09-12-remote-instance-type}/specs/remote-node/spec.md (100%) rename openspec/changes/{remote-instance-type => archive/2026-09-12-remote-instance-type}/tasks.md (100%) diff --git a/openspec/changes/remote-instance-type/.openspec.yaml b/openspec/changes/archive/2026-09-12-remote-instance-type/.openspec.yaml similarity index 100% rename from openspec/changes/remote-instance-type/.openspec.yaml rename to openspec/changes/archive/2026-09-12-remote-instance-type/.openspec.yaml diff --git a/openspec/changes/remote-instance-type/design.md b/openspec/changes/archive/2026-09-12-remote-instance-type/design.md similarity index 100% rename from openspec/changes/remote-instance-type/design.md rename to openspec/changes/archive/2026-09-12-remote-instance-type/design.md diff --git a/openspec/changes/remote-instance-type/proposal.md b/openspec/changes/archive/2026-09-12-remote-instance-type/proposal.md similarity index 100% rename from openspec/changes/remote-instance-type/proposal.md rename to openspec/changes/archive/2026-09-12-remote-instance-type/proposal.md diff --git a/openspec/changes/remote-instance-type/specs/endpoint-lifecycle/spec.md b/openspec/changes/archive/2026-09-12-remote-instance-type/specs/endpoint-lifecycle/spec.md similarity index 100% rename from openspec/changes/remote-instance-type/specs/endpoint-lifecycle/spec.md rename to openspec/changes/archive/2026-09-12-remote-instance-type/specs/endpoint-lifecycle/spec.md diff --git a/openspec/changes/remote-instance-type/specs/environment-deployment/spec.md b/openspec/changes/archive/2026-09-12-remote-instance-type/specs/environment-deployment/spec.md similarity index 100% rename from openspec/changes/remote-instance-type/specs/environment-deployment/spec.md rename to openspec/changes/archive/2026-09-12-remote-instance-type/specs/environment-deployment/spec.md diff --git a/openspec/changes/remote-instance-type/specs/fleet-client/spec.md b/openspec/changes/archive/2026-09-12-remote-instance-type/specs/fleet-client/spec.md similarity index 100% rename from openspec/changes/remote-instance-type/specs/fleet-client/spec.md rename to openspec/changes/archive/2026-09-12-remote-instance-type/specs/fleet-client/spec.md diff --git a/openspec/changes/remote-instance-type/specs/remote-node/spec.md b/openspec/changes/archive/2026-09-12-remote-instance-type/specs/remote-node/spec.md similarity index 100% rename from openspec/changes/remote-instance-type/specs/remote-node/spec.md rename to openspec/changes/archive/2026-09-12-remote-instance-type/specs/remote-node/spec.md diff --git a/openspec/changes/remote-instance-type/tasks.md b/openspec/changes/archive/2026-09-12-remote-instance-type/tasks.md similarity index 100% rename from openspec/changes/remote-instance-type/tasks.md rename to openspec/changes/archive/2026-09-12-remote-instance-type/tasks.md diff --git a/openspec/specs/endpoint-lifecycle/spec.md b/openspec/specs/endpoint-lifecycle/spec.md index 31e4bf4f..b6cd999b 100644 --- a/openspec/specs/endpoint-lifecycle/spec.md +++ b/openspec/specs/endpoint-lifecycle/spec.md @@ -22,10 +22,21 @@ instance SHALL be given the environment's own stable address (its Elastic IP) so the environment's URL does not change between launches, and the request SHALL NOT report success until the model is answering — the caller receives one "ready", never a URL that is not yet serving. When no capacity can be -found anywhere, the response SHALL say so and SHALL be retryable rather than +found anywhere, the response SHALL say so, SHALL name the instance type it was +trying to launch, and SHALL be retryable rather than fatal. One shared set of lifecycle Lambdas SHALL serve every environment in the account, selecting the instance by the environment identifier. +A launch SHALL use the instance type the environment's deploy config names, +when it names one, and the control plane's default type otherwise. The type is +a property of the environment's deployment, read from the same stored deploy +config the start already reads for what to serve, so changing it is a deploy, +not a start. Because EC2 cannot change the type of an existing instance, a +stored type takes effect on a fresh launch only: a re-wake of a stopped +instance SHALL keep the type it was originally launched with, and a changed +type applies once the instance has been terminated — by an explicit stop or +the idle sweep — and relaunched. + Before launching or re-waking the instance, a start SHALL check that the environment's weights are present in shared storage, judged by the same completeness record the seeding writes rather than by the absence of an error. @@ -60,7 +71,8 @@ daemon runs. #### Scenario: No capacity anywhere - **WHEN** every configured zone is out of capacity -- **THEN** the response says so and indicates the caller may retry shortly +- **THEN** the response says so, names the instance type it was trying, and + indicates the caller may retry shortly #### Scenario: Starting the right environment @@ -80,6 +92,31 @@ daemon runs. provisioned throughput at the volume's ceiling and provisioned IOPS at four times that throughput +#### Scenario: A launch uses the environment's stored instance type + +- **WHEN** an environment's deploy config names an instance type and a start + launches a fresh instance for it +- **THEN** the instance is launched as that type + +#### Scenario: A launch with no stored type uses the control plane default + +- **WHEN** an environment's deploy config names no instance type and a start + launches a fresh instance for it +- **THEN** the instance is launched as the control plane's default type + +#### Scenario: A re-wake keeps the instance's original type + +- **WHEN** an environment's instance is stopped, its deploy config's instance + type is changed, and a start re-wakes the stopped instance +- **THEN** the instance comes back as the type it was originally launched + with, because a stopped instance is not resized + +#### Scenario: A changed type applies after the instance is terminated + +- **WHEN** an environment's deploy config instance type is changed, its + instance is terminated, and a later start launches it +- **THEN** the fresh instance is launched as the new type + #### Scenario: The control plane starts the engine on a fresh boot - **WHEN** a fresh instance's daemon first answers its control API diff --git a/openspec/specs/environment-deployment/spec.md b/openspec/specs/environment-deployment/spec.md index 86e068f2..3f05fb38 100644 --- a/openspec/specs/environment-deployment/spec.md +++ b/openspec/specs/environment-deployment/spec.md @@ -163,6 +163,67 @@ SHALL appear alongside the runner and model the plan already prints. - **THEN** the printed plan names `latest` as the spinloop the environment will run +### Requirement: Deploy accepts an optional instance type + +`spinloop remote deploy` SHALL accept an optional `--instance-type` flag naming +the EC2 instance type the environment's instances launch as. When the flag is +given, deploy SHALL record that type in the environment's stored deploy config +so the environment's next fresh launch uses it; when it is absent, deploy SHALL +record no type and the environment launches as the control plane's default. An +empty or whitespace-only value SHALL be treated as if the flag were not given. +A value that is not shaped like an EC2 instance type SHALL be refused before +anything is sent, naming the value. + +The instance type is a property of the environment's deployment, recorded in +the stored deploy config the way the spinloop version pin is — not a property +of a single start, and never derived from the Spinloop. + +#### Scenario: A type is recorded in the deploy config + +- **WHEN** `spinloop remote deploy` runs with `--instance-type g6e.2xlarge` +- **THEN** the environment's stored deploy config carries that type, and the + environment's next fresh launch uses it + +#### Scenario: No type leaves the launch on its default + +- **WHEN** `spinloop remote deploy` runs without `--instance-type` +- **THEN** the stored deploy config carries no instance type, and the + environment's launches use the control plane's default type + +#### Scenario: An empty type value is ignored + +- **WHEN** `spinloop remote deploy` is given an `--instance-type` whose value + is empty or whitespace only +- **THEN** it is treated as if no type were given + +#### Scenario: A malformed type is refused before sending + +- **WHEN** `spinloop remote deploy` is given an `--instance-type` that is not + shaped like an EC2 instance type +- **THEN** the command fails, naming the value, and nothing is sent to the + control plane + +### Requirement: The deploy plan shows the resolved instance type + +The plan `spinloop remote deploy` prints — including under `--dry-run`, before +any AWS work or send — SHALL state the instance type the environment will +launch as: the type named by `--instance-type` when one is given, otherwise a +statement that the environment launches as the control plane's default. It +SHALL appear alongside the runner and model the plan already prints. + +#### Scenario: A typed deploy prints the type + +- **WHEN** `spinloop remote deploy --dry-run` runs with `--instance-type + g6e.2xlarge` +- **THEN** the printed plan names `g6e.2xlarge` as the instance type the + environment will launch as + +#### Scenario: An untyped deploy prints the default + +- **WHEN** `spinloop remote deploy --dry-run` runs without `--instance-type` +- **THEN** the printed plan says the environment launches as the control + plane's default instance type + ### Requirement: Externally provided API key `spinloop remote deploy` SHALL accept an externally provided API key as a diff --git a/openspec/specs/fleet-client/spec.md b/openspec/specs/fleet-client/spec.md index d5633ad9..3d288e9c 100644 --- a/openspec/specs/fleet-client/spec.md +++ b/openspec/specs/fleet-client/spec.md @@ -299,6 +299,14 @@ targeted nodes. The resolved source (the path used, or the alias name when one was used) SHALL be reported alongside that node's plan, so which of the three supplied it is never left to be inferred. +Where a node declares an `instance-type` in the fleet file, the deploy config +derived for it SHALL carry that type, so the node's environment launches as +named — the same value a standalone `spinloop remote deploy --instance-type` +would record for the environment — and a node declaring none SHALL deploy an +environment on the control plane's default type. This keeps `fleet deploy` and +a matching standalone deploy in agreement about what a node's environment +launches as. + Nodes SHALL be deployed independently: one node already registered or live SHALL require `--overwrite` for that node exactly as a standalone `remote deploy` does, and refusing it SHALL NOT stop the other targeted nodes from @@ -319,6 +327,21 @@ any of them, exactly as a standalone `remote deploy --dry-run` does for one. the same as `spinloop remote deploy ./envs/gpu.Spinloop` would produce, and the resolved path is reported against that node +#### Scenario: A node's declared instance type is deployed + +- **WHEN** `fleet deploy` targets a `kind: remote` node declaring + `instance-type: g6e.2xlarge` +- **THEN** the environment it deploys launches as `g6e.2xlarge`, the same + value a standalone `spinloop remote deploy --instance-type g6e.2xlarge` of + the node's source would record + +#### Scenario: A node with no instance type deploys the default + +- **WHEN** `fleet deploy` targets a `kind: remote` node declaring no + `instance-type` +- **THEN** the environment it deploys launches as the control plane's default + instance type + #### Scenario: A node with no resolvable source fails only that node - **WHEN** `fleet deploy` targets two remote nodes and one declares no `file` diff --git a/openspec/specs/remote-node/spec.md b/openspec/specs/remote-node/spec.md index fba78f32..658e63ec 100644 --- a/openspec/specs/remote-node/spec.md +++ b/openspec/specs/remote-node/spec.md @@ -105,6 +105,16 @@ the view. A fleet of remote environments, or of daemons and remote environments SHALL be observable and drivable (status, metrics, start, stop) through the same fan-out as a fleet of daemons alone. +A node of kind `remote` MAY declare an optional `instance-type` naming the EC2 instance +type its environment launches as. It is a property of the remote environment only: a +`kind: daemon` node naming one SHALL be rejected, because a daemon's hardware is the +operator's to choose, not something the fleet file provisions. When present, the value +SHALL be checked for the shape of an EC2 instance type — a lowercase family and size +separated by a single dot, as in `g6e.xlarge` — when the file is read, so a typo is named +at parse rather than at launch. A `kind: remote` node naming no `instance-type` SHALL +deploy an environment that launches as the control plane's default, unchanged from before +the field existed. + #### Scenario: A fleet file lists a remote environment as a node - **WHEN** a fleet file lists a node of kind `remote` whose name is a registered @@ -126,6 +136,24 @@ as a fleet of daemons alone. - **THEN** the fleet file is rejected, naming the node, because the name is the environment key +#### Scenario: A remote node names its instance type + +- **WHEN** a fleet file lists a `kind: remote` node declaring `instance-type: g6e.2xlarge` +- **THEN** the file parses, and that node's environment is deployed to launch as + `g6e.2xlarge` + +#### Scenario: A daemon node naming an instance type is rejected + +- **WHEN** a fleet file lists a `kind: daemon` node declaring an `instance-type` +- **THEN** the file is rejected, naming the node, because instance type is a property of + a remote environment only + +#### Scenario: A malformed instance type is named at parse + +- **WHEN** a fleet file lists a `kind: remote` node whose `instance-type` is not shaped + like an EC2 instance type +- **THEN** the file is rejected, naming the node and the value + ### Requirement: Reading a remote environment's logs resumes without duplicating events A remote environment's log read SHALL resume from the position it last