Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 31 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Each worktree gets a numbered slot. The slot determines everything:
- **Database**: Created via `CREATE DATABASE ... TEMPLATE` (fast filesystem copy, not dump/restore)
- **Docker services**: Run in a dedicated Docker Compose project per worktree, grouped in Docker Desktop
- **Ports**: Offset by `portStride` (default 100) per slot
- **Env files**: Copied from main worktree, filled with safe defaults from configured `.env.example` files, and patched with the slot's values
- **Env files**: Copied from main worktree, filled with safe defaults from configured `.env.example` files, patched with the slot's values, and swept for absolute paths that would reach back into the main checkout

## Quick Start

Expand Down Expand Up @@ -174,7 +174,7 @@ Creates a new git worktree and sets up its isolated environment:
- If `[branch]` is **omitted**, a throwaway branch is auto-named from the base and today's date — like `main-20260723-nemanull` — and forked from `--base` (defaulting to `origin/main`, then `main`). Use this for a clean scratch environment without inventing a name: `wt new --base main`. Because wt invented the name, it is always a fresh branch: no `origin` lookup, and a same-named remote branch is never adopted.
2. Allocates the next available slot (or uses `--slot N`)
3. Creates a new Postgres database from the main DB as template
4. Copies configured `.env` files, fills missing safe defaults from examples, and patches each with slot-specific values
4. Copies configured `.env` files, fills missing safe defaults from examples, patches each with slot-specific values, and repoints absolute paths that still address the main checkout
5. Starts configured Docker services after the slot database exists
6. Runs `postSetup` commands (unless `--no-install`)

Expand Down Expand Up @@ -428,6 +428,35 @@ The `port` and `url` types require a `service` field that matches a name in `ser

Legacy `type: "redis"` patches are no longer supported. Declare Redis in `dockerServices` and patch `REDIS_URL` with `type: "url"` instead.

### Absolute Paths

Patches only touch the keys you list, so anything else in a developer's env file
is copied into the worktree verbatim — including absolute paths that still point
at the main checkout. A worktree that runs another checkout's build artifact
looks completely healthy: the path resolves, the process starts, health checks
pass, and it listens on the worktree's own isolated port. Only the code behind it
is wrong.

`wt new` and `wt setup` therefore sweep every env file they seed for absolute
path values, with no configuration and no key list to keep up to date:

- A path inside the main worktree, or inside one of its other worktrees, is
repointed at the same relative location in this worktree, and reported.
- A path inside an unrelated checkout is reported and left alone.
- Everything else is left alone. `/usr/bin/node` and `/tmp/cache` are shared on
purpose.

```
wt: rewrote 1 env value that pointed outside this worktree
server/.env RUNNER_BIN
was /home/dev/proj/apps/runner/target/release/runner
now /home/dev/proj/.worktrees/my-branch/apps/runner/target/release/runner
```

Worktrees created before this existed still hold the escaped values. Re-run plain
`wt setup` inside one to re-seed and repair it. See
[docs/env-path-isolation.md](docs/env-path-isolation.md).

### Env Seeding

`envFiles[].seedFrom` is for committed safe defaults. It maps a checked-in example file to the local env file in the same `envFiles` entry:
Expand Down
100 changes: 100 additions & 0 deletions docs/env-path-isolation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# Env path isolation

How `wt` keeps filesystem paths inside seeded env files pointing at the worktree
that owns them, and why an unpatched absolute path is dangerous.

## The failure mode

`wt new` and `wt setup` build a worktree's env files in three passes:

1. Copy each configured env file verbatim from the main worktree.
2. Fill in vars missing from the file using the committed `.env.example`.
3. Patch the vars enumerated in `wt.config.json` (database, ports, URLs, branch).

Pass 3 only touches keys the config names. Every other line survives the copy
untouched, including any value that is an absolute filesystem path.

A developer's env file often holds such a path. The committed example may use the
correct relative form, but a local file that was hand-edited once keeps whatever
was typed. Copied into a worktree, that value still addresses the main checkout.

This is worse than a stale setting because it is silent:

- The path resolves, so nothing errors.
- The artifact behind it is real and executable, so process spawns succeed and
health checks pass.
- Port isolation still works. A service started from the wrong binary listens on
exactly the port the worktree allocated, so every observable signal looks right.

The result inverts the point of the worktree: code changed in the worktree is not
the code that runs, and local verification of that component proves nothing.

The concrete report that motivated this
([wt#18](https://github.com/Tokenbooks/wt/issues/18)) was a Rust engine fix
verified in a worktree whose server had been spawning the main checkout's release
binary the whole time, because `server/.env` carried
`ACCOUNTING_RUST_RUNNER_BIN=/abs/path/to/main-checkout/...`.

## The rule

For every value in a seeded env file that is an absolute path (leading `/`, or
`~/` which is expanded first), `wt` locates the git working tree that encloses it
by walking up the directory chain for a `.git` entry. A primary checkout has a
`.git` directory, a linked worktree has a `.git` file, so both are found. The path
itself does not need to exist.

| Enclosing checkout | Action |
| --- | --- |
| None | Leave alone. `/usr/bin/node` and `/tmp/cache` are deliberately shared. |
| This worktree | Leave alone. Already correct. |
| The main worktree, or another worktree of it | Rewrite to the same relative location inside this worktree, and report it. |
| Any other checkout | Leave alone and warn, naming the file and key. |

Rewriting means replacing the enclosing checkout's root with this worktree's root
and keeping the rest of the path. It is an absolute path in, absolute path out;
nothing is made relative, so nothing depends on the working directory a consumer
happens to run from.

Rewriting is not conditional on the target existing. A worktree that has not built
an artifact yet gets a path to where that artifact belongs, and the consumer fails
loudly instead of silently reaching into another checkout.

Only the path token is touched. Quoting, an inline comment, trailing whitespace,
and a CRLF line ending are all written back byte-for-byte. Checkout roots are
compared by their physical location, so a symlinked spelling of the main worktree
is still recognised as the main worktree.

## What you see

`wt new` and `wt setup` print each rewrite and each warning to stderr. Both also
appear in `--json` output under a single `envPathEscapes` array, where an entry
that could be repointed carries a `rewritten` field and a warning does not.

```
wt: rewrote 1 env value that pointed outside this worktree
server/.env ACCOUNTING_RUST_RUNNER_BIN
was /home/dev/proj/apps/runner/target/release/runner
now /home/dev/proj/.worktrees/my-branch/apps/runner/target/release/runner
```

## Repairing an existing worktree

Worktrees created before this behaviour existed still hold the escaped values.
Re-running `wt setup` in the worktree re-copies and re-patches every configured
env file, which applies the rule and prints what it changed. Note that
`wt setup --repair` short-circuits when there is nothing to repair, so use plain
`wt setup`.

## Limits

- Only keys inside files listed in `wt.config.json` under `envFiles` are examined.
A path in a file `wt` does not manage is invisible to it.
- A value that packs several paths into one string, shell-`PATH` style, is not
parsed and is left alone. Any value containing a colon is skipped for this
reason, as is a value whose opening quote is never closed.
- A path into an unrelated checkout is reported, never rewritten. There is no way
to know what the equivalent location in this worktree would be.
- `wt env seed` only fills in missing defaults. It does not sweep paths, because
it has no worktree to repoint them at. Use `wt setup` for that.
- The nearest enclosing `.git` wins, so a path inside a submodule or a nested
repository is judged against that inner checkout, not the main worktree.
3 changes: 3 additions & 0 deletions skills/wt/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ For each `.env` file, examine every variable and classify:
| Redis connection URL (`redis://...`) | `url` | Yes (`redis`) |
| Just a port number | `port` | Yes |
| A URL containing a service port (`http://localhost:3000/...`) | `url` | Yes |
| A filesystem path | Skip — wt repoints these automatically | — |
| Anything else (API keys, secrets, flags) | Skip — do not patch | — |

**Step 3: Generate `wt.config.json`**
Expand Down Expand Up @@ -84,6 +85,8 @@ Validation rules:
- `baseDatabaseName` must match the actual DB name in `DATABASE_URL`
- If using `dockerServices`, Docker must be available locally
- Do not use legacy `type: "redis"` patches; use `type: "url"` for `REDIS_URL`
- Do not invent a patch entry for a filesystem path. `wt` repoints absolute paths that
address the main checkout at every seed, so no key list can fall behind.

**Step 4: Install wt**

Expand Down
2 changes: 2 additions & 0 deletions src/commands/new.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,7 @@ describe('new command branch selection', () => {
mockFindAvailableSlot.mockReturnValue(2);
mockAllocateServicePorts.mockResolvedValue({ ports: { web: 3200 }, drifts: [] });
mockCalculateDbName.mockReturnValue('myapp_wt2');
mockCopyAndPatchAllEnvFiles.mockReturnValue({ escapes: [] });
mockDatabaseExists.mockResolvedValue(false);
mockCreateDatabase.mockResolvedValue();
mockCreateWorktree.mockReturnValue(allocation.worktreePath);
Expand Down Expand Up @@ -485,6 +486,7 @@ describe('new command rollback on failure', () => {
mockFindAvailableSlot.mockReturnValue(2);
mockAllocateServicePorts.mockResolvedValue({ ports: { web: 3200, redis: 6579 }, drifts: [] });
mockCalculateDbName.mockReturnValue('myapp_wt2');
mockCopyAndPatchAllEnvFiles.mockReturnValue({ escapes: [] });
mockEnsureDockerServices.mockReturnValue({
projectName: 'wt-2-myapp-deadbeef',
services: ['redis'],
Expand Down
18 changes: 13 additions & 5 deletions src/commands/new.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,9 @@ import {
type WorktreeBranchSelection,
} from '../core/git';
import { resolveBaseRef } from '../core/audit';
import { extractErrorMessage, formatJson, formatSetupSummary, success, error } from '../output';
import { extractErrorMessage, formatEnvPathEscapes, formatJson, formatSetupSummary, success, error } from '../output';
import { loadConfig } from './setup';
import type { EnvPathEscape } from '../core/env-paths';
import type { Allocation, PortDrift } from '../types';
import { execSync } from 'node:child_process';
import * as fs from 'node:fs';
Expand All @@ -42,6 +43,7 @@ export interface CreateWorktreeResult {
readonly branchSelection: WorktreeBranchSelection;
readonly portDrifts: readonly PortDrift[];
readonly autoNamed: boolean;
readonly envPathEscapes: readonly EnvPathEscape[];
}

/** Read DATABASE_URL from the main worktree's .env file */
Expand Down Expand Up @@ -176,6 +178,7 @@ export async function createNewWorktree(
let worktreePath: string;
let actualBranch: string;
let allocation: Allocation;
let envPathEscapes: readonly EnvPathEscape[] = [];

try {
worktreePath = createWorktree(
Expand Down Expand Up @@ -213,11 +216,11 @@ export async function createNewWorktree(
});

log(`Patching ${config.envFiles.length} env file(s)...`);
copyAndPatchAllEnvFiles(config, mainRoot, worktreePath, {
envPathEscapes = copyAndPatchAllEnvFiles(config, mainRoot, worktreePath, {
dbName,
ports,
branchName: actualBranch,
});
}).escapes;

allocation = {
worktreePath,
Expand Down Expand Up @@ -289,7 +292,7 @@ export async function createNewWorktree(
}

log(`Ready — slot ${slot}, branch '${actualBranch}'.`);
return { slot, allocation, branchSelection, portDrifts, autoNamed };
return { slot, allocation, branchSelection, portDrifts, autoNamed, envPathEscapes };
}

/** Create a new worktree with full environment isolation */
Expand All @@ -298,11 +301,15 @@ export async function newCommand(
options: NewOptions,
): Promise<void> {
try {
const { slot, allocation, branchSelection, portDrifts, autoNamed } = await createNewWorktree(branchName, {
const { slot, allocation, branchSelection, portDrifts, autoNamed, envPathEscapes } = await createNewWorktree(branchName, {
...options,
quiet: options.json,
});

if (!options.json) {
process.stderr.write(formatEnvPathEscapes(envPathEscapes));
}

if (options.json) {
console.log(
formatJson(
Expand All @@ -314,6 +321,7 @@ export async function newCommand(
startPoint: branchSelection.startPoint ?? null,
autoNamed,
portDrifts,
envPathEscapes,
}),
),
);
Expand Down
2 changes: 1 addition & 1 deletion src/commands/setup.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ describe('setup command', () => {
allocations: { ...registry.allocations, [String(slot)]: allocation },
}));
mockWriteRegistry.mockImplementation(() => {});
mockCopyAndPatchAllEnvFiles.mockImplementation(() => {});
mockCopyAndPatchAllEnvFiles.mockReturnValue({ escapes: [] });
mockAllocateServicePorts.mockResolvedValue({ ports: { web: 3200 }, drifts: [] });
process.exitCode = 0;
});
Expand Down
8 changes: 6 additions & 2 deletions src/commands/setup.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ import {
computeServiceHashes,
} from '../core/docker-services';
import { getMainWorktreePath, isMainWorktree, getBranchName } from '../core/git';
import { extractErrorMessage, formatJson, formatRepairPreview, formatSetupSummary, success, error } from '../output';
import { extractErrorMessage, formatEnvPathEscapes, formatJson, formatRepairPreview, formatSetupSummary, success, error } from '../output';
import type { Allocation, PortChange, PortDrift, WtConfig } from '../types';

interface SetupOptions {
Expand Down Expand Up @@ -295,11 +295,14 @@ export async function setupCommand(
});

// Copy and patch env files
copyAndPatchAllEnvFiles(config, mainRoot, worktreePath, {
const envFiles = copyAndPatchAllEnvFiles(config, mainRoot, worktreePath, {
dbName,
ports,
branchName,
});
if (!options.json) {
process.stderr.write(formatEnvPathEscapes(envFiles.escapes));
}

// Update registry
const allocation: Allocation = {
Expand Down Expand Up @@ -330,6 +333,7 @@ export async function setupCommand(
portDrifts,
portChanges,
recreatedDockerServices: recreateServices,
envPathEscapes: envFiles.escapes,
repaired: !!options.repair,
dryRun: !!options.dryRun,
}),
Expand Down
23 changes: 20 additions & 3 deletions src/core/env-patcher.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import * as fs from 'node:fs';
import * as path from 'node:path';
import { rewriteEnvPaths, type EnvPathEscape, type EnvPathRoots } from './env-paths';
import type { EnvFileConfig, PatchConfig, PatchContext, WtConfig } from '../types';

type PortPatch = Extract<PatchConfig, { type: 'port' }>;
Expand All @@ -19,6 +20,10 @@ export interface SeedEnvFilesResult {
readonly files: SeedEnvFileResult[];
}

export interface CopyAndPatchEnvFilesResult {
readonly escapes: readonly EnvPathEscape[];
}

interface EnvAssignment {
readonly varName: string;
readonly line: string;
Expand Down Expand Up @@ -245,14 +250,15 @@ export function seedEnvFileDefaults(

/**
* Copy and patch all env files from the main worktree to the target worktree.
* Reads each source from mainRoot, patches it, writes to worktreeRoot.
* Absolute paths pointing back at the source checkout are repointed and reported.
* See docs/env-path-isolation.md.
*/
export function copyAndPatchAllEnvFiles(
config: WtConfig,
mainRoot: string,
worktreeRoot: string,
context: PatchContext,
): void {
): CopyAndPatchEnvFilesResult {
for (const envFile of config.envFiles) {
const sourcePath = path.join(mainRoot, envFile.source);
if (!fs.existsSync(sourcePath)) continue;
Expand All @@ -265,12 +271,23 @@ export function copyAndPatchAllEnvFiles(

seedEnvFileDefaults(config.envFiles, worktreeRoot, { dryRun: false });

const roots: EnvPathRoots = {
worktreeRoot,
mainRoot,
worktreesDir: path.resolve(mainRoot, config.baseWorktreePath),
};
const escapes: EnvPathEscape[] = [];

for (const envFile of config.envFiles) {
const targetPath = path.join(worktreeRoot, envFile.source);
if (!fs.existsSync(targetPath)) continue;

const content = fs.readFileSync(targetPath, 'utf-8');
const patched = patchEnvContent(content, envFile.patches ?? [], context);
fs.writeFileSync(targetPath, patched, 'utf-8');
const repointed = rewriteEnvPaths(patched, envFile.source, roots);
fs.writeFileSync(targetPath, repointed.content, 'utf-8');
escapes.push(...repointed.escapes);
}

return { escapes };
}
Loading
Loading