Skip to content

feat: restart the bot in place with /restart or restart.sh - #27

Merged
Rikul merged 2 commits into
mainfrom
claude/cool-gates-snl8mg
Oct 21, 2026
Merged

Rikul merged 2 commits into
mainfrom
claude/cool-gates-snl8mg

Conversation

@Rikul

@Rikul Rikul commented Oct 21, 2026

Copy link
Copy Markdown
Owner

restart.sh used to kill the running bot and start a new one. In Docker
(the bot is PID 1) or under a supervisor that kills the process group,
killing the bot also killed the script, so nothing came back up.

Now /restart (any channel) and SIGHUP (sent by restart.sh) request a
restart: the server waits for running turns and queued replies, shuts
the channels down cleanly, then re-executes the same command line in
the same process (after uv sync for new dependencies), so updated
code is loaded from disk and the PID never changes. restart.sh finds
the bot via the PID file or /proc (no ps in the Docker image) and only
starts a new one when none is running. The agent's system prompt now
says how to restart itself after pulling new code.

Co-Authored-By: Claude Opus 5.5 [email protected]
Claude-Session: https://claude.ai/code/session_01BbTezqpYhpjuHnDYNvRX33

restart.sh used to kill the running bot and start a new one. In Docker
(the bot is PID 1) or under a supervisor that kills the process group,
killing the bot also killed the script, so nothing came back up.

Now /restart (any channel) and SIGHUP (sent by restart.sh) request a
restart: the server waits for running turns and queued replies, shuts
the channels down cleanly, then re-executes the same command line in
the same process (after `uv sync` for new dependencies), so updated
code is loaded from disk and the PID never changes. restart.sh finds
the bot via the PID file or /proc (no ps in the Docker image) and only
starts a new one when none is running. The agent's system prompt now
says how to restart itself after pulling new code.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01BbTezqpYhpjuHnDYNvRX33
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-21T08:51:00.551591Z 5b050fd PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5b050fd978

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread restart.sh Resolved
Comment thread app/bg_server.py Resolved
Comment thread app/infra/restart.py Resolved

@Rikul Rikul left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restart-in-place works, and I verified that end-to-end rather than reading it. But the upgrade path — the one this PR exists for — fails on first use.

What I ran

  • Full suite at 5b050fd: 473 passed; CI green.
  • A live instance (web channel only, spare port, throwaway ANOTHERBOT_HOME): both SIGHUP and a WebSocket /restart re-execute in place — same PID 71999 before and after, uvicorn shut down cleanly, the port rebound with no EADDRINUSE, downtime ~1 s, and /restart's reply arrived before the socket closed with 1012 (so the CLAUDE.md wording there is accurate).
  • Probes for the three findings below; numbers are quoted in the comments.

Findings

  1. restart.sh — the "an old build exits on SIGHUP ... a new one is started" fallback is documented but not implemented: the old build is killed and nothing replaces it. This is literally the first upgrade from any pre-PR build, and sys_instructions.md now tells the agent to do exactly this after a git pull.
  2. bg_server.py — scheduled-task runs are not counted as turns, so a restart during one kills it, records it as success with empty output, consumes its slot, and never delivers the result.
  3. bg_server.py — SIGHUP is restored to SIG_DFL for the whole shutdown + uv sync window, so a second HUP terminates the bot. Reproduced at a 1.3 s offset.

Lower-severity: no drain gate behind the 300 s cap, a fixed 1 s sleep standing in for "reply delivered", uv sync flags that differ from the image build, and the /proc scan picking an arbitrary match / risking a second instance.

Checked and correct (so you know what wasn't skimmed):

  • children = all_tasks() - existing - {server} really does capture every channel coro — probe: len(coros)=5 len(children)=5.
  • The /restart reply is already on the outgoing queue when wait_requested() wakes (qsize()==1), so the first q.empty() poll cannot drop that message; _restart_cmd has no suspension point between request_restart() and outgoing_msg().
  • START_CWD + restoring it before execv is what keeps a relative PYTHONPATH=. from run.sh working; the reasoning in the comment holds.
  • config.PROJECT_HOME and restart.sh's ${ANOTHERBOT_HOME:-$HOME/.anotherbot} resolve to the same path, so the PID file round-trips; is_bot also rejects a stale PID that got reused.
  • Windows: signal.SIGHUP raises AttributeError there, and both add_signal_handler and remove_signal_handler sit inside handlers for it.
  • restart.turn() decrements correctly on cancellation.
  • The frontend reconnects on 1012 with backoff and resets agentBusy, so a restart doesn't wedge the UI.

Not verified: Docker itself — the image-specific notes are reasoned from the Dockerfile, not executed. (The token here is the PR author's, so GitHub won't accept APPROVE/REQUEST_CHANGES; this is a COMMENT review.)

Comment thread restart.sh Resolved
Comment thread app/bg_server.py Resolved
Comment thread app/bg_server.py Resolved
Comment thread app/infra/restart.py Resolved
Comment thread app/infra/restart.py Resolved
Comment thread app/infra/restart.py Resolved
Comment thread restart.sh Resolved
- restart.sh: detach, then confirm the bot actually restarted (PID file
  rewritten) or start a new one if it exited. A bot without its own PID
  file (pre-restart build) is stopped and started again instead of being
  sent a SIGHUP it can't handle. Only targets the Python process whose
  cwd is this data dir's workspace.
- Scheduled task runs count as turns, so a restart waits for them; a run
  cancelled anyway is recorded as "cancelled" and keeps its slot.
- While a restart is pending, no new turns or scheduled runs start.
- Wait for replies to be delivered (MessageQueue.outgoing_idle), not just
  dequeued, instead of a fixed 1 s sleep.
- Keep a SIGHUP handler installed through shutdown and ignore SIGHUP
  across execv, so a second SIGHUP can't kill the bot mid-restart.
- uv sync uses --frozen --no-dev --inexact to mirror the image build.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01BbTezqpYhpjuHnDYNvRX33
@Rikul
Rikul merged commit 709bb73 into main Oct 21, 2026
2 checks passed
@Rikul
Rikul deleted the claude/cool-gates-snl8mg branch October 21, 2026 20:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants