Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .agents/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,21 @@
# `.agents/` — engineering record of the python-sdk repository

## Skill storage/state acceptance — 2026-10-26

[Real storage/state evidence](evidence/verify/2026-10-26-skill-storage-state-acceptance.md)
records owned OSS RW/RO recreation, mounted MicroVM pause/resume, Snapshot isolation,
LangGraph conversation boundaries, repository/preview/third-party MCP execution and
retained initial failures. The [proposed acceptance note](notes/proposed/process/2026-10-26-skill-storage-state-acceptance.md)
defines the scoped proof. The [borrowed-connection ownership correction](notes/proposed/feature/2026-10-26-mcp-borrowed-sandbox-ownership.md)
separates manager-created instances from cached connections; it is newer than
public a6, whose historical receipts remain unchanged. Repository-wide server
check failures and delivery gates remain explicit.

[Desktop handover/replay acceptance](evidence/verify/2026-10-26-desktop-handover.md)
and its [proposed note](notes/proposed/process/2026-10-26-desktop-handover-acceptance.md)
record the separate pinned desktop adapter, private ingress, real screenshot/input,
simulated operator handover and replay controls with owned cleanup.

## Current public pair — 2026-10-25

Client/server **0.4.0a6** are public. [Public delivery verification](evidence/verify/2026-10-25-public-a6-delivery.md)
Expand Down
58 changes: 58 additions & 0 deletions .agents/evidence/verify/2026-10-26-desktop-handover.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Verification: real desktop, private stream and simulated operator handover

Result: PASS for the recorded GUI, private ingress, password-authenticated VNC, simulated operator handover and screenshot replay. Real human login, hosted takeover service and full-motion video recording remain NOT_RUN.

The optional fixture uses the official compatibility SDK `e2b-desktop==2.4.1` with `e2b==2.59.0`, Playwright 1.58.0, aiohttp 3.13.4 and the full exact dependency set in `e2e/desktop-requirements.txt`. It does not add desktop methods to the project client or MCP tool catalog. Python 3.11.15 and local Chromium 150.0.7871.114 were used. [Tested source hashes](2026-10-26-desktop-handover/tested-files.json) identify the exact permanent runner and fixtures.

## Real preparation and resource ownership

A unique test-owned generation-2 template was built through the actual official FC management SDK using the configured region's `fc-e2b-registry.<region>.cr.aliyuncs.com/runtime/desktop:v0.0.44`, CPU 4 and memory 8192 MB. No startup/readiness command was invented, no existing template was changed, and no local Docker daemon was needed for this image-registration path. Independent template metadata showed `Ready`, generation 2 and the requested resources. [Build observations](2026-10-26-desktop-handover/build-state.json) preserve the actual progress.

The permanent source was then executed with a prepared template ID and a new output directory:

```bash
python e2e/desktop_acceptance.py --template TEST_OWNED_TEMPLATE_ID --output-dir NEW_EVIDENCE_DIRECTORY
```

The script created its own 600-second sandbox with `secure=True` and `network={"allow_public_traffic": False}`. Independent sandbox metadata confirmed 4 CPU / 8192 MB. The caller-supplied template was left for its separate owner to delete.

## Input, return and observed effect

| Action | Actual return / independent observation | Result |
| --- | --- | --- |
| Open a synthetic local application in real desktop Chrome | Actual 1024×768 PNG; browser page and editable form visible | PASS |
| Agent mouse focus, keyboard input and commit | Downloaded application state recorded `agent-start` | PASS |
| Enable private public-facing ports and inspect traffic token availability | SDK issued a nonempty `traffic_access_token`; no token was logged | PASS |
| HTTP/WS ingress without traffic token | HTML HTTP 403 and WebSocket HTTP 403 | PASS |
| Inject the correct traffic token in the backend proxy | HTML HTTP 200, actual live WebSocket data and rendered desktop canvas | PASS |
| Use the wrong VNC password after reaching the VNC service | Explicit password-check failure and no connected client | PASS |
| Connect with the correct VNC password | Actual rendered frames; 93,910 transferred bytes across the two VNC client probes by stream shutdown | PASS |
| Transfer input to a separate noVNC controller | Actual keyboard/mouse actions changed the independently downloaded state to `operator-step` | PASS simulated operator |
| Return input to the original SDK agent on the same desktop | Independently downloaded exact sequence `agent-start → operator-step → agent-resumed` | PASS |
| Stop stream and delete the test-owned sandbox | Stream stopped, delete succeeded, separate typed not-found query | PASS |
| Reopen offline replay in a real local browser | Initial, last and previous controls selected the expected actual frame files | PASS |

[Final machine-readable report](2026-10-26-desktop-handover/acceptance.json), [actual downloaded application state](2026-10-26-desktop-handover/gui-state.json), [timestamped screenshot hashes](2026-10-26-desktop-handover/timeline.json), [offline player](2026-10-26-desktop-handover/replay.html) and [portable replay archive](2026-10-26-desktop-handover/desktop-handover-replay.zip) preserve input-to-effect evidence. The [player check](2026-10-26-desktop-handover/replay-check.json) was a separate browser run. Six timeline entries include the diagnostic and final operator screenshot sharing the same content hash; this is not six independently different screens or a video recording.

## Authentication findings and preserved failed attempts

The successful private-ingress run kept three credentials separate:

- SDK control operations use `envdAccessToken` in `X-Access-Token`.
- Private application-port traffic uses `Sandbox.traffic_access_token` in `e2b-traffic-access-token`.
- The VNC session separately checks the password created by `stream.start(require_auth=True)`.

These are the [documented network layers](https://help.aliyun.com/en/agent-sandbox/network-access-control). The [official Desktop image guide](https://help.aliyun.com/zh/functioncompute/desktop-template-example) provides image and SDK preparation. The [Computer Use guide](https://help.aliyun.com/zh/functioncompute/use-the-computer-use-sandbox) describes GUI/stream actions; its abbreviated gateway paragraph must be read together with the actual network mode and ingress credential contract.

[Initial attempt reports](2026-10-26-desktop-handover/initial-attempts/) retain failures rather than replacing them with passes: the Chrome testing banner moved the initial coordinates; the connection configuration's public header accessor had to be used correctly; public-default HTML and an RFB banner were insufficient authentication evidence; the FC gateway added `Content-Disposition: attachment`; the first connected event preceded actual framebuffer delivery; and fast VNC shortcut/text entry did not fully apply. The final fixture used inspected screenshot coordinates, an appropriate MIME-preserving controlled proxy, actual frame checks and paced physical key events before independent state verification.

A separate [private run using the wrong token category](2026-10-26-desktop-handover/private-wrong-token-type.json) correctly returned 403 even with an envd token. The final run used the documented traffic token and returned 200. The earlier [public-default successful handover](2026-10-26-desktop-handover/public-default-handover.json) remains its own observation, not proof of private ingress enforcement.

## Checks and cleanup

- Scoped `ruff check e2e/desktop_acceptance.py` and `ruff format --check`: PASS.
- Permanent optional runner against the actual prepared cloud template: exit 0, all its assertions PASS.
- Real local-browser offline-player control check: PASS.
- Full SDK product checks are tracked by the coordinating acceptance task; no full-repository or remote CI success is claimed here.

All 11 exact test-created desktop sandbox IDs across exploratory and final runs were independently queried and returned typed not found: [cleanup observations](2026-10-26-desktop-handover/sandbox-cleanup.json). The unique owned desktop template's name was checked before deletion; a separate management `GetTemplate` returned HTTP 404: [template cleanup](2026-10-26-desktop-handover/template-cleanup.json). No borrowed sandbox, existing template or user account state was changed. The screenshot workflow used synthetic content and a simulated second controller; no actual user login was attempted.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
178 changes: 178 additions & 0 deletions .agents/evidence/verify/2026-10-26-desktop-handover/acceptance.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,178 @@
{
"steps": [
{
"name": "desktop-created-correct-spec",
"result": "PASS",
"cpu": 4,
"memory": 8192
},
{
"name": "agent-mouse-keyboard-effect",
"result": "PASS",
"sequence": [
"agent-start"
]
},
{
"name": "traffic-token-issued",
"result": "PASS"
},
{
"name": "stream-static-resource-access",
"result": "PASS",
"unauthenticated_static_status": 403,
"authenticated_static_status": 200,
"upstream_content_type": "text/html",
"upstream_disposition": "attachment",
"html_document_observed": true,
"connected_without_gateway_token": false,
"http_status": 403
},
{
"name": "gateway-token-enforcement",
"result": "PASS",
"connected_without_gateway_token": false,
"http_status": 403
},
{
"name": "vnc-password-rejected",
"result": "PASS",
"connected": false,
"status": "New connection has been rejected with reason: password check failed!"
},
{
"name": "live-stream-renders",
"result": "PASS",
"websocket_sessions": 2,
"transported_bytes": 33380,
"canvas_width": 1100
},
{
"name": "operator-handover-effect",
"result": "PASS",
"actor": "simulated operator through independent VNC client",
"sequence": [
"agent-start",
"operator-step"
]
},
{
"name": "agent-resume-same-state",
"result": "PASS",
"sequence": [
"agent-start",
"operator-step",
"agent-resumed"
]
},
{
"name": "stream-stopped",
"result": "PASS",
"websocket_sessions": 2,
"transported_bytes": 93910
},
{
"name": "replay-artifacts",
"result": "PASS",
"screenshots": 6,
"format": "offline screenshot timeline with SHA-256; not full-motion video"
},
{
"name": "desktop-cleanup",
"result": "PASS",
"deleted": true,
"independently_absent": true
}
],
"timeline": [
{
"at_seconds": 17.86,
"actor": "agent",
"action": "Open the synthetic local acceptance page in the real desktop browser",
"file": "01-browser-ready.png",
"sha256": "869764cbe433fc11986032712610e68d6de132b0a8b694cba3c0ae1f54a24c0a",
"width": 1024,
"height": 768
},
{
"at_seconds": 20.13,
"actor": "agent",
"action": "Mouse focus, keyboard text and mouse commit",
"file": "02-agent-state.png",
"sha256": "2e599f95b3c886b45255af27c3994826e3068dcfd14d741570635e8a138079b4",
"width": 1024,
"height": 768
},
{
"at_seconds": 35.15,
"actor": "simulated-operator",
"action": "Observe authenticated live stream through controlled backend proxy",
"file": "03-live-stream-view.png",
"sha256": "3ab4de0a27d070385ef42cecf196668bff4963b7f3f8766507e14be0e9c8158e"
},
{
"at_seconds": 42.91,
"actor": "simulated-operator",
"action": "Independent screenshot after VNC input before checking state",
"file": "operator-diagnostic.png",
"sha256": "c68c23a466c467052d70720d6c5da2aa916af4cf68ba11bb2c6b33f94547787d",
"width": 1024,
"height": 768
},
{
"at_seconds": 43.48,
"actor": "simulated-operator",
"action": "Type and commit through the second controller",
"file": "04-operator-state.png",
"sha256": "c68c23a466c467052d70720d6c5da2aa916af4cf68ba11bb2c6b33f94547787d",
"width": 1024,
"height": 768
},
{
"at_seconds": 45.94,
"actor": "agent",
"action": "Resume on the same desktop and preserve prior operator state",
"file": "05-agent-resumed.png",
"sha256": "c9ef3fc538e3576c83f0dbebf8b80d4c9820654e0e752c952bb3f44b1e608409",
"width": 1024,
"height": 768
}
],
"scope": "Real official desktop image; operator is simulated by a second controller, not a human/login session.",
"template_id": "iallftl5zi2vdi0u0ht0",
"creation_controls": {
"secure": true,
"network": {
"allow_public_traffic": false
}
},
"sandbox_id": "sbx-ff3cae53-31f3-4e13-bdc2-cdd3a48ac298",
"gateway_observation": {
"unauthenticated_static_status": 403,
"authenticated_static_status": 200,
"upstream_content_type": "text/html",
"upstream_disposition": "attachment",
"html_document_observed": true,
"connected_without_gateway_token": false,
"http_status": 403
},
"proxy": {
"bind": "127.0.0.1",
"gateway_token_stays_backend": true
},
"canvas_observation": {
"bounding_box": {
"x": 0,
"y": 12.5,
"width": 1100,
"height": 825
},
"dimensions": {
"width": 1024,
"height": 768
},
"view_only_setting": false
},
"result": "PASS",
"result_scope": "GUI, password-authenticated VNC, simulated handover and screenshot replay only; gateway token enforcement is separately observed"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
{
"name": "skills-desktop-00c77839d5fd",
"image": "fc-e2b-registry.cn-hangzhou.cr.aliyuncs.com/runtime/desktop:v0.0.44",
"cpu": 4,
"memory": 8192,
"checks": [
{
"stage": "template-submitted",
"template_id": "iallftl5zi2vdi0u0ht0"
},
{
"stage": "build-progress",
"state": "building",
"elapsed_seconds": 0
},
{
"stage": "build-progress",
"state": "building",
"elapsed_seconds": 16
},
{
"stage": "build-progress",
"state": "building",
"elapsed_seconds": 31
},
{
"stage": "template-ready",
"state": "ready",
"cpu": 4,
"memory": 8192,
"generation": 2
}
],
"owned": true,
"template_id": "iallftl5zi2vdi0u0ht0"
}
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"events": [{"value": "agent-start", "at": 1792987242.7279108}, {"value": "operator-step", "at": 1792987264.8013532}, {"value": "agent-resumed", "at": 1792987268.4544063}], "latest": "agent-resumed"}
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
{
"steps": [
{
"name": "desktop-created-correct-spec",
"result": "PASS",
"cpu": 4,
"memory": 8192
},
{
"name": "desktop-cleanup",
"result": "PASS",
"deleted": true,
"independently_absent": true
}
],
"timeline": [
{
"at_seconds": 17.48,
"actor": "agent",
"action": "Open the synthetic local acceptance page in the real desktop browser",
"file": "01-browser-ready.png",
"sha256": "869764cbe433fc11986032712610e68d6de132b0a8b694cba3c0ae1f54a24c0a",
"width": 1024,
"height": 768
}
],
"scope": "Real official desktop image; operator is simulated by a second controller, not a human/login session.",
"sandbox_id": "sbx-91e9ff49-6cdd-46ef-ab43-cd20bcfb287f",
"failure": {
"type": "AssertionError",
"message": "GUI commit did not produce agent-start"
},
"result": "FAIL"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
{
"steps": [
{
"name": "desktop-created-correct-spec",
"result": "PASS",
"cpu": 4,
"memory": 8192
},
{
"name": "agent-mouse-keyboard-effect",
"result": "PASS",
"sequence": [
"agent-start"
]
},
{
"name": "desktop-cleanup",
"result": "PASS",
"deleted": true,
"independently_absent": true
}
],
"timeline": [
{
"at_seconds": 17.35,
"actor": "agent",
"action": "Open the synthetic local acceptance page in the real desktop browser",
"file": "01-browser-ready.png",
"sha256": "869764cbe433fc11986032712610e68d6de132b0a8b694cba3c0ae1f54a24c0a",
"width": 1024,
"height": 768
},
{
"at_seconds": 19.57,
"actor": "agent",
"action": "Mouse focus, keyboard text and mouse commit",
"file": "02-agent-state.png",
"sha256": "2e599f95b3c886b45255af27c3994826e3068dcfd14d741570635e8a138079b4",
"width": 1024,
"height": 768
}
],
"scope": "Real official desktop image; operator is simulated by a second controller, not a human/login session.",
"sandbox_id": "sbx-0f83659e-9b64-4120-8a81-4b2e906eaa20",
"failure": {
"type": "AttributeError",
"message": "'ConnectionConfig' object has no attribute 'extra_sandbox_headers'"
},
"result": "FAIL"
}
Loading
Loading