From c4d44bbb01920a45436b2c338c517cf517d97980 Mon Sep 17 00:00:00 2001 From: reubenmiller Date: Tue, 10 Nov 2026 14:31:04 +1000 Subject: [PATCH 01/14] docs(openspec): propose serial onboarding A host tool, tedge-onboard, that onboards a device on USB with the computer's Wi-Fi and the user's go-c8y-cli session. On the device, an optional provisioning library beside the client speaks Improv Wi-Fi, plus two tiny-edge commands, over serial first and BLE later. Co-Authored-By: Claude Opus 5.5 --- .../changes/serial-onboarding/.openspec.yaml | 2 + openspec/changes/serial-onboarding/design.md | 247 ++++++++++++++++++ .../changes/serial-onboarding/proposal.md | 128 +++++++++ .../specs/tedge-onboard-tool/spec.md | 186 +++++++++++++ .../specs/tedge-provisioning/spec.md | 168 ++++++++++++ openspec/changes/serial-onboarding/tasks.md | 53 ++++ 6 files changed, 784 insertions(+) create mode 100644 openspec/changes/serial-onboarding/.openspec.yaml create mode 100644 openspec/changes/serial-onboarding/design.md create mode 100644 openspec/changes/serial-onboarding/proposal.md create mode 100644 openspec/changes/serial-onboarding/specs/tedge-onboard-tool/spec.md create mode 100644 openspec/changes/serial-onboarding/specs/tedge-provisioning/spec.md create mode 100644 openspec/changes/serial-onboarding/tasks.md diff --git a/openspec/changes/serial-onboarding/.openspec.yaml b/openspec/changes/serial-onboarding/.openspec.yaml new file mode 100644 index 0000000..2b638f8 --- /dev/null +++ b/openspec/changes/serial-onboarding/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-11-10 diff --git a/openspec/changes/serial-onboarding/design.md b/openspec/changes/serial-onboarding/design.md new file mode 100644 index 0000000..4cff0e1 --- /dev/null +++ b/openspec/changes/serial-onboarding/design.md @@ -0,0 +1,247 @@ +## Context + +Onboarding a tiny-edge device needs three things on the device: Wi-Fi +credentials, the tenant host, and a one-time password that the tenant knows +for the device's external ID. Today they come from the build (Kconfig), the +application, or the Lab Provisioner, which hands them over in an encrypted +bundle after the device joins a provisioning network. + +The client deliberately knows nothing about how they arrive +(`tedge-c8y-onboarding`: "The client SHALL remain unaware of how the +application obtained the password"). It offers `tedge_set_c8y_url()` and +`tedge_set_enroll_otp()`, both called before `tedge_start()`. +Wi-Fi is the application's job on every port. + +microcontroller-devices already ships an Improv Wi-Fi BLE provisioner, as a +separate MCUboot image (`apps/wifi-provisioner/src/improv.c`). It answers +the Improv RPC "send Wi-Fi settings" and "identify" over a GATT service, and +it does not hand anything to the client. + +The developer at a desk has the board on USB, the computer on the Wi-Fi the +board should join, and go-c8y-cli logged in to the tenant. + +## Goals / Non-Goals + +**Goals:** + +- Onboard one board from a computer with one command and no browser, in + about a minute. +- Reuse the computer's Wi-Fi network and go-c8y-cli session, with the + user's consent for the Wi-Fi password. +- One device-side RPC layer for every transport (serial now, BLE next) and + every port. +- Keep the client (`core/`) unchanged and unaware of provisioning. +- Stay compatible with standard Improv clients for Wi-Fi, so ESP Web Tools + and Improv phone apps can at least provision Wi-Fi. + +**Non-Goals:** + +- Fleet provisioning, trust bootstrapping without a person present, and + anything the Lab Provisioner does. +- Provisioning over Wi-Fi (SoftAP, a web page on the device). +- Configuring the device beyond Wi-Fi and Cumulocity. + +## Decisions + +### 1. Improv Wi-Fi as the protocol, with two tiny-edge commands + +Improv is an open protocol with a serial and a BLE transport that share +their RPC commands. It is already used by this project (the BLE +provisioner), by ESP Web Tools, and by phone apps. Its RPC packet is +`[command][length][data][checksum]`, the checksum being the low byte of +the sum of the preceding bytes. Strings are length-prefixed. + +The library answers: + +| Command | Name | Transport | Notes | +|---|---|---|---| +| 0x01 | Send Wi-Fi settings | both | Improv standard | +| 0x02 | Request current state | serial | Improv standard (0x02 is "identify" on BLE) | +| 0x02 | Identify | BLE | Improv standard; the application's hook, if any | +| 0x03 | Request device info | serial | Improv standard: firmware name and version, chip, device name | +| 0xE0 | tiny-edge info | both | External ID, auth mode, tenant if set, client state | +| 0xE1 | tiny-edge onboard | both | Tenant host and one-time password | +| 0xE2 | tiny-edge identify | serial | Blink or log, to tell boards apart (BLE uses 0x02) | + +The tiny-edge commands are numbered high, away from the commands Improv +has assigned (0x01 to 0x05 so far). A standard Improv client never sends +them; a device without the library answers "unknown RPC" (0x02 error), which +the tool reports as "this firmware does not support tiny-edge onboarding". + +*Alternatives*: a line protocol on the shell (`tedge onboard `) +is simpler to type but has no framing, no checksum, and would need parsing +of free-form output; it also does nothing for BLE. A protocol of our own +(CBOR, protobuf) costs a parser per side for no gain over Improv's +length-prefixed strings. + +### 2. The library sits beside the client, not in it + +`provisioning/` is a library at the repository root with its own public +header (`include/tedge/provisioning.h`). It is portable C under the same +rules as `core/` (checked by `check-portability.py`): it uses the public +`tedge_*` API and the port interface, never an RTOS API. It owns the RPC +layer and the Improv state machine, and calls: + +- the application's Wi-Fi hook (`connect(ssid, password)`), because joining + and storing Wi-Fi is the application's job; +- `tedge_set_c8y_url()` and `tedge_set_enroll_otp()` for the onboarding + command. + +Both setters must be called before `tedge_start()`. When the client is +already running (it usually is, waiting for registration), the library +stops it with `tedge_stop()`, applies the values, and starts it again with +`tedge_start()`. The public API is unchanged; a task verifies that a +stop-start cycle is clean on every port. + +*Alternatives*: putting Improv in `core/` would break the +`tedge-c8y-onboarding` requirement and make every build carry it. Leaving it +in microcontroller-devices would leave every other tiny-edge user to write +it again. + +### 3. Transports are thin and port-supplied + +The RPC layer takes complete RPC packets and returns result packets. Each +transport wraps them: + +- **Serial**: Improv Serial framing (`IMPROV`, version 1, type, length, + payload, checksum). The framing code is portable and part of the library; + the port supplies "bytes arrived" and "write these bytes as one unit". +- **BLE** (phase 2): the Improv GATT service. The GATT table and + advertising are RTOS code, so they go in the port. + +On Zephyr, the console is usually the shell's UART. The glue uses the +shell's bypass mode: a shell command, `tedge improv`, hands the console to +the library until onboarding finishes or nothing valid arrives for 60 +seconds, then gives it back. The tool sends that command first; without a +shell the line is plain bytes that the frame parser skips. A build without +the shell feeds the library from the console UART directly, which also makes +the device answer ESP Web Tools with no command first. + +*Alternatives*: scanning the shell's input for `IMPROV` would need a hook +the Zephyr shell does not have. A second UART would need a second cable. + +### 4. The tool is Go and drives go-c8y-cli + +`tools/onboard/` is a Go module producing one static binary per platform. +Go matches go-c8y-cli, has serial (`go.bug.st/serial`) and BLE +(`tinygo.org/x/bluetooth`) libraries for macOS, Linux and Windows, and +needs no runtime on the user's computer. + +The tool runs `c8y` as a subprocess rather than calling the REST API +itself, so it uses whatever session the user has active (including +encrypted sessions and SSO tokens), and needs no credentials of its own. +The calls are: + +1. `c8y currenttenant get` to learn the tenant host and check the session. +2. `c8y identity get --name ` to find an existing device. +3. `c8y deviceregistration register-ca --id + --one-time-password --force`. +4. `c8y identity get` again until the device exists, then + `c8y inventory get` until it reports availability. + +The one-time password is 24 characters from `[A-Za-z0-9]`, from the OS +random source. It is passed to `c8y` as an argument and to the device over +the serial line, and is never printed. + +*Alternatives*: Python with pyserial is quicker to write but needs a Python +environment; a go-c8y-cli extension in shell cannot open a serial port +portably. The tool can be packaged as a go-c8y-cli extension later, since +extensions may be binaries. + +### 5. Wi-Fi credentials come from the OS, with consent + +| OS | Network | Password | +|---|---|---| +| macOS | `ipconfig getsummary ` | `security find-generic-password -D "AirPort network password" -a -w` (the Keychain asks the user) | +| Linux | `nmcli -t -f active,ssid dev wifi` | `nmcli -s -g 802-11-wireless-security.psk connection show ` | +| Windows | `netsh wlan show interfaces` | `netsh wlan show profile name= key=clear` | + +The tool proposes the current network and lets the user pick another the +computer knows. Where the OS will not give the password, it asks for it +with hidden input. The password is never printed or logged. For scripts, +`--wifi-ssid`, `--wifi-password-stdin` and `--skip-wifi` replace the +questions. + +### 6. The tool finds the device by asking each port + +It lists serial ports with their USB vendor IDs, keeps those of known +USB-serial chips (Espressif USB-Serial-JTAG `303A`, WCH `1A86`, Silicon Labs +`10C4`, FTDI `0403`), opens each with DTR and RTS low so no board resets, +and asks it for its tiny-edge info. A port that answers is a candidate, +described by what it reports (model, firmware, whether it is in Cumulocity) +rather than by its device path. One candidate is used directly; several are +offered as a list, with "make this board blink" to tell them apart; none +means waiting for the user to plug one in. Asking is what makes the choice +safe: a port that does not answer is never sent credentials. + +### 7. Guided by default, scriptable on request + +The tool is written for someone who has never used a terminal. With no +arguments it runs as a short guided conversation: each step shows a +spinner and then a one-line result, and the only input is choosing from a +list (with the likely answer selected) or confirming once. It never asks +the user to type a port, an ID, a URL or a password it can find. Messages +avoid protocol terms ("board ID" for external ID, "set up" for onboard), +and every failure says what to do next and offers to retry instead of +exiting. + +The prompts use `charmbracelet/huh`, which draws lists and hidden password +inputs on macOS, Linux and Windows terminals and has an accessible mode for +screen readers. Logins to Cumulocity stay go-c8y-cli's: the tool runs +`c8y sessions login` or the session picker in the same terminal and picks +up from there. + +Started by double-click, the tool runs in a terminal window that closes +when it exits, so it waits for Enter before exiting. + +For scripts and CI, `--yes` turns every question into a failure with a +message, and `--json` prints the result. + +*Alternatives*: a graphical app would be friendlier still, but a +cross-platform GUI with serial and Keychain access is a much larger +project; the guided terminal flow covers the need first, and its steps can +sit behind a GUI later. A browser flow (Web Serial) cannot read the +computer's Wi-Fi password or use go-c8y-cli. + +## Risks / Trade-offs + +- **Log lines can interleave with a frame** on a shared console → the + device writes each frame as one unit through the transport's write call + (on Zephyr, under the shell's output lock), and the tool skips any bytes + between frames and rejects frames with a bad checksum. +- **macOS hides the SSID** from processes without Location Services on + recent releases → the tool offers the networks the computer knows + (`networksetup -listpreferredwirelessnetworks`) as a list to choose from. +- **Probing ports sends bytes to boards that are not ours** → the probe is + one short line and one Improv frame, both ignored by firmware that does not + speak Improv, and nothing else is sent to a port that does not answer. +- **Anyone with the USB cable can re-onboard the device** → physical access + is the trust boundary, as it is for flashing. The onboard command on a + device that is already connected is refused unless the tool sends its + re-onboard flag, which it does only after the user agrees (or with + `--reonboard` in a script). +- **The one-time password re-onboards the device** (`tedge_set_enroll_otp()` + discards the certificate) → the tool generates a fresh password each time + and registers it before sending it, so the device never holds a password + the tenant does not know. +- **Improv may assign 0xE0 or 0xE1 one day** → the commands are in one + table on each side and can move; the tool checks the device's tiny-edge + protocol version in the info reply. +- **BLE costs RAM the C6 and WROOM may not have** → it is a separate option + in a later phase, and measured per board before it is offered there. +- **A stop-start cycle of the client may not be clean on every port** → a + task tests it on Zephyr and POSIX before the library relies on it. + +## Migration Plan + +New and optional: nothing to migrate. microcontroller-devices can move its +Improv BLE provisioner onto the library in a later change. + +## Open Questions + +- Should the Improv "provisioned" result carry a URL? Improv lets the + device return one after Wi-Fi is set. The device does not know its + Cumulocity managed object ID, so the tool prints the device link instead; + the device could return the tenant's Device Management URL. +- Release packaging for the tool: GitHub release assets built by a matrix + job, or GoReleaser. Decided when the release workflow is touched. diff --git a/openspec/changes/serial-onboarding/proposal.md b/openspec/changes/serial-onboarding/proposal.md new file mode 100644 index 0000000..85b895b --- /dev/null +++ b/openspec/changes/serial-onboarding/proposal.md @@ -0,0 +1,128 @@ +## Why + +Getting a new tiny-edge device onto Wi-Fi and into Cumulocity takes too many +steps today. Someone has to put the Wi-Fi credentials into the build or a +provisioner image, then either open the registration URL the device prints +or register the device beforehand and pass its one-time password in. The +Lab Provisioner (lab-ztp-provisioner) automates this for a fleet, but it is +a server to run and its own development effort, which is too much for one +board on a desk. + +A developer with the board on a USB cable already has two things the device +needs: the computer is on the Wi-Fi network the device should join, and +go-c8y-cli (`c8y`) is logged in to the tenant the device should report to. +A small tool can hand both to the device in about a minute, with no browser +and nothing typed into a build. + +## What Changes + +- **A host tool, `tedge-onboard`**, in `tools/onboard/` (Go, one binary per + platform), written for users who are not technical. Run with no + arguments, it guides the user through a few plain-language steps: it + finds the board by asking each USB port (offering a list when there are + several), proposes the computer's current Wi-Fi network and takes its + password from the OS credential store with the user's approval, helps the + user pick or log in to a go-c8y-cli session, registers the device with + `c8y deviceregistration register-ca` and a one-time password it + generates, sends the Wi-Fi and the tenant and password to the device, and + waits until the device shows as connected in Cumulocity. It never asks + for a port, an ID or a URL; where it cannot decide, it offers choices. + Flags remain for scripts. +- **A device-side provisioning library**, `provisioning/` at the repository + root, beside the client and not in it. It speaks the Improv Wi-Fi RPC + protocol, plus one tiny-edge command for the tenant and one-time password, + and passes what it receives to the application (Wi-Fi) and to the client + through the existing public API (`tedge_set_c8y_url()`, + `tedge_set_enroll_otp()`). +- **Transports**: Improv Serial first, over the board's USB console. Improv + BLE (a GATT service) later, on the same RPC layer, so the tool and a phone + can onboard a device without a cable. The Improv RPC commands are the same + on both transports. +- **Port glue** for the serial transport: the Zephyr port feeds the library + from its console. Other ports add theirs when they add the library. +- **A sample** that uses the library: Zephyr's `samples/minimal` gains an + onboarding overlay. + +The library is optional and off by default. Nothing changes for an +application that does not enable it. + +### Build options added + +| Option | Default | What it adds | +|---|---|---| +| `TEDGE_PROVISIONING` | off | The Improv RPC layer and the tiny-edge command | +| `TEDGE_PROVISIONING_SERIAL` | on with `TEDGE_PROVISIONING` | Improv Serial over the console | +| `TEDGE_PROVISIONING_BLE` | off (phase 2) | Improv over a BLE GATT service | + +The flash and RAM cost per board is measured in the tasks (ESP32-S3-DevKitC, +ESP32-C6, ESP32-WROOM-32) and recorded here before the change is archived. +The serial transport is expected to be small (a frame parser and buffers of +a few hundred bytes). BLE costs what the Bluetooth stack costs, which is why +it is a separate option and a separate phase. + +### Interface changes + +- **Public `tedge_*` API**: none. The library is a client of the existing + API. It has a public header of its own, `include/tedge/provisioning.h`, + through which the application supplies the Wi-Fi hook and reads the + progress. +- **Port interface**: none. The serial transport is fed by the port's glue + code, outside the port interface. +- **Messages the device sends to Cumulocity**: none. +- **Messages on the console**: the device answers Improv frames on the + console when the library is enabled. Improv frames start with `IMPROV` and + a version byte, and the tool ignores any other output, so log lines and + Improv frames can share the console. + +### Where the change is + +The library is portable C outside `core/`, under the same rules as `core/` +(no RTOS API). The serial glue is in `ports/zephyr/`. The tool is in +`tools/onboard/`. `core/` does not change. + +## Non-goals + +- Fleet provisioning. The Lab Provisioner stays the path for many devices + at once, with its own trust model. +- Replacing the registration URL. A device without the library still prints + it, and an operator can still open it. +- Provisioning over the network (SoftAP, a web page on the device). +- Bootstrap authentication. The tool onboards devices that use the + Cumulocity CA. A bootstrap device needs only Wi-Fi, which the tool can + still send. +- Device-specific settings beyond Wi-Fi and Cumulocity (parameters, remote + access policy). Those are set from Cumulocity once the device is connected. +- Changing the existing Improv BLE provisioner image in + microcontroller-devices. It can move to the library later. + +## Capabilities + +### New Capabilities + +- `tedge-provisioning`: the device-side provisioning library: the Improv RPC + commands it answers, the tiny-edge extension command, the transports, how + it hands Wi-Fi to the application and onboarding data to the client, and + what it refuses. +- `tedge-onboard-tool`: the host tool: how it finds the device, where it + takes the Wi-Fi credentials from and with what consent, how it uses `c8y` + to register the device, what it waits for, and how it reports failure. + +### Modified Capabilities + +None. `tedge-c8y-onboarding` already requires that the client stays unaware +of how the application obtained the one-time password and depends on no +provisioning transport. This change keeps that requirement: the library +sits outside the client and uses the public API. + +## Impact + +- New directories: `provisioning/` (library), `tools/onboard/` (Go module), + `include/tedge/provisioning.h`. +- `ports/zephyr/`: Kconfig options, CMake for the library, console glue, + and a sample overlay. +- `scripts/check-portability.py`: applies the core rules to `provisioning/`. +- CI: builds and tests the tool (Go), and the library's frame parser on the + host. +- Dependencies: the tool needs go-c8y-cli on the PATH, logged in to the + tenant. It uses a Go serial library, and in phase 2 a Go BLE library. +- Releases: the tool's binaries are attached to each tiny-edge release. diff --git a/openspec/changes/serial-onboarding/specs/tedge-onboard-tool/spec.md b/openspec/changes/serial-onboarding/specs/tedge-onboard-tool/spec.md new file mode 100644 index 0000000..9f83ae3 --- /dev/null +++ b/openspec/changes/serial-onboarding/specs/tedge-onboard-tool/spec.md @@ -0,0 +1,186 @@ +## ADDED Requirements + +### Requirement: The tool guides a non-technical user from start to finish + +`tedge-onboard` SHALL need no arguments. Run on its own, it SHALL find the +device, the Wi-Fi network and the Cumulocity tenant by itself, and SHALL ask +the user only where it cannot decide, by offering a list of choices with the +likely one selected. It SHALL NOT ask the user to type a port, an ID or a +URL. Every message SHALL be in plain language, without terms such as +"serial port", "external ID", "one-time password" or "RPC", and every +problem SHALL say what the user can do next. Each step SHALL show progress +while it runs and a clear result when it ends. + +#### Scenario: First use with one board + +- **WHEN** a user with one board on USB runs `tedge-onboard` with no + arguments +- **THEN** the tool finds the board, proposes the computer's Wi-Fi network + and the user's tenant, asks the user to confirm once, onboards the device, + and ends with the device's name and a link to open it in Cumulocity + +#### Scenario: A problem the user can fix + +- **WHEN** a step fails for a reason the user can fix (the board is not + plugged in, Cumulocity is not logged in, the Wi-Fi password is wrong) +- **THEN** the tool says what went wrong and what to do in one or two plain + sentences, and offers to try again rather than exiting + +#### Scenario: Started by double-clicking + +- **WHEN** the user starts the tool by double-clicking it, so it runs in a + terminal window that closes when it exits +- **THEN** the tool waits for the user to press Enter before exiting, so the + result stays readable + +### Requirement: The board is found without the user naming it + +The tool SHALL look at every serial port whose USB vendor ID belongs to a +supported USB-serial chip (Espressif `303A`, WCH `1A86`, Silicon Labs +`10C4`, FTDI `0403`), open each with DTR and RTS low so no board resets, +and ask each for its tiny-edge info. It SHALL use the one board that +answers. When several answer, it SHALL list them by what they are (for +example "ESP32-S3 running tedge-minimal 0.3.0, not yet in Cumulocity") and +let the user choose, with an option to make a board identify itself (blink +or log) when the firmware supports it. When none is found, it SHALL tell +the user to plug the board in, and keep looking until a board appears or +the user cancels. `--port` SHALL remain available for scripts. + +#### Scenario: Two boards connected + +- **WHEN** two boards with tiny-edge firmware are on USB +- **THEN** the tool lists both by model, firmware and Cumulocity status, and + onboards the one the user picks + +#### Scenario: Board plugged in after the start + +- **WHEN** no board is connected when the tool starts +- **THEN** the tool asks the user to plug one in and continues by itself + once it appears + +#### Scenario: Board with other firmware + +- **WHEN** a board answers no tiny-edge info request within 5 seconds, or + answers that the command is unknown +- **THEN** the tool says the board's firmware cannot be set up this way and + points to the firmware download, and sends it nothing + +#### Scenario: Newer firmware than the tool + +- **WHEN** the board's provisioning protocol version is newer than the + tool's +- **THEN** the tool asks the user to download the latest tool, and sends + nothing + +### Requirement: Wi-Fi is taken from the computer, with consent + +The tool SHALL propose the Wi-Fi network the computer is connected to, and +SHALL let the user choose another from the networks the computer knows or +type one. It SHALL take the password from the operating system's credential +store (the macOS Keychain, NetworkManager, or the Windows WLAN profiles), +which may ask the user to allow it. When the password is not available, it +SHALL ask for it with hidden input. It SHALL NOT print or log the password. +When the board reports that it could not join the network, the tool SHALL +say so and offer to choose the network or enter the password again. A user +who does not want to change the board's Wi-Fi SHALL be able to skip the +step. `--wifi-ssid`, `--wifi-password-stdin` and `--skip-wifi` SHALL remain +available for scripts. + +#### Scenario: macOS with the Keychain + +- **WHEN** the tool runs on macOS connected to Wi-Fi +- **THEN** it proposes that network, macOS asks the user to allow access to + its password, and the tool sends it only after the user confirms + +#### Scenario: The OS hides the current network + +- **WHEN** the operating system does not reveal the current network's name +- **THEN** the tool lists the networks the computer knows for the user to + choose from + +#### Scenario: Wrong password + +- **WHEN** the board reports that it could not connect to the network +- **THEN** the tool says the network or password was not accepted and asks + again, without leaving the device half set up + +### Requirement: Cumulocity is reached through go-c8y-cli, set up with the user + +The tool SHALL use the `c8y` command (go-c8y-cli) with the user's session +for every call to Cumulocity, and SHALL hold no Cumulocity credentials +itself. When `c8y` is missing, it SHALL explain how to install it for the +user's operating system. When there is no active session, or more than one +to choose from, it SHALL let the user pick a session by tenant name or start +go-c8y-cli's own login, in the same terminal, and continue afterwards. It +SHALL show the tenant it will use and include it in the one confirmation. + +#### Scenario: Not logged in + +- **WHEN** `c8y` has no working session +- **THEN** the tool offers the user's saved sessions or a new login, runs + go-c8y-cli's interactive login, and continues with that tenant + +#### Scenario: go-c8y-cli missing + +- **WHEN** `c8y` is not on the PATH +- **THEN** the tool explains how to install go-c8y-cli on this operating + system, and offers to check again + +### Requirement: The device gets a name and a fresh registration + +The tool SHALL propose a device name for Cumulocity, defaulting to the +board's own ID, which the user may change. It SHALL generate a one-time +password of 24 characters from `[A-Za-z0-9]` with the operating system's +random source for every run, register it with +`c8y deviceregistration register-ca` for the board's ID and the chosen +name, and only then send it to the board. It SHALL NOT print the password. +When the board's ID already belongs to a device in the tenant, or the board +reports that it is connected, the tool SHALL say so by name and ask whether +to set the board up again (which gives it a new certificate), defaulting to +no. `--reonboard` and `--yes` SHALL answer these questions for scripts. + +#### Scenario: Device already in Cumulocity + +- **WHEN** the board's ID already belongs to a device in the tenant +- **THEN** the tool names that device and asks whether to set the board up + again, and changes nothing unless the user agrees + +#### Scenario: Setting a board up again + +- **WHEN** the user agrees to set up a board that is already registered +- **THEN** the tool registers a new password, sends it with the re-onboard + flag, and the device enrolls again with a new certificate + +### Requirement: The tool confirms the device is connected + +After sending the tenant and password, the tool SHALL wait until the board +reports that it is connected and Cumulocity shows the device as available, +for up to 3 minutes. It SHALL then show the device name and its Cumulocity +link, and offer to set up another board. It SHALL exit with status 0 only +when every device it set up is connected. + +#### Scenario: Device connects + +- **WHEN** the device enrolls and connects +- **THEN** the tool shows the device's name and link and asks whether to set + up another board + +#### Scenario: Device does not connect in time + +- **WHEN** the device has not connected 3 minutes after the tenant and + password were sent +- **THEN** the tool shows the last state the board reported, explains the + likely causes in plain words, and offers to try again + +### Requirement: Scripts can run the tool without questions + +With `--yes`, the tool SHALL ask no questions: it SHALL use the detected or +given values, fail with a non-zero status and a message where it would have +asked, and print one line per step. `--json` SHALL print the result (device +name, ID, link, status) as JSON. + +#### Scenario: Unattended run + +- **WHEN** a script runs `tedge-onboard --yes --port --skip-wifi` +- **THEN** the tool onboards the board without prompting, and exits with a + non-zero status instead of prompting when something is missing diff --git a/openspec/changes/serial-onboarding/specs/tedge-provisioning/spec.md b/openspec/changes/serial-onboarding/specs/tedge-provisioning/spec.md new file mode 100644 index 0000000..e4dd219 --- /dev/null +++ b/openspec/changes/serial-onboarding/specs/tedge-provisioning/spec.md @@ -0,0 +1,168 @@ +## ADDED Requirements + +### Requirement: Provisioning is an optional library outside the client + +tiny-edge SHALL provide device-side provisioning as a library separate from +the client, enabled with the build option `TEDGE_PROVISIONING` and off by +default. The library SHALL use only the public `tedge_*` API and the port +interface, and SHALL NOT use an RTOS API directly. The client SHALL NOT +depend on the library. A build without the option SHALL contain none of the +library's code. + +#### Scenario: Build without the option + +- **WHEN** an application is built without `TEDGE_PROVISIONING` +- **THEN** the image contains no provisioning code, and the client behaves as + before + +#### Scenario: Portability check + +- **WHEN** `scripts/check-portability.py` runs +- **THEN** it applies the rules for `core/` to `provisioning/` as well + +### Requirement: The library answers Improv Wi-Fi RPC commands + +The library SHALL answer the Improv Wi-Fi RPC commands "send Wi-Fi +settings" (0x01) on every transport, "request current state" (0x02) and +"request device info" (0x03) on the serial transport, and "identify" (0x02) +on the BLE transport. It SHALL validate each RPC packet's length and +checksum, answer an invalid packet with the "invalid RPC" error (0x01), and +an unknown command with the "unknown RPC" error (0x02). The device info +SHALL report the firmware name and version, the chip, and the external ID as +the device name. + +#### Scenario: Standard Improv client sends Wi-Fi settings + +- **WHEN** an Improv client sends "send Wi-Fi settings" with an SSID and + password +- **THEN** the library reports the "provisioning" state, calls the + application's Wi-Fi hook with them, and reports "provisioned" once the + network is up, or the "unable to connect" error (0x03) if it is not up + within 30 seconds + +#### Scenario: Corrupted packet + +- **WHEN** a packet arrives whose checksum does not match +- **THEN** the library answers with the "invalid RPC" error and changes + nothing + +#### Scenario: Device info + +- **WHEN** a serial client sends "request device info" +- **THEN** the library answers with the firmware name, firmware version, + chip, and the device's external ID + +### Requirement: Joining Wi-Fi stays the application's job + +The library SHALL hand Wi-Fi settings to the application through a hook +given in `tedge_provisioning_init()`, and SHALL NOT join a network or store +Wi-Fi credentials itself. It SHALL judge success by the network coming up +(`tedge_port_net_is_up()`). An application without the hook SHALL get the +"unable to connect" error for every Wi-Fi settings command. + +#### Scenario: Application stores and joins the network + +- **WHEN** the library calls the Wi-Fi hook with an SSID and password +- **THEN** the application joins the network and stores the credentials in + its own way, and the library reports "provisioned" when the network is up + +### Requirement: The tiny-edge info command reports what onboarding needs + +The library SHALL answer a tiny-edge info command (0xE0) with the +provisioning protocol version, the external ID, the authentication mode +(`c8y-ca` or `bootstrap`), the tenant host if one is set, and the client's +state. It SHALL NOT include any password or credential in the reply. + +#### Scenario: Unregistered device + +- **WHEN** a tool sends the info command to a device waiting for + registration +- **THEN** the reply carries the external ID, `c8y-ca`, the tenant if set, + and the state "awaiting-registration" + +### Requirement: The tiny-edge onboard command hands the tenant and password to the client + +The library SHALL accept a tiny-edge onboard command (0xE1) carrying a +tenant host, a one-time password, and a flags byte. It SHALL validate both +strings as the client's setters do, and SHALL apply them through +`tedge_set_c8y_url()` and `tedge_set_enroll_otp()`. When the client is +running, it SHALL stop it with `tedge_stop()` first and start it again with +`tedge_start()` afterwards. It SHALL refuse the command, with the "not +authorized" error (0x04), when the client is already connected and the +flags byte does not have its re-onboard bit set. It SHALL NOT log the +one-time password above debug level. + +#### Scenario: Onboard a device waiting for registration + +- **WHEN** a tool sends the onboard command to a device in the + awaiting-registration state +- **THEN** the library sets the tenant and one-time password, restarts the + client, and answers with an empty result; the client enrolls with that + password + +#### Scenario: Connected device without the re-onboard flag + +- **WHEN** the onboard command arrives while the client is connected and + the re-onboard bit is clear +- **THEN** the library answers with the "not authorized" error and the + client keeps its certificate + +#### Scenario: Bootstrap build + +- **WHEN** the onboard command arrives on a build with bootstrap + authentication +- **THEN** the library answers with the "unknown RPC" error, since there is + no one-time password to set + +### Requirement: The device can identify itself on request + +The library SHALL accept a tiny-edge identify command (0xE2), so a tool +can tell several connected boards apart. It SHALL call the application's +identify hook when there is one (to blink an LED, for example) for about +10 seconds, and SHALL log a line naming the device in any case. It SHALL +answer the command with an empty result, with or without a hook. On BLE, +the Improv identify command (0x02) SHALL do the same. + +#### Scenario: Two boards on one computer + +- **WHEN** a tool sends the identify command to one of two boards +- **THEN** that board blinks (when the application has a hook) and logs that + it was asked to identify itself, and the other board does nothing + +### Requirement: Improv Serial on the console + +With `TEDGE_PROVISIONING_SERIAL`, the library SHALL speak Improv Serial +(header `IMPROV`, version 1) on the console the port provides. It SHALL +ignore any bytes outside a valid frame, and SHALL write each of its own +frames to the console as one unit, so log output never splits a frame. On a +port whose console is a shell, the port SHALL provide a shell command, +`tedge improv`, that hands the console to the library until onboarding +finishes or no valid frame arrives for 60 seconds, then returns it to the +shell. + +#### Scenario: Log output between frames + +- **WHEN** the device logs lines while a tool exchanges frames with it +- **THEN** the tool receives every frame intact, with the log lines between + them + +#### Scenario: Shell console + +- **WHEN** a tool sends `tedge improv` followed by Improv frames on a Zephyr + console with the shell +- **THEN** the library answers the frames, and the shell prompt returns once + onboarding finishes or after 60 seconds without a valid frame + +### Requirement: Improv BLE on the same RPC layer + +With `TEDGE_PROVISIONING_BLE`, the port SHALL expose the Improv Wi-Fi GATT +service and pass its RPC packets to the same RPC layer as the serial +transport. The commands and their results SHALL be the same on both +transports, apart from command 0x02. + +#### Scenario: Onboard over BLE + +- **WHEN** a BLE client sends the Wi-Fi settings and then the tiny-edge + onboard command +- **THEN** the device joins the network and enrolls exactly as it would over + serial diff --git a/openspec/changes/serial-onboarding/tasks.md b/openspec/changes/serial-onboarding/tasks.md new file mode 100644 index 0000000..30a717b --- /dev/null +++ b/openspec/changes/serial-onboarding/tasks.md @@ -0,0 +1,53 @@ +## 1. Client prerequisites + +- [ ] 1.1 Verify that `tedge_stop()` followed by `tedge_set_c8y_url()`, `tedge_set_enroll_otp()` and `tedge_start()` is clean while the client waits for registration, on the POSIX port (sample against a tenant) and on Zephyr native_sim; fix the core if a thread, socket or event survives the stop +- [ ] 1.2 Add a host test of that stop-start cycle to the POSIX port tests, so the library can rely on it + +## 2. Provisioning library (portable, `provisioning/`) + +- [ ] 2.1 Public header `include/tedge/provisioning.h`: init with the Wi-Fi and identify hooks, the transport's write callback, feeding received bytes, and the current Improv state +- [ ] 2.2 RPC layer: packet validation (length, checksum), Improv commands 0x01 to 0x03, errors 0x01 to 0x04, result packets with length-prefixed strings +- [ ] 2.3 The Wi-Fi flow: provisioning state, the application's hook, success when `tedge_port_net_is_up()`, "unable to connect" after 30 s +- [ ] 2.4 tiny-edge info (0xE0), onboard (0xE1) and identify (0xE2, with the application's hook), with the protocol version, the re-onboard flag, stop-set-start of the client, and no password in logs above debug +- [ ] 2.5 Improv Serial framing: a byte-at-a-time parser that skips non-frame bytes, and a frame writer that hands one whole frame to the transport +- [ ] 2.6 Host unit tests of the parser and the RPC layer (a C test under `tests/`, built by CMake and run in CI): every command, corrupted and split frames, log bytes between frames, oversized strings +- [ ] 2.7 Extend `scripts/check-portability.py` to apply the core rules to `provisioning/`, and `tests/host-compile/run.sh` to compile it + +## 3. Zephyr glue + +- [ ] 3.1 Kconfig `TEDGE_PROVISIONING`, `TEDGE_PROVISIONING_SERIAL` (and `TEDGE_PROVISIONING_BLE` declared but depending on `TEDGE_EXPERIMENTAL_FEATURES` until phase 2); CMake builds the library through `provisioning/sources.cmake` +- [ ] 3.2 Console transport: the `tedge improv` shell command using the shell bypass, the 60 s idle timeout, frame writes under the shell's output lock; the shell-less path reading the console UART +- [ ] 3.3 `samples/minimal`: an `overlay-onboarding.conf` and a Wi-Fi hook that joins and stores the network (Wi-Fi credentials subsystem) +- [ ] 3.4 Kconfig case for the new options in `tests/kconfig/cases`, and a twister build of the overlay for the ESP32-S3 and ESP32-C6 +- [ ] 3.5 Measure flash and RAM with and without the library on the ESP32-S3-DevKitC, ESP32-C6 and ESP32-WROOM-32 sample builds, and record them in proposal.md + +## 4. Host tool (`tools/onboard/`) + +- [ ] 4.1 Go module and the guided shell: steps with a spinner and a one-line result, choices with `charmbracelet/huh` (and its accessible mode), retry offers, "press Enter to close" when started outside a terminal, plain-language message catalogue +- [ ] 4.2 Script mode: `--yes`, `--json`, `--port`, `--wifi-ssid`, `--wifi-password-stdin`, `--skip-wifi`, `--reonboard`, `--timeout`, exit codes +- [ ] 4.3 Improv Serial client: frame codec, requests with timeouts, skipping log output; unit tests against recorded byte streams +- [ ] 4.4 Board discovery: list ports by USB vendor ID, open with DTR and RTS low, send `tedge improv` and the info request, describe each board that answers; wait for a board to be plugged in; the identify choice +- [ ] 4.5 Wi-Fi: current network and known networks on macOS (ipconfig, networksetup, Keychain), Linux (nmcli), Windows (netsh); hidden password prompt; re-ask when the board cannot join; unit tests with a fake command runner +- [ ] 4.6 go-c8y-cli: detect and explain installation, list sessions by tenant, run the interactive login in the same terminal, tenant host, existing-device check, `register-ca` with the chosen name and a generated password, waiting for the identity and availability; unit tests with a fake runner +- [ ] 4.7 The end-to-end flow tying 4.3 to 4.6, with the device's name and link at the end and "set up another board" +- [ ] 4.8 README for the tool written for a non-technical reader: download, run, what it reads from the computer and why, what to do when a step fails +- [ ] 4.9 Walk through the flow with someone unfamiliar with it (or a scripted fresh-user check), and fix every question that needed explaining + +## 5. CI and release + +- [ ] 5.1 Workflow: `go vet`, `go test`, and builds of the tool for Linux, macOS and Windows (amd64 and arm64) +- [ ] 5.2 Build and run the library's host tests in the posix workflow +- [ ] 5.3 Attach the tool's binaries to tiny-edge releases + +## 6. End-to-end verification + +- [ ] 6.1 Flash the Zephyr sample with the onboarding overlay to the ESP32-S3-DevKitC on this computer, run `tedge-onboard`, and confirm the device joins Wi-Fi, enrolls, and connects in Cumulocity +- [ ] 6.2 Run it again without `--reonboard` (refused), and with it (new certificate) +- [ ] 6.3 Check that ESP Web Tools or another standard Improv Serial client can at least set Wi-Fi on a shell-less build +- [ ] 6.4 Remove the test device from the tenant + +## 7. Phase 2: BLE (a separate pull request) + +- [ ] 7.1 Zephyr: the Improv GATT service and advertising, feeding the same RPC layer; measure RAM on each board +- [ ] 7.2 Tool: `--ble` transport with `tinygo.org/x/bluetooth` +- [ ] 7.3 End-to-end over BLE on the ESP32-S3 From fbd02193ef4f76953ad40f0c352105b8e7398d45 Mon Sep 17 00:00:00 2001 From: reubenmiller Date: Tue, 10 Nov 2026 14:52:16 +1000 Subject: [PATCH 02/14] fix(core): stop the client promptly while it waits for registration The enrollment loop slept up to 60 s between attempts, and nothing woke it. tedge_stop() gave up its 10 s join and reported success with the thread still running, so a following tedge_start() put a second thread on the same stack. The wait now ends when a stop is requested. tedge_stop() returns -EBUSY when the thread does not end in time, and the client stays marked as running. A second tedge_start() in the same boot no longer declares the client's own parameters and log type again. A POSIX test stops and starts the client while it is enrolling, with new onboarding data in between, as a provisioning library will. Co-Authored-By: Claude Opus 5.5 --- core/src/tedge_core.c | 35 +++++-- core/src/tedge_enroll.c | 5 +- core/src/tedge_internal.h | 5 + include/tedge/tedge.h | 11 ++- openspec/changes/serial-onboarding/tasks.md | 4 +- ports/posix/tests/CMakeLists.txt | 7 ++ ports/posix/tests/test_lifecycle.c | 101 ++++++++++++++++++++ 7 files changed, 157 insertions(+), 11 deletions(-) create mode 100644 ports/posix/tests/test_lifecycle.c diff --git a/core/src/tedge_core.c b/core/src/tedge_core.c index 1685e20..8d03f50 100644 --- a/core/src/tedge_core.c +++ b/core/src/tedge_core.c @@ -330,6 +330,9 @@ static void client_thread(void *arg) tedge_set_state(TEDGE_STATE_CONNECTING); rc = transport->connect(); + if (rc == -ECANCELED) { + break; /* tedge_stop() during enrollment */ + } if (rc != 0) { backoff = tedge_backoff_next(backoff, TEDGE_RECONNECT_BACKOFF_MAX_S); @@ -398,6 +401,7 @@ static void client_thread(void *arg) TEDGE_PORT_STACK_DEFINE(client_stack, TEDGE_THREAD_STACK_SIZE); static tedge_port_thread_t client_tid; static bool running; +static bool started_once; /* ------------------------------------------------------------------------ */ /* Public lifecycle */ @@ -461,18 +465,25 @@ int tedge_start(void) return -EALREADY; } tedge_port_event_clear(&events, EV_STOP); + /* Once per boot: a client stopped and started again (to take new + * onboarding data, say) has declared all this already. */ + if (!started_once) { + started_once = true; #if defined(TEDGE_LOG_UPLOAD) - tedge_log_upload_init(); + tedge_log_upload_init(); #endif #if defined(TEDGE_PARAMETERS_SELF) - /* After the application's sets, so a declaration of its own wins a - * name clash, and before the sweep, which needs them all. */ - tedge_self_params_declare(); + /* After the application's sets, so a declaration of its own + * wins a name clash, and before the sweep, which needs them + * all. */ + tedge_self_params_declare(); #endif #if defined(TEDGE_PARAMETERS) - /* Every set the application means to declare is declared by now. */ - tedge_params_on_start(); + /* Every set the application means to declare is declared by + * now. */ + tedge_params_on_start(); #endif + } running = true; (void)tedge_port_thread_start(&client_tid, client_stack, TEDGE_PORT_STACK_SIZEOF(client_stack), @@ -486,7 +497,17 @@ int tedge_stop(void) return -EALREADY; } tedge_port_event_post(&events, EV_STOP | EV_WAKE); - (void)tedge_port_thread_join(&client_tid, 10000); + /* A thread still running after the wait (stuck in a network call + * longer than any of its timeouts) keeps the client "running": a + * start now would put a second thread on the same stack. */ + if (tedge_port_thread_join(&client_tid, 10000) != 0) { + return -EBUSY; + } running = false; return 0; } + +bool tedge_wait_for_stop(uint32_t ms) +{ + return (tedge_port_event_wait(&events, EV_STOP, ms) & EV_STOP) != 0; +} diff --git a/core/src/tedge_enroll.c b/core/src/tedge_enroll.c index beae188..d58e43c 100644 --- a/core/src/tedge_enroll.c +++ b/core/src/tedge_enroll.c @@ -450,7 +450,10 @@ int tedge_auth_prepare(char *id_out, size_t id_len) if (ret == 0) { break; } - tedge_port_sleep_ms(wait_s * 1000); + if (tedge_wait_for_stop(wait_s * 1000)) { + ret = -ECANCELED; + break; + } wait_s = TEDGE_MIN(wait_s * 2, 60U); ret = -EAGAIN; } diff --git a/core/src/tedge_internal.h b/core/src/tedge_internal.h index e9b547a..517dcf9 100644 --- a/core/src/tedge_internal.h +++ b/core/src/tedge_internal.h @@ -65,6 +65,11 @@ int tedge_c8y_host_set(const char *host); /** Ask the client to drop this session and connect again (a new credential). */ void tedge_request_reconnect(void); +/** Wait up to @p ms, or less when tedge_stop() asks the client thread to + * end. @return true when a stop was asked for. For the client thread's + * long waits, so a stop does not wait them out. */ +bool tedge_wait_for_stop(uint32_t ms); + /** Republish every stored twin fragment (after a reconnect). */ void tedge_twin_republish(void); diff --git a/include/tedge/tedge.h b/include/tedge/tedge.h index 7c966fa..b0fda26 100644 --- a/include/tedge/tedge.h +++ b/include/tedge/tedge.h @@ -156,7 +156,16 @@ int tedge_init(const struct tedge_identity *identity, */ int tedge_start(void); -/** @brief Disconnect and stop the client thread. */ +/** + * @brief Disconnect and stop the client thread. + * + * Waits up to 10 seconds for the thread to end. The client can then be + * given new settings (tedge_set_c8y_url(), tedge_set_enroll_otp()) and + * started again with tedge_start(). + * + * @return 0; -EALREADY when the client is not running; -EBUSY when the + * thread did not end in time (call again later). + */ int tedge_stop(void); /** @brief Current state. */ diff --git a/openspec/changes/serial-onboarding/tasks.md b/openspec/changes/serial-onboarding/tasks.md index 30a717b..c7cc3cc 100644 --- a/openspec/changes/serial-onboarding/tasks.md +++ b/openspec/changes/serial-onboarding/tasks.md @@ -1,7 +1,7 @@ ## 1. Client prerequisites -- [ ] 1.1 Verify that `tedge_stop()` followed by `tedge_set_c8y_url()`, `tedge_set_enroll_otp()` and `tedge_start()` is clean while the client waits for registration, on the POSIX port (sample against a tenant) and on Zephyr native_sim; fix the core if a thread, socket or event survives the stop -- [ ] 1.2 Add a host test of that stop-start cycle to the POSIX port tests, so the library can rely on it +- [x] 1.1 Verify that `tedge_stop()` followed by `tedge_set_c8y_url()`, `tedge_set_enroll_otp()` and `tedge_start()` is clean while the client waits for registration, on the POSIX port (sample against a tenant) and on Zephyr native_sim; fix the core if a thread, socket or event survives the stop +- [x] 1.2 Add a host test of that stop-start cycle to the POSIX port tests, so the library can rely on it ## 2. Provisioning library (portable, `provisioning/`) diff --git a/ports/posix/tests/CMakeLists.txt b/ports/posix/tests/CMakeLists.txt index 481d95f..fc8411c 100644 --- a/ports/posix/tests/CMakeLists.txt +++ b/ports/posix/tests/CMakeLists.txt @@ -6,3 +6,10 @@ target_compile_definitions(tedge-port-test PRIVATE _POSIX_C_SOURCE=200809L _DEFAULT_SOURCE _DARWIN_C_SOURCE) target_link_libraries(tedge-port-test PRIVATE tedge::tedge) add_test(NAME port COMMAND tedge-port-test) + +add_executable(tedge-lifecycle-test test_lifecycle.c) +set_target_properties(tedge-lifecycle-test PROPERTIES C_STANDARD 11) +target_compile_definitions(tedge-lifecycle-test PRIVATE _POSIX_C_SOURCE=200809L + _DEFAULT_SOURCE _DARWIN_C_SOURCE) +target_link_libraries(tedge-lifecycle-test PRIVATE tedge::tedge) +add_test(NAME lifecycle COMMAND tedge-lifecycle-test) diff --git a/ports/posix/tests/test_lifecycle.c b/ports/posix/tests/test_lifecycle.c new file mode 100644 index 0000000..645e16e --- /dev/null +++ b/ports/posix/tests/test_lifecycle.c @@ -0,0 +1,101 @@ +/* SPDX-License-Identifier: Apache-2.0 + * + * The client stopped and started again, as a provisioning library does to + * hand it new onboarding data: the stop must end the client thread + * promptly even while it waits between enrollment attempts, and the client + * must start again without declaring anything twice. + * + * The tenant is 127.0.0.1, which refuses the connection, so the client + * sits in its enrollment wait (10 s and more) when it is stopped. + */ + +#include +#include +#include + +#include +#include +#include +#include +#include + +static int failures; + +#define CHECK(cond) \ + do { \ + if (!(cond)) { \ + fprintf(stderr, "%s:%d: %s\n", __FILE__, __LINE__, \ + #cond); \ + failures++; \ + } \ + } while (0) + +static enum tedge_state last_state; + +static void on_state(enum tedge_state state, void *user_data) +{ + (void)user_data; + last_state = state; +} + +static const struct tedge_hooks hooks = { .on_state = on_state }; + +/* Wait up to @p ms for the client to wait for registration. */ +static bool awaiting(uint32_t ms) +{ + for (uint32_t t = 0; t < ms; t += 50) { + if (last_state == TEDGE_STATE_AWAITING_REGISTRATION) { + return true; + } + tedge_port_sleep_ms(50); + } + return false; +} + +int main(void) +{ + char dir[] = "/tmp/tedge-lifecycle-test-XXXXXX"; + struct tedge_posix_config cfg = { 0 }; + struct tedge_identity id = { .external_id = "tedge-lifecycle-test" }; + int64_t t0; + + if (mkdtemp(dir) == NULL) { + perror("mkdtemp"); + return 1; + } + cfg.state_dir = dir; + CHECK(tedge_posix_configure(&cfg) == 0); + CHECK(tedge_init(&id, &hooks) == 0); + CHECK(tedge_set_c8y_url("127.0.0.1") == 0); + CHECK(tedge_stop() == -EALREADY); + CHECK(tedge_start() == 0); + CHECK(tedge_start() == -EALREADY); + + /* Enrolling: the first attempt is refused, then it waits. */ + CHECK(awaiting(5000)); + tedge_port_sleep_ms(500); + t0 = tedge_port_uptime_ms(); + CHECK(tedge_stop() == 0); + CHECK(tedge_port_uptime_ms() - t0 < 2000); /* not the 10 s wait */ + + /* New onboarding data, as a provisioning library hands it over. */ + CHECK(tedge_set_c8y_url("127.0.0.1") == 0); + CHECK(tedge_set_enroll_otp("Abcdefghijklmnopqrstuvwx") == 0); + last_state = TEDGE_STATE_STOPPED; + CHECK(tedge_start() == 0); + CHECK(awaiting(5000)); + t0 = tedge_port_uptime_ms(); + CHECK(tedge_stop() == 0); + CHECK(tedge_port_uptime_ms() - t0 < 2000); + + { + char rm[80]; + + snprintf(rm, sizeof(rm), "rm -rf %s", dir); + (void)!system(rm); + } + if (failures == 0) { + printf("stop and start: all checks passed\n"); + } + return failures == 0 ? 0 : 1; +} From baf7af4a05fbebe87e28d06868d484d8349b6533 Mon Sep 17 00:00:00 2001 From: reubenmiller Date: Tue, 10 Nov 2026 14:57:36 +1000 Subject: [PATCH 03/14] feat(core): read the device ID and tenant through the public API tedge_get_device_id() returns the external ID that tedge_init() settled on. tedge_get_c8y_url() returns the tenant host the client uses. Both are read-only. A provisioning tool needs them to tell the user which device it is talking to and where it reports. Co-Authored-By: Claude Opus 5.5 --- core/src/tedge_core.c | 27 +++++++++++++++++++++++++++ include/tedge/tedge.h | 16 ++++++++++++++++ 2 files changed, 43 insertions(+) diff --git a/core/src/tedge_core.c b/core/src/tedge_core.c index 8d03f50..9b4b539 100644 --- a/core/src/tedge_core.c +++ b/core/src/tedge_core.c @@ -179,6 +179,33 @@ int tedge_set_c8y_url(const char *host) return tedge_c8y_host_set(host); } +int tedge_get_c8y_url(char *buf, size_t len) +{ + const char *host = tedge_c8y_host(); + size_t n = strlen(host); + + if (n >= len) { + return -ENOSPC; + } + memcpy(buf, host, n + 1); + return (int)n; +} + +int tedge_get_device_id(char *buf, size_t len) +{ + size_t n; + + if (!initialised) { + return -EPERM; + } + n = strlen(identity.external_id); + if (n >= len) { + return -ENOSPC; + } + memcpy(buf, identity.external_id, n + 1); + return (int)n; +} + static void load_c8y_host(void) { int n = tedge_port_kv_get(TEDGE_KEY_C8Y_URL, c8y_host, diff --git a/include/tedge/tedge.h b/include/tedge/tedge.h index b0fda26..4240fa1 100644 --- a/include/tedge/tedge.h +++ b/include/tedge/tedge.h @@ -171,6 +171,22 @@ int tedge_stop(void); /** @brief Current state. */ enum tedge_state tedge_get_state(void); +/** + * @brief The device's external ID in Cumulocity, as tedge_init() settled it. + * + * @return Its length; -EPERM before tedge_init(); -ENOSPC when @p len is too + * small. + */ +int tedge_get_device_id(char *buf, size_t len); + +/** + * @brief The Cumulocity tenant host the client uses: the one set with + * tedge_set_c8y_url(), else TEDGE_C8Y_URL. Empty when neither is set. + * + * @return Its length; -ENOSPC when @p len is too small. + */ +int tedge_get_c8y_url(char *buf, size_t len); + /* ------------------------------------------------------------------------ */ /* Onboarding (TEDGE_AUTH_C8Y_CA) */ /* ------------------------------------------------------------------------ */ From 96eeb7d622043e28caa422a876397cae5d97c93b Mon Sep 17 00:00:00 2001 From: reubenmiller Date: Tue, 10 Nov 2026 14:57:36 +1000 Subject: [PATCH 04/14] feat(provisioning): an Improv Wi-Fi library for setting a device up An optional library beside the client (TEDGE_PROVISIONING) that answers the Improv Wi-Fi protocol. It hands the Wi-Fi network to the application through a hook, and the Cumulocity tenant and one-time password to the client through the public API. The client stays unaware of it. It answers Improv's Wi-Fi settings, current state, and device info commands, plus three of tiny-edge's own: - info (0xE0): the protocol version, the device ID, the authentication, the tenant, and the client's state; - onboard (0xE1): the tenant and password; the library stops the client, sets them, and starts it again, and refuses a connected device unless the re-onboard flag is set; - identify (0xE2): a hook to blink the board, and a log line. The RPC layer is shared by every transport. Improv Serial framing is here too: the port feeds the console's bytes, and the library skips anything that is not a valid frame, so log lines and typed text can share the console. Every answer is written from one worker thread. The library follows the core's portability rules, is compiled by the host-compile check, and is tested on the POSIX port. The tests cover every command, damaged and split frames, log text between frames, and the hand-over to a real client. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/posix.yml | 4 +- include/tedge/provisioning.h | 107 +++++ .../changes/serial-onboarding/proposal.md | 11 +- openspec/changes/serial-onboarding/tasks.md | 16 +- ports/posix/CMakeLists.txt | 3 + ports/posix/cmake/options.cmake | 8 + ports/posix/tests/CMakeLists.txt | 9 + ports/posix/tests/test_provisioning.c | 324 ++++++++++++++ provisioning/sources.cmake | 22 + provisioning/src/tedge_improv_serial.c | 163 +++++++ provisioning/src/tedge_provisioning.c | 403 ++++++++++++++++++ .../src/tedge_provisioning_internal.h | 50 +++ scripts/check-portability.py | 7 +- tests/host-compile/include/tedge_config.h | 4 + tests/host-compile/run.sh | 6 +- 15 files changed, 1117 insertions(+), 20 deletions(-) create mode 100644 include/tedge/provisioning.h create mode 100644 ports/posix/tests/test_provisioning.c create mode 100644 provisioning/sources.cmake create mode 100644 provisioning/src/tedge_improv_serial.c create mode 100644 provisioning/src/tedge_provisioning.c create mode 100644 provisioning/src/tedge_provisioning_internal.h diff --git a/.github/workflows/posix.yml b/.github/workflows/posix.yml index f7418b0..4d6653e 100644 --- a/.github/workflows/posix.yml +++ b/.github/workflows/posix.yml @@ -34,9 +34,9 @@ jobs: steps: - uses: actions/checkout@v4 - - name: Every feature the port has + - name: Every feature the port has, and the provisioning library run: | - cmake -S ports/posix -B build -DTEDGE_WERROR=ON + cmake -S ports/posix -B build -DTEDGE_WERROR=ON -DTEDGE_PROVISIONING=ON cmake --build build -j4 ctest --test-dir build --output-on-failure diff --git a/include/tedge/provisioning.h b/include/tedge/provisioning.h new file mode 100644 index 0000000..c4016fe --- /dev/null +++ b/include/tedge/provisioning.h @@ -0,0 +1,107 @@ +/* SPDX-License-Identifier: Apache-2.0 */ + +/** + * @file + * @brief Setting a device up over a cable or Bluetooth (TEDGE_PROVISIONING). + * + * An optional library beside the client: it answers the Improv Wi-Fi + * protocol (https://www.improv-wifi.com), which tools such as + * `tedge-onboard`, ESP Web Tools and Improv phone apps speak, and hands what + * it receives to the application (the Wi-Fi network) and to the client (the + * Cumulocity tenant and one-time password, through tedge_set_c8y_url() and + * tedge_set_enroll_otp()). The client itself knows nothing about it. + * + * The application calls tedge_provisioning_init() after tedge_init(). The + * port connects a transport: for Improv Serial, it feeds the console's + * bytes to tedge_provisioning_serial_feed() and writes what the library + * gives its write function. + * + * Besides the Improv commands, the library answers three of tiny-edge's + * own (TEDGE_PROVISIONING_CMD_*): what onboarding needs to know about the + * device, the onboarding data itself, and a request to identify the board. + */ + +#ifndef TEDGE_PROVISIONING_H_ +#define TEDGE_PROVISIONING_H_ + +#include +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +/** The version of tiny-edge's commands, reported by the info command. A + * tool that knows an older version refuses a newer device. */ +#define TEDGE_PROVISIONING_PROTOCOL_VERSION 1 + +/** tiny-edge's RPC commands, numbered away from Improv's own. */ +#define TEDGE_PROVISIONING_CMD_INFO 0xE0 +#define TEDGE_PROVISIONING_CMD_ONBOARD 0xE1 +#define TEDGE_PROVISIONING_CMD_IDENTIFY 0xE2 + +/** In the onboard command's flags: set the device up again even though it + * is connected (it gets a new certificate). */ +#define TEDGE_PROVISIONING_FLAG_REONBOARD 0x01 + +/** What the application provides. Every member may be NULL. */ +struct tedge_provisioning_hooks { + /** + * Join @p ssid with @p password (empty for an open network) and keep + * them for the next boot, in the application's own way. Block until + * the network is joined or the attempt failed, for 30 s at most. + * Without it, every Wi-Fi request is answered "unable to connect". + * + * @return 0 once joined, or a negative errno. + */ + int (*wifi_connect)(const char *ssid, const char *password, void *ctx); + + /** Make this board recognisable for about 10 s (blink an LED, say). + * The library also logs a line naming the device. */ + void (*identify)(void *ctx); + + /** The chip or board, for Improv's device info ("ESP32-S3"). NULL + * reports "unknown". */ + const char *chip; + + void *ctx; +}; + +/** + * Start the library: its worker thread, and the state it reports. Call once, + * after tedge_init(). @p hooks is kept, so it must stay valid. + * + * @return 0, or a negative errno. + */ +int tedge_provisioning_init(const struct tedge_provisioning_hooks *hooks); + +/* --- Improv Serial (TEDGE_PROVISIONING_SERIAL) --------------------------- */ + +/** + * Writes @p len bytes to the console, all of them, as one unit: no other + * output (log lines) may land in between. Called from the library's + * worker thread. + */ +typedef void (*tedge_provisioning_write_t)(const void *data, size_t len, + void *ctx); + +/** Connect the serial transport: the library writes its frames with + * @p write. NULL disconnects it. */ +void tedge_provisioning_serial_attach(tedge_provisioning_write_t write, + void *ctx); + +/** + * Feed bytes received on the console. Bytes outside a valid Improv frame + * (typed text, a tool's other output) are skipped. May be called from any + * thread; complete frames are handled on the library's worker thread. + * + * @return How many valid frames these bytes completed. + */ +size_t tedge_provisioning_serial_feed(const void *data, size_t len); + +#ifdef __cplusplus +} +#endif + +#endif /* TEDGE_PROVISIONING_H_ */ diff --git a/openspec/changes/serial-onboarding/proposal.md b/openspec/changes/serial-onboarding/proposal.md index 85b895b..37b27cb 100644 --- a/openspec/changes/serial-onboarding/proposal.md +++ b/openspec/changes/serial-onboarding/proposal.md @@ -62,10 +62,13 @@ it is a separate option and a separate phase. ### Interface changes -- **Public `tedge_*` API**: none. The library is a client of the existing - API. It has a public header of its own, `include/tedge/provisioning.h`, - through which the application supplies the Wi-Fi hook and reads the - progress. +- **Public `tedge_*` API**: two read-only getters, + `tedge_get_device_id()` and `tedge_get_c8y_url()`, which the library + needs for its info command and which any application can use. + `tedge_stop()` now returns -EBUSY when the client thread does not end in + time, instead of reporting success. The library has a public header of + its own, `include/tedge/provisioning.h`, through which the application + supplies the Wi-Fi and identify hooks and the port connects a transport. - **Port interface**: none. The serial transport is fed by the port's glue code, outside the port interface. - **Messages the device sends to Cumulocity**: none. diff --git a/openspec/changes/serial-onboarding/tasks.md b/openspec/changes/serial-onboarding/tasks.md index c7cc3cc..2ae1ff1 100644 --- a/openspec/changes/serial-onboarding/tasks.md +++ b/openspec/changes/serial-onboarding/tasks.md @@ -5,13 +5,13 @@ ## 2. Provisioning library (portable, `provisioning/`) -- [ ] 2.1 Public header `include/tedge/provisioning.h`: init with the Wi-Fi and identify hooks, the transport's write callback, feeding received bytes, and the current Improv state -- [ ] 2.2 RPC layer: packet validation (length, checksum), Improv commands 0x01 to 0x03, errors 0x01 to 0x04, result packets with length-prefixed strings -- [ ] 2.3 The Wi-Fi flow: provisioning state, the application's hook, success when `tedge_port_net_is_up()`, "unable to connect" after 30 s -- [ ] 2.4 tiny-edge info (0xE0), onboard (0xE1) and identify (0xE2, with the application's hook), with the protocol version, the re-onboard flag, stop-set-start of the client, and no password in logs above debug -- [ ] 2.5 Improv Serial framing: a byte-at-a-time parser that skips non-frame bytes, and a frame writer that hands one whole frame to the transport -- [ ] 2.6 Host unit tests of the parser and the RPC layer (a C test under `tests/`, built by CMake and run in CI): every command, corrupted and split frames, log bytes between frames, oversized strings -- [ ] 2.7 Extend `scripts/check-portability.py` to apply the core rules to `provisioning/`, and `tests/host-compile/run.sh` to compile it +- [x] 2.1 Public header `include/tedge/provisioning.h`: init with the Wi-Fi and identify hooks, the transport's write callback, feeding received bytes, and the current Improv state +- [x] 2.2 RPC layer: packet validation (length, checksum), Improv commands 0x01 to 0x03, errors 0x01 to 0x04, result packets with length-prefixed strings +- [x] 2.3 The Wi-Fi flow: provisioning state, the application's hook, success when `tedge_port_net_is_up()`, "unable to connect" after 30 s +- [x] 2.4 tiny-edge info (0xE0), onboard (0xE1) and identify (0xE2, with the application's hook), with the protocol version, the re-onboard flag, stop-set-start of the client, and no password in logs above debug +- [x] 2.5 Improv Serial framing: a byte-at-a-time parser that skips non-frame bytes, and a frame writer that hands one whole frame to the transport +- [x] 2.6 Host unit tests of the parser and the RPC layer (a C test in the POSIX port's tests, built by CMake and run in CI): every command, corrupted and split frames, log bytes between frames, oversized strings +- [x] 2.7 Extend `scripts/check-portability.py` to apply the core rules to `provisioning/`, and `tests/host-compile/run.sh` to compile it ## 3. Zephyr glue @@ -36,7 +36,7 @@ ## 5. CI and release - [ ] 5.1 Workflow: `go vet`, `go test`, and builds of the tool for Linux, macOS and Windows (amd64 and arm64) -- [ ] 5.2 Build and run the library's host tests in the posix workflow +- [x] 5.2 Build and run the library's host tests in the posix workflow - [ ] 5.3 Attach the tool's binaries to tiny-edge releases ## 6. End-to-end verification diff --git a/ports/posix/CMakeLists.txt b/ports/posix/CMakeLists.txt index aa995ef..a6a253d 100644 --- a/ports/posix/CMakeLists.txt +++ b/ports/posix/CMakeLists.txt @@ -32,13 +32,16 @@ option(TEDGE_POSIX_TESTS "Build the port's tests (ctest)" ${_top}) include(cmake/options.cmake) include(cmake/mbedtls.cmake) include(${TEDGE_ROOT}/core/sources.cmake) +include(${TEDGE_ROOT}/provisioning/sources.cmake) set(THREADS_PREFER_PTHREAD_FLAG ON) find_package(Threads REQUIRED) tedge_core_sources(_core_sources "") +tedge_provisioning_sources(_provisioning_sources "") add_library(tedge STATIC ${_core_sources} + ${_provisioning_sources} src/tedge_posix.c src/tedge_port_os.c src/tedge_port_log.c diff --git a/ports/posix/cmake/options.cmake b/ports/posix/cmake/options.cmake index 5867456..371eb11 100644 --- a/ports/posix/cmake/options.cmake +++ b/ports/posix/cmake/options.cmake @@ -147,6 +147,14 @@ tedge_option(TEDGE_REMOTE_ACCESS_RESOLVE_RETRY_DELAY_MS INT 500 "Pause between l tedge_option(TEDGE_HTTP_MAX_REDIRECTS INT 3 "HTTP redirects to follow") +# The provisioning library (include/tedge/provisioning.h), beside the client. +tedge_option(TEDGE_PROVISIONING BOOL OFF "Setting the device up over Improv (provisioning library)") +tedge_option(TEDGE_PROVISIONING_SERIAL BOOL ON "Improv Serial, fed by the application") +tedge_option(TEDGE_PROVISIONING_STACK_SIZE INT 4096 "Stack of the provisioning worker (unused on POSIX)") +if(NOT TEDGE_PROVISIONING) + set(TEDGE_PROVISIONING_SERIAL OFF) +endif() + # Not in this port yet: each needs a POSIX design of its own (an image # format and a launcher that can roll back; a command runner; crash dumps). foreach(_missing TEDGE_FIRMWARE_UPDATE TEDGE_SHELL_COMMAND TEDGE_COREDUMP) diff --git a/ports/posix/tests/CMakeLists.txt b/ports/posix/tests/CMakeLists.txt index fc8411c..cd20c39 100644 --- a/ports/posix/tests/CMakeLists.txt +++ b/ports/posix/tests/CMakeLists.txt @@ -13,3 +13,12 @@ target_compile_definitions(tedge-lifecycle-test PRIVATE _POSIX_C_SOURCE=200809L _DEFAULT_SOURCE _DARWIN_C_SOURCE) target_link_libraries(tedge-lifecycle-test PRIVATE tedge::tedge) add_test(NAME lifecycle COMMAND tedge-lifecycle-test) + +if(TEDGE_PROVISIONING) + add_executable(tedge-provisioning-test test_provisioning.c) + set_target_properties(tedge-provisioning-test PROPERTIES C_STANDARD 11) + target_compile_definitions(tedge-provisioning-test PRIVATE + _POSIX_C_SOURCE=200809L _DEFAULT_SOURCE _DARWIN_C_SOURCE) + target_link_libraries(tedge-provisioning-test PRIVATE tedge::tedge) + add_test(NAME provisioning COMMAND tedge-provisioning-test) +endif() diff --git a/ports/posix/tests/test_provisioning.c b/ports/posix/tests/test_provisioning.c new file mode 100644 index 0000000..1dc2f01 --- /dev/null +++ b/ports/posix/tests/test_provisioning.c @@ -0,0 +1,324 @@ +/* SPDX-License-Identifier: Apache-2.0 + * + * The provisioning library (include/tedge/provisioning.h) over Improv + * Serial, with the real client on the POSIX port: every command, damaged + * and split frames, log text between frames, and the onboarding hand-over + * to the client. + */ + +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include + +static int failures; + +#define CHECK(cond) \ + do { \ + if (!(cond)) { \ + fprintf(stderr, "%s:%d: %s\n", __FILE__, __LINE__, \ + #cond); \ + failures++; \ + } \ + } while (0) + +/* --- What the device writes ---------------------------------------------- */ + +static pthread_mutex_t out_lock = PTHREAD_MUTEX_INITIALIZER; +static uint8_t out[8192]; +static size_t out_len; + +static void capture(const void *data, size_t len, void *ctx) +{ + (void)ctx; + pthread_mutex_lock(&out_lock); + if (out_len + len <= sizeof(out)) { + memcpy(out + out_len, data, len); + out_len += len; + } + pthread_mutex_unlock(&out_lock); +} + +struct frame { + uint8_t type; + uint8_t len; + uint8_t data[255]; +}; + +/* The @p index-th valid frame the device has written, if there is one. */ +static bool frame_at(size_t index, struct frame *f) +{ + size_t found = 0; + bool ok = false; + + pthread_mutex_lock(&out_lock); + for (size_t i = 0; i + 10 <= out_len; i++) { + uint8_t sum = 0; + size_t len; + + if (memcmp(out + i, "IMPROV", 6) != 0 || out[i + 6] != 1) { + continue; + } + len = out[i + 8]; + if (i + 9 + len + 1 > out_len) { + break; + } + for (size_t k = 0; k < 9 + len; k++) { + sum += out[i + k]; + } + if (sum != out[i + 9 + len]) { + continue; + } + if (found++ == index) { + f->type = out[i + 7]; + f->len = (uint8_t)len; + memcpy(f->data, out + i + 9, len); + ok = true; + break; + } + i += 9 + len; + } + pthread_mutex_unlock(&out_lock); + return ok; +} + +static size_t next_frame; + +/* Wait for the device's next frame. */ +static bool expect(struct frame *f) +{ + for (int t = 0; t < 400; t++) { + if (frame_at(next_frame, f)) { + next_frame++; + return true; + } + tedge_port_sleep_ms(10); + } + return false; +} + +/* The @p i-th string of an RPC result frame, NUL-terminated into @p s. */ +static bool result_string(const struct frame *f, int i, char *s, size_t cap) +{ + size_t pos = 2; /* after [command][length] */ + + for (int k = 0; pos < f->len; k++) { + size_t n = f->data[pos++]; + + if (pos + n > f->len || n >= cap) { + return false; + } + if (k == i) { + memcpy(s, f->data + pos, n); + s[n] = '\0'; + return true; + } + pos += n; + } + return false; +} + +/* --- What a tool sends --------------------------------------------------- */ + +static size_t encode(uint8_t *buf, uint8_t cmd, const uint8_t *data, + size_t len) +{ + uint8_t sum = 0; + size_t n = 0; + + memcpy(buf, "IMPROV", 6); + n = 6; + buf[n++] = 1; /* version */ + buf[n++] = 0x03; /* RPC command */ + buf[n++] = (uint8_t)(len + 2); + buf[n++] = cmd; + buf[n++] = (uint8_t)len; + memcpy(buf + n, data, len); + n += len; + for (size_t i = 0; i < n; i++) { + sum += buf[i]; + } + buf[n++] = sum; + return n; +} + +static void send_rpc(uint8_t cmd, const uint8_t *data, size_t len) +{ + uint8_t buf[300]; + size_t n = encode(buf, cmd, data, len); + + CHECK(tedge_provisioning_serial_feed(buf, n) == 1); +} + +/* Length-prefixed strings, as Improv's payloads carry them. */ +static size_t strings(uint8_t *buf, const char *a, const char *b) +{ + size_t n = 0; + + buf[n++] = (uint8_t)strlen(a); + memcpy(buf + n, a, strlen(a)); + n += strlen(a); + if (b != NULL) { + buf[n++] = (uint8_t)strlen(b); + memcpy(buf + n, b, strlen(b)); + n += strlen(b); + } + return n; +} + +/* --- The application's hooks --------------------------------------------- */ + +static int wifi_rc; +static char wifi_ssid[40], wifi_password[80]; +static int identified; + +static int wifi_connect(const char *ssid, const char *password, void *ctx) +{ + (void)ctx; + snprintf(wifi_ssid, sizeof(wifi_ssid), "%s", ssid); + snprintf(wifi_password, sizeof(wifi_password), "%s", password); + return wifi_rc; +} + +static void identify(void *ctx) +{ + (void)ctx; + identified++; +} + +static const struct tedge_provisioning_hooks hooks = { + .wifi_connect = wifi_connect, + .identify = identify, + .chip = "host", +}; + +int main(void) +{ + char dir[] = "/tmp/tedge-provisioning-test-XXXXXX"; + struct tedge_posix_config cfg = { 0 }; + struct tedge_identity id = { .external_id = "tedge-prov-test" }; + struct frame f; + uint8_t payload[200]; + char s[80]; + size_t n; + + if (mkdtemp(dir) == NULL) { + perror("mkdtemp"); + return 1; + } + cfg.state_dir = dir; + CHECK(tedge_posix_configure(&cfg) == 0); + CHECK(tedge_init(&id, NULL) == 0); + CHECK(tedge_provisioning_init(&hooks) == 0); + CHECK(tedge_provisioning_init(&hooks) == -EALREADY); + tedge_provisioning_serial_attach(capture, NULL); + + /* Improv's device info: firmware, version, chip, the device's ID. */ + send_rpc(0x03, NULL, 0); + CHECK(expect(&f) && f.type == 0x04 && f.data[0] == 0x03); + CHECK(result_string(&f, 2, s, sizeof(s)) && strcmp(s, "host") == 0); + CHECK(result_string(&f, 3, s, sizeof(s)) && strcmp(s, "tedge-prov-test") == 0); + + /* tiny-edge's info, before any tenant is set. */ + send_rpc(TEDGE_PROVISIONING_CMD_INFO, NULL, 0); + CHECK(expect(&f) && f.type == 0x04 && f.data[0] == 0xE0); + CHECK(result_string(&f, 0, s, sizeof(s)) && strcmp(s, "1") == 0); + CHECK(result_string(&f, 1, s, sizeof(s)) && strcmp(s, "tedge-prov-test") == 0); + CHECK(result_string(&f, 2, s, sizeof(s)) && strcmp(s, "c8y-ca") == 0); + CHECK(result_string(&f, 3, s, sizeof(s)) && strcmp(s, "") == 0); + CHECK(result_string(&f, 4, s, sizeof(s)) && strcmp(s, "stopped") == 0); + + /* A command nobody knows. */ + send_rpc(0x7F, NULL, 0); + CHECK(expect(&f) && f.type == 0x02 && f.data[0] == 0x02); + + /* Log text around a frame that arrives a byte at a time. */ + { + const char *noise = " tedge: IMPRO something IMPROV\x02 more\r\n"; + uint8_t buf[64]; + + n = encode(buf, TEDGE_PROVISIONING_CMD_IDENTIFY, NULL, 0); + CHECK(tedge_provisioning_serial_feed(noise, strlen(noise)) == 0); + for (size_t i = 0; i + 1 < n; i++) { + CHECK(tedge_provisioning_serial_feed(&buf[i], 1) == 0); + } + CHECK(tedge_provisioning_serial_feed(&buf[n - 1], 1) == 1); + CHECK(expect(&f) && f.type == 0x04 && f.data[0] == 0xE2); + CHECK(identified == 1); + } + + /* A damaged frame: answered "invalid RPC", and nothing happens. */ + { + uint8_t buf[64]; + + n = encode(buf, TEDGE_PROVISIONING_CMD_IDENTIFY, NULL, 0); + buf[n - 1] ^= 0x55; + CHECK(tedge_provisioning_serial_feed(buf, n) == 0); + CHECK(expect(&f) && f.type == 0x02 && f.data[0] == 0x01); + CHECK(identified == 1); + } + + /* Wi-Fi: joined (the host's network is up). */ + wifi_rc = 0; + n = strings(payload, "lab-net", "secret-pw"); + send_rpc(0x01, payload, n); + CHECK(expect(&f) && f.type == 0x01 && f.data[0] == 0x03); /* provisioning */ + CHECK(expect(&f) && f.type == 0x01 && f.data[0] == 0x04); /* provisioned */ + CHECK(expect(&f) && f.type == 0x04 && f.data[0] == 0x01); + CHECK(strcmp(wifi_ssid, "lab-net") == 0 && + strcmp(wifi_password, "secret-pw") == 0); + + /* Wi-Fi: the network refuses. */ + wifi_rc = -ECONNREFUSED; + send_rpc(0x01, payload, n); + CHECK(expect(&f) && f.type == 0x01 && f.data[0] == 0x03); + CHECK(expect(&f) && f.type == 0x02 && f.data[0] == 0x03); /* unable */ + CHECK(expect(&f) && f.type == 0x01 && f.data[0] == 0x02); /* ready */ + + /* Wi-Fi: no SSID, and a string running past the packet. */ + n = strings(payload, "", "pw"); + send_rpc(0x01, payload, n); + CHECK(expect(&f) && f.type == 0x02 && f.data[0] == 0x01); + payload[0] = 50; + send_rpc(0x01, payload, 4); + CHECK(expect(&f) && f.type == 0x02 && f.data[0] == 0x01); + + /* Onboarding without a password is refused. */ + payload[0] = 0; + n = 1 + strings(payload + 1, "127.0.0.1", NULL); + send_rpc(TEDGE_PROVISIONING_CMD_ONBOARD, payload, n); + CHECK(expect(&f) && f.type == 0x02 && f.data[0] == 0x01); + + /* Onboarding: the client takes the tenant and starts enrolling. */ + payload[0] = 0; + n = 1 + strings(payload + 1, "127.0.0.1", "Abcdefghijklmnopqrstuvwx"); + send_rpc(TEDGE_PROVISIONING_CMD_ONBOARD, payload, n); + CHECK(expect(&f) && f.type == 0x04 && f.data[0] == 0xE1); + send_rpc(TEDGE_PROVISIONING_CMD_INFO, NULL, 0); + CHECK(expect(&f) && f.type == 0x04); + CHECK(result_string(&f, 3, s, sizeof(s)) && strcmp(s, "127.0.0.1") == 0); + CHECK(result_string(&f, 4, s, sizeof(s)) && strcmp(s, "stopped") != 0); + + /* And again while it enrolls: it stops, takes the new data, starts. */ + send_rpc(TEDGE_PROVISIONING_CMD_ONBOARD, payload, n); + CHECK(expect(&f) && f.type == 0x04 && f.data[0] == 0xE1); + CHECK(tedge_stop() == 0); + + { + char rm[80]; + + snprintf(rm, sizeof(rm), "rm -rf %s", dir); + (void)!system(rm); + } + if (failures == 0) { + printf("provisioning: all checks passed\n"); + } + return failures == 0 ? 0 : 1; +} diff --git a/provisioning/sources.cmake b/provisioning/sources.cmake new file mode 100644 index 0000000..b6e22d1 --- /dev/null +++ b/provisioning/sources.cmake @@ -0,0 +1,22 @@ +# SPDX-License-Identifier: Apache-2.0 +# +# The provisioning library's source files (include/tedge/provisioning.h), +# for every port, as core/sources.cmake lists the client's: +# +# include(/provisioning/sources.cmake) +# tedge_provisioning_sources(srcs "") # options are TEDGE_* variables +# tedge_provisioning_sources(srcs ) # options are TEDGE_* +# +# Empty unless TEDGE_PROVISIONING is on. + +function(tedge_provisioning_sources out prefix) + set(src "") + if(${prefix}TEDGE_PROVISIONING) + list(APPEND src tedge_provisioning.c) + if(${prefix}TEDGE_PROVISIONING_SERIAL) + list(APPEND src tedge_improv_serial.c) + endif() + endif() + list(TRANSFORM src PREPEND ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/src/) + set(${out} ${src} PARENT_SCOPE) +endfunction() diff --git a/provisioning/src/tedge_improv_serial.c b/provisioning/src/tedge_improv_serial.c new file mode 100644 index 0000000..218bed2 --- /dev/null +++ b/provisioning/src/tedge_improv_serial.c @@ -0,0 +1,163 @@ +/* SPDX-License-Identifier: Apache-2.0 + * + * Improv Serial framing: each message on the console is + * + * "IMPROV" | version (1) | type | length | data | checksum + * + * the checksum being the low byte of the sum of every byte before it. The + * device ends its frames with a newline, so they stand on lines of their + * own between log lines. Received bytes outside a valid frame are skipped, + * so a console can carry Improv, typed text and logs at once. + * + * Portable: the port feeds bytes and writes frames (tedge/provisioning.h). + */ + +#include "tedge_provisioning_internal.h" + +#include + +#include + +#define VERSION 1 + +#define TYPE_STATE 0x01 +#define TYPE_ERROR 0x02 +#define TYPE_RPC 0x03 +#define TYPE_RESULT 0x04 + +static const uint8_t header[6] = { 'I', 'M', 'P', 'R', 'O', 'V' }; + +/* --- Writing ------------------------------------------------------------- */ + +static tedge_provisioning_write_t writer; +static void *writer_ctx; + +void tedge_provisioning_serial_attach(tedge_provisioning_write_t write, + void *ctx) +{ + writer = write; + writer_ctx = ctx; +} + +/* Only the worker thread writes, so one buffer serves every frame. */ +static void send_frame(uint8_t type, const uint8_t *data, size_t len) +{ + static uint8_t frame[sizeof(header) + 3 + 255 + 2]; + uint8_t sum = 0; + size_t n = 0; + + if (writer == NULL || len > 255) { + return; + } + memcpy(frame, header, sizeof(header)); + n = sizeof(header); + frame[n++] = VERSION; + frame[n++] = type; + frame[n++] = (uint8_t)len; + memcpy(&frame[n], data, len); + n += len; + for (size_t i = 0; i < n; i++) { + sum += frame[i]; + } + frame[n++] = sum; + frame[n++] = '\n'; + writer(frame, n, writer_ctx); +} + +static void out_state(uint8_t state) +{ + send_frame(TYPE_STATE, &state, 1); +} + +static void out_error(uint8_t error) +{ + send_frame(TYPE_ERROR, &error, 1); +} + +static void out_result(const uint8_t *data, size_t len) +{ + send_frame(TYPE_RESULT, data, len); +} + +const struct rpc_out tedge_prov_serial_out = { + .state = out_state, + .error = out_error, + .result = out_result, +}; + +/* --- Reading ------------------------------------------------------------- */ + +static struct { + size_t pos; /* bytes of the current frame so far */ + uint8_t type; + uint8_t len; + uint8_t sum; + uint8_t data[255]; +} rx; + +TEDGE_PORT_MUTEX_DEFINE(rx_lock); + +/* One byte. @return true when it completed a valid frame. */ +static bool rx_byte(uint8_t b) +{ + size_t at = rx.pos; + + if (at < sizeof(header)) { + if (b == header[at]) { + rx.pos++; + rx.sum = (uint8_t)(at == 0 ? b : rx.sum + b); + } else { + /* A new frame may start right here. */ + rx.pos = (b == header[0]) ? 1 : 0; + rx.sum = b; + } + return false; + } + if (at == sizeof(header)) { /* version */ + if (b != VERSION) { + rx.pos = (b == header[0]) ? 1 : 0; + rx.sum = b; + return false; + } + } else if (at == sizeof(header) + 1) { + rx.type = b; + } else if (at == sizeof(header) + 2) { + rx.len = b; + } else if (at < sizeof(header) + 3 + rx.len) { + rx.data[at - sizeof(header) - 3] = b; + } else { /* the checksum */ + bool valid = (b == rx.sum); + + rx.pos = 0; + if (!valid) { + /* A whole frame arrived damaged: say so (an empty packet + * is answered "invalid RPC"), from the worker, which + * writes every answer. */ + (void)tedge_prov_rpc_submit(NULL, 0, &tedge_prov_serial_out); + return false; + } + if (rx.type == TYPE_RPC) { + (void)tedge_prov_rpc_submit(rx.data, rx.len, + &tedge_prov_serial_out); + } + return true; + } + rx.sum += b; + rx.pos++; + return false; +} + +size_t tedge_provisioning_serial_feed(const void *data, size_t len) +{ + const uint8_t *p = data; + size_t frames = 0; + + tedge_port_mutex_lock(&rx_lock); + for (size_t i = 0; i < len; i++) { + if (rx_byte(p[i])) { + frames++; + } + } + tedge_port_mutex_unlock(&rx_lock); + return frames; +} diff --git a/provisioning/src/tedge_provisioning.c b/provisioning/src/tedge_provisioning.c new file mode 100644 index 0000000..bd4483a --- /dev/null +++ b/provisioning/src/tedge_provisioning.c @@ -0,0 +1,403 @@ +/* SPDX-License-Identifier: Apache-2.0 + * + * The RPC layer: Improv's commands and tiny-edge's own, the same on every + * transport. Packets are queued by the transport and handled on one + * worker thread, since joining a network or restarting the client takes + * seconds and must not hold up the console. + * + * Portable: the public tedge_* API and the port interface only. + */ + +#include "tedge_provisioning_internal.h" + +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +TEDGE_LOG_DECLARE(); + +/* After the application's hook has joined the network, how long to wait + * for an address. */ +#define ADDRESS_WAIT_MS 15000 + +#define EV_RPC (1u << 0) + +#define STR_(x) #x +#define STR(x) STR_(x) + +struct rpc_msg { + const struct rpc_out *out; + uint8_t len; + uint8_t data[RPC_MAX]; +}; + +TEDGE_PORT_MSGQ_DEFINE(tedge_prov_queue, sizeof(struct rpc_msg), 2); +TEDGE_PORT_EVENT_DEFINE(prov_events); +TEDGE_PORT_STACK_DEFINE(prov_stack, TEDGE_PROVISIONING_STACK_SIZE); +static tedge_port_thread_t prov_thread; + +static const struct tedge_provisioning_hooks *hooks; +static uint8_t improv_state = IMPROV_STATE_READY; + +/* --- Results --------------------------------------------------------------- */ + +/* A result being built: [command][length][strings...]. */ +struct result { + uint8_t buf[2 + 255]; + size_t len; + bool overflow; +}; + +static void result_start(struct result *r, uint8_t cmd) +{ + r->buf[0] = cmd; + r->len = 2; + r->overflow = false; +} + +static void result_add(struct result *r, const char *s) +{ + size_t n = strlen(s); + + if (n > 255 || r->len + 1 + n > sizeof(r->buf)) { + r->overflow = true; + return; + } + r->buf[r->len++] = (uint8_t)n; + memcpy(&r->buf[r->len], s, n); + r->len += n; +} + +static void result_send(struct result *r, const struct rpc_out *out) +{ + if (r->overflow) { + out->error(IMPROV_ERR_UNKNOWN); + return; + } + r->buf[1] = (uint8_t)(r->len - 2); + out->result(r->buf, r->len); +} + +static void send_empty_result(uint8_t cmd, const struct rpc_out *out) +{ + struct result r; + + result_start(&r, cmd); + result_send(&r, out); +} + +static void set_state(uint8_t state, const struct rpc_out *out) +{ + improv_state = state; + out->state(state); +} + +/* --- Reading a packet ---------------------------------------------------- */ + +/* A length-prefixed string at @p *pos of @p data (length @p len), copied + * NUL-terminated into @p out (room for @p cap bytes with the NUL). + * @return false when it does not fit or runs past the packet. */ +static bool take_string(const uint8_t *data, size_t len, size_t *pos, + char *out, size_t cap) +{ + size_t n; + + if (*pos >= len) { + return false; + } + n = data[(*pos)++]; + if (n >= cap || *pos + n > len) { + return false; + } + memcpy(out, &data[*pos], n); + out[n] = '\0'; + *pos += n; + /* A NUL inside would cut the string short where it is used. */ + return strlen(out) == n; +} + +/* --- Improv's commands --------------------------------------------------- */ + +static void wifi_settings(const uint8_t *data, size_t len, + const struct rpc_out *out) +{ + char ssid[33], password[65]; + size_t pos = 0; + int rc; + + if (!take_string(data, len, &pos, ssid, sizeof(ssid)) || ssid[0] == '\0' || + !take_string(data, len, &pos, password, sizeof(password)) || + pos != len) { + out->error(IMPROV_ERR_INVALID_RPC); + return; + } + if (hooks == NULL || hooks->wifi_connect == NULL) { + TEDGE_LOG_WRN("provisioning: Wi-Fi settings received, but this " + "application cannot join a network"); + out->error(IMPROV_ERR_UNABLE_CONNECT); + return; + } + + TEDGE_LOG_INF("provisioning: joining Wi-Fi \"%s\"", ssid); + set_state(IMPROV_STATE_PROVISIONING, out); + rc = hooks->wifi_connect(ssid, password, hooks->ctx); + memset(password, 0, sizeof(password)); + if (rc == 0) { + int64_t until = tedge_port_uptime_ms() + ADDRESS_WAIT_MS; + + while (!tedge_port_net_is_up() && tedge_port_uptime_ms() < until) { + tedge_port_sleep_ms(250); + } + if (!tedge_port_net_is_up()) { + rc = -ETIMEDOUT; + } + } + if (rc != 0) { + TEDGE_LOG_WRN("provisioning: could not join \"%s\" (%d)", ssid, rc); + out->error(IMPROV_ERR_UNABLE_CONNECT); + set_state(IMPROV_STATE_READY, out); + return; + } + TEDGE_LOG_INF("provisioning: joined \"%s\"", ssid); + set_state(IMPROV_STATE_PROVISIONED, out); + /* Improv's result after provisioning: URLs to open next. The device + * does not know its page in Cumulocity, so there are none. */ + send_empty_result(IMPROV_CMD_WIFI_SETTINGS, out); +} + +static void current_state(const struct rpc_out *out) +{ + if (improv_state != IMPROV_STATE_PROVISIONING) { + improv_state = tedge_port_net_is_up() ? IMPROV_STATE_PROVISIONED + : IMPROV_STATE_READY; + } + out->state(improv_state); + if (improv_state == IMPROV_STATE_PROVISIONED) { + send_empty_result(IMPROV_CMD_CURRENT_STATE, out); + } +} + +static void device_info(const struct rpc_out *out) +{ + char id[65]; + struct result r; + + if (tedge_get_device_id(id, sizeof(id)) < 0) { + id[0] = '\0'; + } + result_start(&r, IMPROV_CMD_DEVICE_INFO); + result_add(&r, tedge_port_firmware_name()); + result_add(&r, tedge_port_firmware_version()); + result_add(&r, (hooks != NULL && hooks->chip != NULL) ? hooks->chip + : "unknown"); + result_add(&r, id); + result_send(&r, out); +} + +/* --- tiny-edge's commands ------------------------------------------------ */ + +static const char *state_name(enum tedge_state state) +{ + switch (state) { + case TEDGE_STATE_STOPPED: + return "stopped"; + case TEDGE_STATE_WAITING_NETWORK: + return "waiting-network"; + case TEDGE_STATE_WAITING_TIME: + return "waiting-time"; + case TEDGE_STATE_AWAITING_REGISTRATION: + return "awaiting-registration"; + case TEDGE_STATE_CONNECTING: + return "connecting"; + case TEDGE_STATE_CONNECTED: + return "connected"; + case TEDGE_STATE_UPDATING: + return "updating"; + default: + return "unknown"; + } +} + +static void info(const struct rpc_out *out) +{ + char id[65], host[65]; + struct result r; + + if (tedge_get_device_id(id, sizeof(id)) < 0) { + id[0] = '\0'; + } + if (tedge_get_c8y_url(host, sizeof(host)) < 0) { + host[0] = '\0'; + } + result_start(&r, TEDGE_PROVISIONING_CMD_INFO); + result_add(&r, STR(TEDGE_PROVISIONING_PROTOCOL_VERSION)); + result_add(&r, id); + result_add(&r, TEDGE_IS_ENABLED(TEDGE_AUTH_C8Y_CA) ? "c8y-ca" : "bootstrap"); + result_add(&r, host); + result_add(&r, state_name(tedge_get_state())); + result_send(&r, out); +} + +static void onboard(const uint8_t *data, size_t len, const struct rpc_out *out) +{ + char host[65], otp[65]; + size_t pos = 1; + uint8_t flags; + int rc; + + if (!TEDGE_IS_ENABLED(TEDGE_AUTH_C8Y_CA)) { + /* Bootstrap devices have no one-time password to set. */ + out->error(IMPROV_ERR_UNKNOWN_RPC); + return; + } + if (len < 1 || !take_string(data, len, &pos, host, sizeof(host)) || + host[0] == '\0' || + !take_string(data, len, &pos, otp, sizeof(otp)) || otp[0] == '\0' || + pos != len) { + out->error(IMPROV_ERR_INVALID_RPC); + return; + } + flags = data[0]; + if (tedge_get_state() == TEDGE_STATE_CONNECTED && + !(flags & TEDGE_PROVISIONING_FLAG_REONBOARD)) { + TEDGE_LOG_WRN("provisioning: refused to set up a connected device " + "again without the re-onboard flag"); + memset(otp, 0, sizeof(otp)); + out->error(IMPROV_ERR_NOT_AUTHORIZED); + return; + } + + /* The client takes its tenant and password before it starts. */ + rc = tedge_stop(); + if (rc != 0 && rc != -EALREADY) { + TEDGE_LOG_ERR("provisioning: the client did not stop (%d)", rc); + memset(otp, 0, sizeof(otp)); + out->error(IMPROV_ERR_UNKNOWN); + return; + } + rc = tedge_set_c8y_url(host); + if (rc == 0) { + rc = tedge_set_enroll_otp(otp); + } + memset(otp, 0, sizeof(otp)); + if (rc != 0) { + TEDGE_LOG_WRN("provisioning: onboarding data refused (%d)", rc); + (void)tedge_start(); + out->error(IMPROV_ERR_INVALID_RPC); + return; + } + TEDGE_LOG_INF("provisioning: set up for %s", host); + rc = tedge_start(); + if (rc != 0 && rc != -EALREADY) { + TEDGE_LOG_ERR("provisioning: the client did not start (%d)", rc); + out->error(IMPROV_ERR_UNKNOWN); + return; + } + send_empty_result(TEDGE_PROVISIONING_CMD_ONBOARD, out); +} + +static void identify(const struct rpc_out *out) +{ + char id[65]; + + if (tedge_get_device_id(id, sizeof(id)) < 0) { + id[0] = '\0'; + } + TEDGE_LOG_INF("provisioning: this is \"%s\"", id); + if (hooks != NULL && hooks->identify != NULL) { + hooks->identify(hooks->ctx); + } + send_empty_result(TEDGE_PROVISIONING_CMD_IDENTIFY, out); +} + +/* --- Dispatch ------------------------------------------------------------ */ + +static void handle(const uint8_t *pkt, size_t n, const struct rpc_out *out) +{ + const uint8_t *data = pkt + 2; + size_t len; + + if (n < 2 || (size_t)pkt[1] + 2 != n) { + out->error(IMPROV_ERR_INVALID_RPC); + return; + } + len = pkt[1]; + switch (pkt[0]) { + case IMPROV_CMD_WIFI_SETTINGS: + wifi_settings(data, len, out); + break; + case IMPROV_CMD_CURRENT_STATE: + current_state(out); + break; + case IMPROV_CMD_DEVICE_INFO: + device_info(out); + break; + case TEDGE_PROVISIONING_CMD_INFO: + info(out); + break; + case TEDGE_PROVISIONING_CMD_ONBOARD: + onboard(data, len, out); + break; + case TEDGE_PROVISIONING_CMD_IDENTIFY: + identify(out); + break; + default: + out->error(IMPROV_ERR_UNKNOWN_RPC); + break; + } +} + +int tedge_prov_rpc_submit(const uint8_t *pkt, size_t len, + const struct rpc_out *out) +{ + struct rpc_msg msg; + + if (len > sizeof(msg.data)) { + len = 0; /* too long for any command: answered "invalid RPC" */ + } + msg.out = out; + msg.len = (uint8_t)len; + if (len > 0) { + memcpy(msg.data, pkt, len); + } + if (tedge_port_msgq_put(&tedge_prov_queue, &msg) != 0) { + return -ENOMEM; /* a tool sends one request at a time */ + } + tedge_port_event_post(&prov_events, EV_RPC); + return 0; +} + +static void worker(void *arg) +{ + struct rpc_msg msg; + + (void)arg; + for (;;) { + (void)tedge_port_event_wait(&prov_events, EV_RPC, 1000); + tedge_port_event_clear(&prov_events, EV_RPC); + while (tedge_port_msgq_get(&tedge_prov_queue, &msg) == 0) { + handle(msg.data, msg.len, msg.out); + } + } +} + +int tedge_provisioning_init(const struct tedge_provisioning_hooks *h) +{ + static bool started; + + if (started) { + return -EALREADY; + } + hooks = h; + started = true; + return tedge_port_thread_start(&prov_thread, prov_stack, + TEDGE_PORT_STACK_SIZEOF(prov_stack), worker, + NULL, "tedge_prov"); +} diff --git a/provisioning/src/tedge_provisioning_internal.h b/provisioning/src/tedge_provisioning_internal.h new file mode 100644 index 0000000..570665d --- /dev/null +++ b/provisioning/src/tedge_provisioning_internal.h @@ -0,0 +1,50 @@ +/* SPDX-License-Identifier: Apache-2.0 */ + +/* Shared between the provisioning library's sources; not public. */ + +#ifndef TEDGE_PROVISIONING_INTERNAL_H_ +#define TEDGE_PROVISIONING_INTERNAL_H_ + +#include + +#include +#include + +/* Improv's states, errors and commands (https://www.improv-wifi.com). */ +#define IMPROV_STATE_READY 0x02 /* "authorized" on BLE */ +#define IMPROV_STATE_PROVISIONING 0x03 +#define IMPROV_STATE_PROVISIONED 0x04 + +#define IMPROV_ERR_NONE 0x00 +#define IMPROV_ERR_INVALID_RPC 0x01 +#define IMPROV_ERR_UNKNOWN_RPC 0x02 +#define IMPROV_ERR_UNABLE_CONNECT 0x03 +#define IMPROV_ERR_NOT_AUTHORIZED 0x04 +#define IMPROV_ERR_UNKNOWN 0xFF + +#define IMPROV_CMD_WIFI_SETTINGS 0x01 +#define IMPROV_CMD_CURRENT_STATE 0x02 /* serial; "identify" on BLE */ +#define IMPROV_CMD_DEVICE_INFO 0x03 + +/* An RPC packet as the transports hand it over: [command][length][data]. + * The largest the library accepts: the onboard command with a 64-byte host + * and a 64-byte password. */ +#define RPC_MAX 160 + +/* How the RPC layer answers, whatever the transport. */ +struct rpc_out { + void (*state)(uint8_t state); + void (*error)(uint8_t error); + /* An RPC result: [command][length][strings, each length-prefixed]. */ + void (*result)(const uint8_t *data, size_t len); +}; + +/* Queue one RPC packet from @p out's transport, for the worker thread. + * @return 0, or -ENOMEM when the queue is full. */ +int tedge_prov_rpc_submit(const uint8_t *pkt, size_t len, + const struct rpc_out *out); + +/* The serial transport's answers (tedge_improv_serial.c). */ +extern const struct rpc_out tedge_prov_serial_out; + +#endif /* TEDGE_PROVISIONING_INTERNAL_H_ */ diff --git a/scripts/check-portability.py b/scripts/check-portability.py index 7cd0857..9bc015b 100755 --- a/scripts/check-portability.py +++ b/scripts/check-portability.py @@ -4,7 +4,8 @@ Rules: -1. core/, include/tedge/tedge.h and the port interface (include/tedge/port/) +1. core/, provisioning/, their public headers (include/tedge/tedge.h, + include/tedge/provisioning.h) and the port interface (include/tedge/port/) use no RTOS API: no RTOS or vendor SDK headers, no Kconfig symbols (CONFIG_*), and none of the Zephyr calls and macros the core was written with (k_*, LOG_*, ARRAY_SIZE, ...), which an RTOS header included by @@ -57,8 +58,8 @@ def is_code(path: Path) -> bool: def is_portable(rel: str) -> bool: - return rel.startswith(("core/", "include/tedge/port/")) \ - or rel == "include/tedge/tedge.h" + return rel.startswith(("core/", "provisioning/", "include/tedge/port/")) \ + or rel in ("include/tedge/tedge.h", "include/tedge/provisioning.h") def rtos_api(line: str) -> bool: diff --git a/tests/host-compile/include/tedge_config.h b/tests/host-compile/include/tedge_config.h index 6244fe3..44443a6 100644 --- a/tests/host-compile/include/tedge_config.h +++ b/tests/host-compile/include/tedge_config.h @@ -82,4 +82,8 @@ #define TEDGE_REMOTE_ACCESS_ALLOW_LIST "" #define TEDGE_REMOTE_ACCESS_TWIN_SESSIONS 1 +#define TEDGE_PROVISIONING 1 +#define TEDGE_PROVISIONING_SERIAL 1 +#define TEDGE_PROVISIONING_STACK_SIZE 4096 + #endif /* TEDGE_CONFIG_H_ */ diff --git a/tests/host-compile/run.sh b/tests/host-compile/run.sh index b266665..f8526b0 100755 --- a/tests/host-compile/run.sh +++ b/tests/host-compile/run.sh @@ -1,7 +1,7 @@ #!/usr/bin/env bash # SPDX-License-Identifier: Apache-2.0 # -# Compiles every file of the core with the host's C compiler and C library, +# Compiles every file of the core and the provisioning library with the host's C compiler and C library, # against a port that has only types (include/), with every feature on. # It proves the core needs nothing from an RTOS: a Zephyr header, macro or # libc extension it relied on would fail here even when the portability @@ -22,7 +22,7 @@ for dir in "$@"; do done failed=0 -for src in "$root"/core/src/*.c; do +for src in "$root"/core/src/*.c "$root"/provisioning/src/*.c; do if ! "$cc" -std=c11 -D_POSIX_C_SOURCE=200809L -Wall -Wextra \ -Wno-unused-parameter -Wno-sign-compare -Werror \ -DTEDGE_VERSION_STRING='"0.0.0"' \ @@ -32,5 +32,5 @@ for src in "$root"/core/src/*.c; do failed=1 fi done -[ $failed -eq 0 ] && echo "every core file compiles without an RTOS" +[ $failed -eq 0 ] && echo "every core and provisioning file compiles without an RTOS" exit $failed From 6a537b1cabb15ba199a5944f8dac20c67a0fe3d6 Mon Sep 17 00:00:00 2001 From: reubenmiller Date: Tue, 10 Nov 2026 15:21:42 +1000 Subject: [PATCH 05/14] feat(zephyr): Improv Serial on the console, and an onboarding sample With TEDGE_PROVISIONING, the provisioning library is built. TEDGE_PROVISIONING_SERIAL connects it to the console: - With the shell, "tedge improv" lends the console through the shell's bypass mode, until no valid frame has come for TEDGE_PROVISIONING_SERIAL_IDLE_S (60 s). Each frame is written raw, through the shell's transport, while holding the lock its log output takes, so a log line never lands inside a frame. - Without the shell, the console is polled from boot. The minimal sample's overlay-onboarding.conf builds it with the shell and Zephyr's Wi-Fi credentials store. The sample's Wi-Fi hook joins the network, stores it, and joins it again on the next boot. On an ESP32-S3-DevKitC, the library answers info, device info, and identify within milliseconds. A network that does not exist ends in "unable to connect". The library costs 5 to 6 KB of flash and about 5 KB of RAM, 4 KB of it the worker's stack. Twister builds the overlay for the ESP32-S3 and ESP32-C6. Two Kconfig cases cover the new options. Co-Authored-By: Claude Opus 5.5 --- ports/zephyr/CMakeLists.txt | 5 + ports/zephyr/Kconfig | 42 +++++ .../samples/minimal/overlay-onboarding.conf | 19 ++ ports/zephyr/samples/minimal/sample.yaml | 18 ++ ports/zephyr/samples/minimal/src/main.c | 92 +++++++++- ports/zephyr/src/tedge_provisioning_console.c | 162 ++++++++++++++++++ .../cases/provisioning-ble-experimental.conf | 7 + .../kconfig/cases/provisioning-defaults.conf | 8 + 8 files changed, 352 insertions(+), 1 deletion(-) create mode 100644 ports/zephyr/samples/minimal/overlay-onboarding.conf create mode 100644 ports/zephyr/src/tedge_provisioning_console.c create mode 100644 ports/zephyr/tests/kconfig/cases/provisioning-ble-experimental.conf create mode 100644 ports/zephyr/tests/kconfig/cases/provisioning-defaults.conf diff --git a/ports/zephyr/CMakeLists.txt b/ports/zephyr/CMakeLists.txt index 4ba3bca..335f9df 100644 --- a/ports/zephyr/CMakeLists.txt +++ b/ports/zephyr/CMakeLists.txt @@ -16,9 +16,12 @@ if(CONFIG_TEDGE) # The core's files for the features this build has (core/sources.cmake), # then the Zephyr side of the port interface. include(${TEDGE_ROOT}/core/sources.cmake) + include(${TEDGE_ROOT}/provisioning/sources.cmake) tedge_core_sources(_core_sources CONFIG_) + tedge_provisioning_sources(_provisioning_sources CONFIG_) zephyr_library_sources( ${_core_sources} + ${_provisioning_sources} src/tedge_platform.c src/tedge_port.c src/tedge_port_os.c @@ -32,6 +35,8 @@ if(CONFIG_TEDGE) zephyr_library_sources_ifdef(CONFIG_TEDGE_COREDUMP src/tedge_coredump.c) zephyr_library_sources_ifdef(CONFIG_SHELL src/tedge_shell.c) zephyr_library_sources_ifdef(CONFIG_TEDGE_AUTH_C8Y_CA src/tedge_port_crypto.c) + zephyr_library_sources_ifdef(CONFIG_TEDGE_PROVISIONING_SERIAL + src/tedge_provisioning_console.c) zephyr_library_compile_definitions(TEDGE_VERSION_STRING="${TEDGE_VERSION}") # gmtime_r() is POSIX, not C17. picolibc declares it anyway; a host libc # (native_sim) only does when asked, and an undeclared gmtime_r() returns diff --git a/ports/zephyr/Kconfig b/ports/zephyr/Kconfig index 35a2cdb..8271875 100644 --- a/ports/zephyr/Kconfig +++ b/ports/zephyr/Kconfig @@ -821,6 +821,48 @@ config TEDGE_TLS_TAG_BASE endmenu +menuconfig TEDGE_PROVISIONING + bool "Provisioning library: set the device up over Improv Wi-Fi" + help + An optional library beside the client (include/tedge/provisioning.h) + that answers the Improv Wi-Fi protocol, as tedge-onboard, ESP Web + Tools and Improv phone apps speak it. It hands a Wi-Fi network to + the application's hook, and the Cumulocity tenant and one-time + password to the client. The application calls + tedge_provisioning_init() after tedge_init(). + +if TEDGE_PROVISIONING + +config TEDGE_PROVISIONING_SERIAL + bool "Improv Serial on the console" + default y + help + With the shell, "tedge improv" hands the console to a provisioning + tool until it has been quiet for TEDGE_PROVISIONING_SERIAL_IDLE_S. + Without the shell, the console speaks Improv from boot, beside the + log output. + +config TEDGE_PROVISIONING_SERIAL_IDLE_S + int "Seconds without a valid frame before the shell gets the console back" + default 60 + depends on TEDGE_PROVISIONING_SERIAL && SHELL + +config TEDGE_PROVISIONING_BLE + bool "Improv over Bluetooth LE (not implemented yet)" + depends on TEDGE_EXPERIMENTAL_FEATURES + help + Phase 2 of the serial-onboarding change: the Improv GATT service on + the same RPC layer. + +config TEDGE_PROVISIONING_STACK_SIZE + int "Stack of the provisioning worker thread" + default 4096 + help + The worker answers each request; it calls the application's Wi-Fi + hook and stops and starts the client. + +endif # TEDGE_PROVISIONING + module = TEDGE module-str = tedge source "subsys/logging/Kconfig.template.log_config" diff --git a/ports/zephyr/samples/minimal/overlay-onboarding.conf b/ports/zephyr/samples/minimal/overlay-onboarding.conf new file mode 100644 index 0000000..85539eb --- /dev/null +++ b/ports/zephyr/samples/minimal/overlay-onboarding.conf @@ -0,0 +1,19 @@ +# SPDX-License-Identifier: Apache-2.0 +# +# Set the device up with a tool over the console (tedge-onboard): the +# provisioning library speaks Improv Serial after "tedge improv", the +# sample joins and stores the Wi-Fi network it is given, and the client +# enrolls with the tenant and one-time password it is given. Nothing +# about the network or the tenant is built in. +# +# west build -b esp32s3_devkitc/esp32s3/procpu ports/zephyr/samples/minimal -- \ +# -DEXTRA_CONF_FILE="$PWD/ports/zephyr/profiles/minimal.conf;\ +# $PWD/ports/zephyr/profiles/boards/esp32s3-devkitc.conf;\ +# $PWD/ports/zephyr/samples/minimal/overlay-onboarding.conf" + +CONFIG_TEDGE_PROVISIONING=y + +CONFIG_SHELL=y +CONFIG_WIFI_CREDENTIALS=y +CONFIG_WIFI_CREDENTIALS_BACKEND_SETTINGS=y +CONFIG_WIFI_CREDENTIALS_CONNECT_STORED=y diff --git a/ports/zephyr/samples/minimal/sample.yaml b/ports/zephyr/samples/minimal/sample.yaml index 5806ae1..a4d4146 100644 --- a/ports/zephyr/samples/minimal/sample.yaml +++ b/ports/zephyr/samples/minimal/sample.yaml @@ -86,3 +86,21 @@ tests: - prj.conf - ../../profiles/full.conf - ../../profiles/boards/esp32s3-devkitc.conf + sample.tedge.minimal.esp32s3.onboarding: + build_only: true + platform_allow: esp32s3_devkitc/esp32s3/procpu + integration_platforms: [esp32s3_devkitc/esp32s3/procpu] + extra_conf_files: + - prj.conf + - ../../profiles/minimal.conf + - ../../profiles/boards/esp32s3-devkitc.conf + - overlay-onboarding.conf + sample.tedge.minimal.esp32c6.onboarding: + build_only: true + platform_allow: esp32c6_devkitc/esp32c6/hpcore + integration_platforms: [esp32c6_devkitc/esp32c6/hpcore] + extra_conf_files: + - prj.conf + - ../../profiles/minimal.conf + - ../../profiles/boards/esp32c6-devkitc.conf + - overlay-onboarding.conf diff --git a/ports/zephyr/samples/minimal/src/main.c b/ports/zephyr/samples/minimal/src/main.c index 507de0a..d09cf5b 100644 --- a/ports/zephyr/samples/minimal/src/main.c +++ b/ports/zephyr/samples/minimal/src/main.c @@ -14,6 +14,12 @@ #include +#if defined(CONFIG_TEDGE_PROVISIONING) +#include +#include +#include +#endif + LOG_MODULE_REGISTER(tedge_minimal, LOG_LEVEL_INF); static void on_state(enum tedge_state state, void *user_data) @@ -61,17 +67,101 @@ static int wifi_connect(void) } #endif +#if defined(CONFIG_TEDGE_PROVISIONING) +/* Setting the device up with a tool (tedge-onboard) over the console: the + * library hands this application the Wi-Fi network to join, and the client + * the Cumulocity tenant. The network is stored with Zephyr's Wi-Fi + * credentials, so the next boot joins it by itself. */ + +static K_SEM_DEFINE(wifi_done, 0, 1); +static int wifi_status; +static struct net_mgmt_event_callback wifi_cb; + +static void wifi_event(struct net_mgmt_event_callback *cb, uint64_t event, + struct net_if *iface) +{ + ARG_UNUSED(iface); + if (event == NET_EVENT_WIFI_CONNECT_RESULT) { + const struct wifi_status *status = cb->info; + + wifi_status = status->status; + k_sem_give(&wifi_done); + } +} + +static int provision_wifi(const char *ssid, const char *password, void *ctx) +{ + struct net_if *iface = net_if_get_first_wifi(); + enum wifi_security_type security = password[0] != '\0' + ? WIFI_SECURITY_TYPE_PSK + : WIFI_SECURITY_TYPE_NONE; + struct wifi_connect_req_params p = { + .ssid = (const uint8_t *)ssid, + .ssid_length = strlen(ssid), + .psk = (const uint8_t *)password, + .psk_length = strlen(password), + .security = security, + .channel = WIFI_CHANNEL_ANY, + .band = WIFI_FREQ_BAND_2_4_GHZ, + }; + int rc; + + ARG_UNUSED(ctx); + if (iface == NULL) { + return -ENODEV; + } + (void)net_mgmt(NET_REQUEST_WIFI_DISCONNECT, iface, NULL, 0); + k_sem_reset(&wifi_done); + rc = net_mgmt(NET_REQUEST_WIFI_CONNECT, iface, &p, sizeof(p)); + if (rc != 0) { + return rc; + } + if (k_sem_take(&wifi_done, K_SECONDS(30)) != 0) { + return -ETIMEDOUT; + } + if (wifi_status != 0) { + return -ECONNREFUSED; + } + /* Joined: keep it, and only it, for the next boot. */ + (void)wifi_credentials_delete_all(); + return wifi_credentials_set_personal(ssid, strlen(ssid), security, NULL, 0, + password, strlen(password), 0, 0, 0); +} + +static const struct tedge_provisioning_hooks provisioning_hooks = { + .wifi_connect = provision_wifi, + .chip = CONFIG_SOC, +}; +#endif + int main(void) { LOG_INF("tedge-zephyr %s", tedge_version()); -#if defined(CONFIG_WIFI) +#if defined(CONFIG_TEDGE_PROVISIONING) + net_mgmt_init_event_callback(&wifi_cb, wifi_event, + NET_EVENT_WIFI_CONNECT_RESULT); + net_mgmt_add_event_callback(&wifi_cb); + if (!wifi_credentials_is_empty()) { + /* Set up before: join the network it was given. */ + (void)net_mgmt(NET_REQUEST_WIFI_CONNECT_STORED, + net_if_get_first_wifi(), NULL, 0); + } else if (strlen(CONFIG_TEDGE_SAMPLE_WIFI_SSID) > 0) { + (void)wifi_connect(); + } else { + LOG_INF("no Wi-Fi network yet: set the device up with " + "tedge-onboard"); + } +#elif defined(CONFIG_WIFI) /* The client waits for an address by itself, so a failure here only * delays it: the application can retry however it likes. */ (void)wifi_connect(); #endif if (tedge_init(NULL, &hooks) == 0) { +#if defined(CONFIG_TEDGE_PROVISIONING) + (void)tedge_provisioning_init(&provisioning_hooks); +#endif (void)tedge_start(); } return 0; diff --git a/ports/zephyr/src/tedge_provisioning_console.c b/ports/zephyr/src/tedge_provisioning_console.c new file mode 100644 index 0000000..65f4041 --- /dev/null +++ b/ports/zephyr/src/tedge_provisioning_console.c @@ -0,0 +1,162 @@ +/* SPDX-License-Identifier: Apache-2.0 + * + * Improv Serial on the Zephyr console, for the provisioning library + * (include/tedge/provisioning.h). + * + * With the shell, the console is the shell's, and "tedge improv" lends it + * to a provisioning tool: the shell's bypass mode passes every received + * byte to the library, until no valid frame has arrived for + * CONFIG_TEDGE_PROVISIONING_SERIAL_IDLE_S. Frames go out through the + * shell's own transport, holding the lock its log output takes, so a log + * line never lands inside a frame, and unchanged (the shell's printing + * would turn a 0x0A byte into CR LF). + * + * Without the shell, the console speaks Improv from boot: its UART is read + * by polling, which every UART driver supports, and a frame is written + * with the scheduler locked, so the log thread cannot cut in. + */ + +#include + +#include +#include +#include +#include +#include + +#if defined(CONFIG_SHELL) +#include +#endif + +LOG_MODULE_DECLARE(tedge, CONFIG_TEDGE_LOG_LEVEL); + +#if defined(CONFIG_SHELL) + +static const struct shell *lent; /* the shell whose console is lent */ + +static void give_back(struct k_work *work); +static K_WORK_DELAYABLE_DEFINE(idle_work, give_back); + +static void shell_write(const void *data, size_t len, void *ctx) +{ + const struct shell *sh = ctx; + const uint8_t *p = data; + int spins = 0; + + (void)k_sem_take(&sh->ctx->lock_sem, K_FOREVER); + while (len > 0) { + size_t n = 0; + + if (sh->iface->api->write(sh->iface, p, len, &n) != 0) { + break; + } + if (n == 0) { + /* The transport's buffer is full: let it drain, for + * a second at most. */ + if (++spins > 1000) { + break; + } + k_msleep(1); + continue; + } + p += n; + len -= n; + } + k_sem_give(&sh->ctx->lock_sem); +} + +/* Runs on the shell's thread, with every byte the console received. */ +static void bypass(const struct shell *sh, uint8_t *data, size_t len, + void *user_data) +{ + ARG_UNUSED(sh); + ARG_UNUSED(user_data); + if (tedge_provisioning_serial_feed(data, len) > 0) { + (void)k_work_reschedule(&idle_work, + K_SECONDS(CONFIG_TEDGE_PROVISIONING_SERIAL_IDLE_S)); + } +} + +static void give_back(struct k_work *work) +{ + const struct shell *sh = lent; + + ARG_UNUSED(work); + if (sh == NULL) { + return; + } + lent = NULL; + tedge_provisioning_serial_attach(NULL, NULL); + shell_set_bypass(sh, NULL, NULL); + LOG_INF("provisioning: the console is the shell's again"); +} + +static int cmd_improv(const struct shell *sh, size_t argc, char **argv) +{ + ARG_UNUSED(argc); + ARG_UNUSED(argv); + shell_print(sh, "The console speaks Improv now. It returns to the shell " + "after %d s without a request.", + CONFIG_TEDGE_PROVISIONING_SERIAL_IDLE_S); + lent = sh; + tedge_provisioning_serial_attach(shell_write, (void *)sh); + shell_set_bypass(sh, bypass, NULL); + (void)k_work_reschedule(&idle_work, + K_SECONDS(CONFIG_TEDGE_PROVISIONING_SERIAL_IDLE_S)); + return 0; +} + +SHELL_SUBCMD_ADD((tedge), improv, NULL, + "Lend the console to a setup tool such as tedge-onboard", + cmd_improv, 1, 0); + +#else /* !CONFIG_SHELL */ + +#define POLL_MS 20 + +static const struct device *const console = DEVICE_DT_GET(DT_CHOSEN(zephyr_console)); + +static void uart_write(const void *data, size_t len, void *ctx) +{ + const uint8_t *p = data; + + ARG_UNUSED(ctx); + k_sched_lock(); + for (size_t i = 0; i < len; i++) { + uart_poll_out(console, p[i]); + } + k_sched_unlock(); +} + +static void poll_rx(struct k_work *work); +static K_WORK_DELAYABLE_DEFINE(rx_work, poll_rx); + +static void poll_rx(struct k_work *work) +{ + uint8_t buf[64]; + size_t n = 0; + unsigned char c; + + ARG_UNUSED(work); + while (n < sizeof(buf) && uart_poll_in(console, &c) == 0) { + buf[n++] = c; + } + if (n > 0) { + (void)tedge_provisioning_serial_feed(buf, n); + } + (void)k_work_schedule(&rx_work, K_MSEC(n == sizeof(buf) ? 0 : POLL_MS)); +} + +static int console_init(void) +{ + if (!device_is_ready(console)) { + return -ENODEV; + } + tedge_provisioning_serial_attach(uart_write, NULL); + (void)k_work_schedule(&rx_work, K_MSEC(POLL_MS)); + return 0; +} + +SYS_INIT(console_init, APPLICATION, 99); + +#endif /* CONFIG_SHELL */ diff --git a/ports/zephyr/tests/kconfig/cases/provisioning-ble-experimental.conf b/ports/zephyr/tests/kconfig/cases/provisioning-ble-experimental.conf new file mode 100644 index 0000000..e947cd6 --- /dev/null +++ b/ports/zephyr/tests/kconfig/cases/provisioning-ble-experimental.conf @@ -0,0 +1,7 @@ +# Improv over BLE is not implemented yet: only experimental builds may ask +# for it, and others get a warning and no BLE. +# expect: CONFIG_TEDGE_PROVISIONING=y +# expect: CONFIG_TEDGE_PROVISIONING_BLE=n +# expect-warning: TEDGE_PROVISIONING_BLE +CONFIG_TEDGE_PROVISIONING=y +CONFIG_TEDGE_PROVISIONING_BLE=y diff --git a/ports/zephyr/tests/kconfig/cases/provisioning-defaults.conf b/ports/zephyr/tests/kconfig/cases/provisioning-defaults.conf new file mode 100644 index 0000000..b0fd2bc --- /dev/null +++ b/ports/zephyr/tests/kconfig/cases/provisioning-defaults.conf @@ -0,0 +1,8 @@ +# The provisioning library is off unless asked for. Turned on, it brings +# Improv Serial on the console and its worker; BLE stays off (phase 2, +# experimental only). +# expect: CONFIG_TEDGE_PROVISIONING=y +# expect: CONFIG_TEDGE_PROVISIONING_SERIAL=y +# expect: CONFIG_TEDGE_PROVISIONING_STACK_SIZE=4096 +# expect: CONFIG_TEDGE_PROVISIONING_BLE=n +CONFIG_TEDGE_PROVISIONING=y From 7637e064728fd9fc7322a760f9bee07c3d11a889 Mon Sep 17 00:00:00 2001 From: reubenmiller Date: Tue, 10 Nov 2026 15:22:12 +1000 Subject: [PATCH 06/14] feat(onboard): tedge-onboard, set a device up from a computer A guided tool for people who are not technical. Run with no arguments, it: - checks the go-c8y-cli login, and offers go-c8y-cli's own session picker when there is none; - finds the board by asking each USB serial port of a known vendor, opened without resetting it, and offers a list (with "make one blink") when there are several; - proposes the computer's Wi-Fi network and reads its password from the Keychain, NetworkManager, or the WLAN profiles, asking only when that fails; - asks for the device's name and one confirmation, then registers the device through go-c8y-cli with a fresh one-time password, gives the board its Wi-Fi and Cumulocity details over Improv Serial, and waits until the device is connected. Every failure says what to do next and offers to retry. With --yes, questions become errors that name the option that answers them, for scripts. It builds for macOS, Linux, and Windows; macOS needs cgo for the USB port list. Go tests cover the codec, the client against a fake device, the Wi-Fi parsing per operating system, and the go-c8y-cli wrapper. The hardware tag runs checks against a real board. The repository ignored all of /tools/, for the west modules a workspace may put there; tools/onboard is now excepted. Co-Authored-By: Claude Opus 5.5 --- .gitignore | 3 +- README.md | 3 + tools/onboard/.gitignore | 1 + tools/onboard/README.md | 129 ++++ tools/onboard/build.sh | 36 ++ tools/onboard/go.mod | 42 ++ tools/onboard/go.sum | 94 +++ tools/onboard/internal/cumulocity/c8y.go | 262 ++++++++ tools/onboard/internal/cumulocity/c8y_test.go | 90 +++ tools/onboard/internal/device/device.go | 180 ++++++ tools/onboard/internal/device/device_test.go | 28 + .../onboard/internal/device/hardware_test.go | 77 +++ tools/onboard/internal/improv/client.go | 237 ++++++++ tools/onboard/internal/improv/client_test.go | 84 +++ tools/onboard/internal/improv/codec.go | 191 ++++++ tools/onboard/internal/improv/codec_test.go | 80 +++ tools/onboard/internal/ui/ui.go | 210 +++++++ tools/onboard/internal/wifi/wifi.go | 202 ++++++ tools/onboard/internal/wifi/wifi_test.go | 82 +++ tools/onboard/main.go | 573 ++++++++++++++++++ tools/onboard/pause_other.go | 9 + tools/onboard/pause_windows.go | 31 + 22 files changed, 2643 insertions(+), 1 deletion(-) create mode 100644 tools/onboard/.gitignore create mode 100644 tools/onboard/README.md create mode 100755 tools/onboard/build.sh create mode 100644 tools/onboard/go.mod create mode 100644 tools/onboard/go.sum create mode 100644 tools/onboard/internal/cumulocity/c8y.go create mode 100644 tools/onboard/internal/cumulocity/c8y_test.go create mode 100644 tools/onboard/internal/device/device.go create mode 100644 tools/onboard/internal/device/device_test.go create mode 100644 tools/onboard/internal/device/hardware_test.go create mode 100644 tools/onboard/internal/improv/client.go create mode 100644 tools/onboard/internal/improv/client_test.go create mode 100644 tools/onboard/internal/improv/codec.go create mode 100644 tools/onboard/internal/improv/codec_test.go create mode 100644 tools/onboard/internal/ui/ui.go create mode 100644 tools/onboard/internal/wifi/wifi.go create mode 100644 tools/onboard/internal/wifi/wifi_test.go create mode 100644 tools/onboard/main.go create mode 100644 tools/onboard/pause_other.go create mode 100644 tools/onboard/pause_windows.go diff --git a/.gitignore b/.gitignore index c1c8d7d..801fd32 100644 --- a/.gitignore +++ b/.gitignore @@ -11,7 +11,8 @@ build*.log # /zephyr/: that directory holds this repository's module.yml. /modules/ /bootloader/ -/tools/ +/tools/* +!/tools/onboard/ /.west/ # Local overlays and credentials: never commit secrets. diff --git a/README.md b/README.md index 505b1b1..7651226 100644 --- a/README.md +++ b/README.md @@ -34,6 +34,9 @@ tiny-edge/ │ ├── zephyr/ build glue, Kconfig, profiles, samples, tests, and the │ │ Zephyr-only sources │ └── posix/ CMake library, options, sample, and tests +├── provisioning/ optional library: set a device up over Improv Wi-Fi +│ (include/tedge/provisioning.h) +├── tools/onboard/ tedge-onboard: set a device up from a computer ├── certs/ default trust anchors ├── cumulocity/ tenant setup: Smart Functions and DTM definitions ├── openspec/ specifications and change history diff --git a/tools/onboard/.gitignore b/tools/onboard/.gitignore new file mode 100644 index 0000000..849ddff --- /dev/null +++ b/tools/onboard/.gitignore @@ -0,0 +1 @@ +dist/ diff --git a/tools/onboard/README.md b/tools/onboard/README.md new file mode 100644 index 0000000..083be3f --- /dev/null +++ b/tools/onboard/README.md @@ -0,0 +1,129 @@ +# tedge-onboard + +Sets up a device for Cumulocity over a USB cable. It puts the device on +your Wi-Fi network, registers it in Cumulocity, and checks that it is +connected. It takes about a minute, and you don't need to type anything +technical. + +## Before you start + +You need: + +- **The device**, with firmware that supports this tool. If you are not + sure, try it: the tool tells you if the firmware cannot be set up this + way. +- **A USB data cable.** Some cables only charge. If the tool cannot find + your device, try another cable. +- **go-c8y-cli**, the Cumulocity command-line tool, logged in to your + Cumulocity tenant. If it is not installed, the tool tells you how to + install it. If it is not logged in, the tool helps you log in. +- **This computer on the Wi-Fi network** the device should use. Most + devices only use 2.4 GHz networks. + +## Download + +Download the file for your computer from the +[tiny-edge releases](https://github.com/thin-edge/tiny-edge/releases) +(look under "Assets"): + +| Computer | File | +|---|---| +| Mac with Apple silicon (M1 and later) | `tedge-onboard__darwin_arm64.tar.gz` | +| Mac with an Intel processor | `tedge-onboard__darwin_amd64.tar.gz` | +| Windows | `tedge-onboard__windows_amd64.zip` | +| Linux | `tedge-onboard__linux_amd64.tar.gz` | + +Open the downloaded file to unpack it. + +**On a Mac**, the first time you start it, macOS may say it "cannot be +opened because the developer cannot be verified". Open **System Settings**, +then **Privacy & Security**, and select **Open Anyway** next to the +message about `tedge-onboard`. + +## Set up a device + +1. Plug the device into your computer with the USB cable. +2. Start `tedge-onboard`: + - **Mac or Linux:** open Terminal, type `./tedge-onboard` in the + folder where you unpacked it, and press Enter. + - **Windows:** double-click `tedge-onboard.exe`. +3. Answer the questions. Each one has the most likely answer already + selected, so you can usually just press Enter. Use the arrow keys to + choose something else. + +The tool: + +1. checks your Cumulocity login; +2. finds the device (if several are plugged in, it lets you choose, and + can make one blink so you can tell them apart); +3. offers this computer's Wi-Fi network; +4. asks for the device's name in Cumulocity; +5. shows a summary and asks you to confirm; +6. sets the device up, and shows a link to it in Cumulocity. + +At the end you can set up another device. + +## What the tool reads from your computer + +- **The Wi-Fi network and its password.** The tool reads the password + your computer already has stored, so you don't have to type it. On a + Mac, macOS asks you to allow this: enter your Mac's password and select + **Allow**. If you would rather type the Wi-Fi password, select **Deny**, + and the tool asks you for it. The password goes only to the device, + over the cable. The tool never shows it or saves it. +- **Your Cumulocity login, through go-c8y-cli.** The tool has no login of + its own and stores no Cumulocity password. + +The tool registers the device in Cumulocity with a new one-time password, +which the device uses once to get its certificate. + +## If something goes wrong + +| What you see | What to do | +|---|---| +| No device was found | Check the cable is a data cable, and plug it in again. Try another USB port. | +| The firmware cannot be set up this way | The device needs firmware with tiny-edge's provisioning library. | +| The device could not join the Wi-Fi network | Check the password. Make sure the network is 2.4 GHz and the device is in range. | +| The device did not connect in time | Check that the Wi-Fi network reaches the internet. The device keeps trying, so it may still appear in Cumulocity a few minutes later. | +| go-c8y-cli is not logged in | Choose **Choose or create a Cumulocity login** when the tool asks. | +| The device is already set up | Choose **Yes** to set it up again. It gets a new certificate, and keeps its history in Cumulocity. | + +## For scripts + +With `--yes`, the tool asks no questions and fails where it would have +asked one, saying which option answers it. + +```sh +tedge-onboard --yes --port /dev/ttyUSB0 --wifi-ssid lab-net --wifi-password-stdin < pw.txt +tedge-onboard --yes --skip-wifi --reonboard --json +``` + +| Option | | +|---|---| +| `--port` | The device's serial port (found automatically otherwise) | +| `--wifi-ssid` | The Wi-Fi network (default: this computer's) | +| `--wifi-password-stdin` | Read the Wi-Fi password from standard input | +| `--skip-wifi` | Leave the device's Wi-Fi as it is | +| `--name` | The device's name in Cumulocity (default: its ID) | +| `--reonboard` | Set up a device that is already in Cumulocity again | +| `--timeout` | How long to wait for the device to connect (default 3m) | +| `--json` | Print the result as JSON | +| `--yes` | Ask no questions | + +`ACCESSIBLE=1` switches the questions to plain prompts that work with +screen readers. + +## How it works + +The tool speaks [Improv Wi-Fi](https://www.improv-wifi.com) to the device +over its serial console, plus a few tiny-edge commands for the Cumulocity +details. On the device, tiny-edge's provisioning library answers them +(`include/tedge/provisioning.h`). The design is in +`openspec/changes/serial-onboarding/`. + +Checks for a real board: + +```sh +go test -tags hardware ./internal/device -run Hardware -v +TEDGE_ONBOARD_WIFI_TEST=1 go test -tags hardware ./internal/device -run HardwareWiFi -v +``` diff --git a/tools/onboard/build.sh b/tools/onboard/build.sh new file mode 100755 index 0000000..2062a46 --- /dev/null +++ b/tools/onboard/build.sh @@ -0,0 +1,36 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: Apache-2.0 +# +# Build tedge-onboard for each platform into dist/, one archive each: +# +# tools/onboard/build.sh linux/amd64 windows/arm64 ... +# +# macOS needs cgo (the USB port list comes from IOKit), so build darwin/* +# on macOS. Linux and Windows build anywhere, without cgo. +set -euo pipefail +cd "$(dirname "$0")" +version=$1 +shift +mkdir -p dist +for target in "$@"; do + goos=${target%/*} + goarch=${target#*/} + name=tedge-onboard + [ "$goos" = windows ] && name=tedge-onboard.exe + cgo=0 + [ "$goos" = darwin ] && cgo=1 + out=dist/tedge-onboard_${version}_${goos}_${goarch} + rm -rf "$out" + mkdir -p "$out" + CGO_ENABLED=$cgo GOOS=$goos GOARCH=$goarch \ + go build -trimpath -ldflags "-s -w -X main.version=$version" \ + -o "$out/$name" . + cp README.md "$out/" + if [ "$goos" = windows ]; then + (cd dist && zip -qr "$(basename "$out").zip" "$(basename "$out")") + else + tar -C dist -czf "$out.tar.gz" "$(basename "$out")" + fi + rm -rf "$out" + echo "built $out" +done diff --git a/tools/onboard/go.mod b/tools/onboard/go.mod new file mode 100644 index 0000000..d30bc2c --- /dev/null +++ b/tools/onboard/go.mod @@ -0,0 +1,42 @@ +module github.com/thin-edge/tiny-edge/tools/onboard + +go 1.26.0 + +require ( + github.com/charmbracelet/huh v1.0.0 + github.com/charmbracelet/huh/spinner v0.0.0-20260223110133-9dc45e34a40b + github.com/charmbracelet/lipgloss v1.1.0 + go.bug.st/serial v1.6.4 + golang.org/x/sys v0.50.0 + golang.org/x/term v0.48.0 +) + +require ( + github.com/atotto/clipboard v0.1.4 // indirect + github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect + github.com/catppuccin/go v0.3.0 // indirect + github.com/charmbracelet/bubbles v1.0.0 // indirect + github.com/charmbracelet/bubbletea v1.3.10 // indirect + github.com/charmbracelet/colorprofile v0.4.1 // indirect + github.com/charmbracelet/x/ansi v0.11.6 // indirect + github.com/charmbracelet/x/cellbuf v0.0.15 // indirect + github.com/charmbracelet/x/exp/strings v0.0.0-20240722160745-212f7b056ed0 // indirect + github.com/charmbracelet/x/term v0.2.2 // indirect + github.com/clipperhouse/displaywidth v0.9.0 // indirect + github.com/clipperhouse/stringish v0.1.1 // indirect + github.com/clipperhouse/uax29/v2 v2.5.0 // indirect + github.com/creack/goselect v0.1.2 // indirect + github.com/dustin/go-humanize v1.0.1 // indirect + github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f // indirect + github.com/lucasb-eyer/go-colorful v1.3.0 // indirect + github.com/mattn/go-isatty v0.0.20 // indirect + github.com/mattn/go-localereader v0.0.1 // indirect + github.com/mattn/go-runewidth v0.0.19 // indirect + github.com/mitchellh/hashstructure/v2 v2.0.2 // indirect + github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 // indirect + github.com/muesli/cancelreader v0.2.2 // indirect + github.com/muesli/termenv v0.16.0 // indirect + github.com/rivo/uniseg v0.4.7 // indirect + github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect + golang.org/x/text v0.23.0 // indirect +) diff --git a/tools/onboard/go.sum b/tools/onboard/go.sum new file mode 100644 index 0000000..ee9a623 --- /dev/null +++ b/tools/onboard/go.sum @@ -0,0 +1,94 @@ +github.com/MakeNowJust/heredoc v1.0.0 h1:cXCdzVdstXyiTqTvfqk9SDHpKNjxuom+DOlyEeQ4pzQ= +github.com/MakeNowJust/heredoc v1.0.0/go.mod h1:mG5amYoWBHf8vpLOuehzbGGw0EHxpZZ6lCpQ4fNJ8LE= +github.com/atotto/clipboard v0.1.4 h1:EH0zSVneZPSuFR11BlR9YppQTVDbh5+16AmcJi4g1z4= +github.com/atotto/clipboard v0.1.4/go.mod h1:ZY9tmq7sm5xIbd9bOK4onWV4S6X0u6GY7Vn0Yu86PYI= +github.com/aymanbagabas/go-osc52/v2 v2.0.1 h1:HwpRHbFMcZLEVr42D4p7XBqjyuxQH5SMiErDT4WkJ2k= +github.com/aymanbagabas/go-osc52/v2 v2.0.1/go.mod h1:uYgXzlJ7ZpABp8OJ+exZzJJhRNQ2ASbcXHWsFqH8hp8= +github.com/aymanbagabas/go-udiff v0.3.1 h1:LV+qyBQ2pqe0u42ZsUEtPiCaUoqgA9gYRDs3vj1nolY= +github.com/aymanbagabas/go-udiff v0.3.1/go.mod h1:G0fsKmG+P6ylD0r6N/KgQD/nWzgfnl8ZBcNLgcbrw8E= +github.com/catppuccin/go v0.3.0 h1:d+0/YicIq+hSTo5oPuRi5kOpqkVA5tAsU6dNhvRu+aY= +github.com/catppuccin/go v0.3.0/go.mod h1:8IHJuMGaUUjQM82qBrGNBv7LFq6JI3NnQCF6MOlZjpc= +github.com/charmbracelet/bubbles v1.0.0 h1:12J8/ak/uCZEMQ6KU7pcfwceyjLlWsDLAxB5fXonfvc= +github.com/charmbracelet/bubbles v1.0.0/go.mod h1:9d/Zd5GdnauMI5ivUIVisuEm3ave1XwXtD1ckyV6r3E= +github.com/charmbracelet/bubbletea v1.3.10 h1:otUDHWMMzQSB0Pkc87rm691KZ3SWa4KUlvF9nRvCICw= +github.com/charmbracelet/bubbletea v1.3.10/go.mod h1:ORQfo0fk8U+po9VaNvnV95UPWA1BitP1E0N6xJPlHr4= +github.com/charmbracelet/colorprofile v0.4.1 h1:a1lO03qTrSIRaK8c3JRxJDZOvhvIeSco3ej+ngLk1kk= +github.com/charmbracelet/colorprofile v0.4.1/go.mod h1:U1d9Dljmdf9DLegaJ0nGZNJvoXAhayhmidOdcBwAvKk= +github.com/charmbracelet/huh v1.0.0 h1:wOnedH8G4qzJbmhftTqrpppyqHakl/zbbNdXIWJyIxw= +github.com/charmbracelet/huh v1.0.0/go.mod h1:5YVc+SlZ1IhQALxRPpkGwwEKftN/+OlJlnJYlDRFqN4= +github.com/charmbracelet/huh/spinner v0.0.0-20260223110133-9dc45e34a40b h1:deQbW7eR/gYwkXonGX6a1now6H6f8v4kfv0OIKECu0I= +github.com/charmbracelet/huh/spinner v0.0.0-20260223110133-9dc45e34a40b/go.mod h1:Y68nuKJuC/Q2lmiq18EkHWkVWi2VGLrwaOfOyPKLkkE= +github.com/charmbracelet/lipgloss v1.1.0 h1:vYXsiLHVkK7fp74RkV7b2kq9+zDLoEU4MZoFqR/noCY= +github.com/charmbracelet/lipgloss v1.1.0/go.mod h1:/6Q8FR2o+kj8rz4Dq0zQc3vYf7X+B0binUUBwA0aL30= +github.com/charmbracelet/x/ansi v0.11.6 h1:GhV21SiDz/45W9AnV2R61xZMRri5NlLnl6CVF7ihZW8= +github.com/charmbracelet/x/ansi v0.11.6/go.mod h1:2JNYLgQUsyqaiLovhU2Rv/pb8r6ydXKS3NIttu3VGZQ= +github.com/charmbracelet/x/cellbuf v0.0.15 h1:ur3pZy0o6z/R7EylET877CBxaiE1Sp1GMxoFPAIztPI= +github.com/charmbracelet/x/cellbuf v0.0.15/go.mod h1:J1YVbR7MUuEGIFPCaaZ96KDl5NoS0DAWkskup+mOY+Q= +github.com/charmbracelet/x/conpty v0.1.0 h1:4zc8KaIcbiL4mghEON8D72agYtSeIgq8FSThSPQIb+U= +github.com/charmbracelet/x/conpty v0.1.0/go.mod h1:rMFsDJoDwVmiYM10aD4bH2XiRgwI7NYJtQgl5yskjEQ= +github.com/charmbracelet/x/errors v0.0.0-20240508181413-e8d8b6e2de86 h1:JSt3B+U9iqk37QUU2Rvb6DSBYRLtWqFqfxf8l5hOZUA= +github.com/charmbracelet/x/errors v0.0.0-20240508181413-e8d8b6e2de86/go.mod h1:2P0UgXMEa6TsToMSuFqKFQR+fZTO9CNGUNokkPatT/0= +github.com/charmbracelet/x/exp/golden v0.0.0-20241011142426-46044092ad91 h1:payRxjMjKgx2PaCWLZ4p3ro9y97+TVLZNaRZgJwSVDQ= +github.com/charmbracelet/x/exp/golden v0.0.0-20241011142426-46044092ad91/go.mod h1:wDlXFlCrmJ8J+swcL/MnGUuYnqgQdW9rhSD61oNMb6U= +github.com/charmbracelet/x/exp/strings v0.0.0-20240722160745-212f7b056ed0 h1:qko3AQ4gK1MTS/de7F5hPGx6/k1u0w4TeYmBFwzYVP4= +github.com/charmbracelet/x/exp/strings v0.0.0-20240722160745-212f7b056ed0/go.mod h1:pBhA0ybfXv6hDjQUZ7hk1lVxBiUbupdw5R31yPUViVQ= +github.com/charmbracelet/x/term v0.2.2 h1:xVRT/S2ZcKdhhOuSP4t5cLi5o+JxklsoEObBSgfgZRk= +github.com/charmbracelet/x/term v0.2.2/go.mod h1:kF8CY5RddLWrsgVwpw4kAa6TESp6EB5y3uxGLeCqzAI= +github.com/charmbracelet/x/termios v0.1.1 h1:o3Q2bT8eqzGnGPOYheoYS8eEleT5ZVNYNy8JawjaNZY= +github.com/charmbracelet/x/termios v0.1.1/go.mod h1:rB7fnv1TgOPOyyKRJ9o+AsTU/vK5WHJ2ivHeut/Pcwo= +github.com/charmbracelet/x/xpty v0.1.2 h1:Pqmu4TEJ8KeA9uSkISKMU3f+C1F6OGBn8ABuGlqCbtI= +github.com/charmbracelet/x/xpty v0.1.2/go.mod h1:XK2Z0id5rtLWcpeNiMYBccNNBrP2IJnzHI0Lq13Xzq4= +github.com/clipperhouse/displaywidth v0.9.0 h1:Qb4KOhYwRiN3viMv1v/3cTBlz3AcAZX3+y9OLhMtAtA= +github.com/clipperhouse/displaywidth v0.9.0/go.mod h1:aCAAqTlh4GIVkhQnJpbL0T/WfcrJXHcj8C0yjYcjOZA= +github.com/clipperhouse/stringish v0.1.1 h1:+NSqMOr3GR6k1FdRhhnXrLfztGzuG+VuFDfatpWHKCs= +github.com/clipperhouse/stringish v0.1.1/go.mod h1:v/WhFtE1q0ovMta2+m+UbpZ+2/HEXNWYXQgCt4hdOzA= +github.com/clipperhouse/uax29/v2 v2.5.0 h1:x7T0T4eTHDONxFJsL94uKNKPHrclyFI0lm7+w94cO8U= +github.com/clipperhouse/uax29/v2 v2.5.0/go.mod h1:Wn1g7MK6OoeDT0vL+Q0SQLDz/KpfsVRgg6W7ihQeh4g= +github.com/creack/goselect v0.1.2 h1:2DNy14+JPjRBgPzAd1thbQp4BSIihxcBf0IXhQXDRa0= +github.com/creack/goselect v0.1.2/go.mod h1:a/NhLweNvqIYMuxcMOuWY516Cimucms3DglDzQP3hKY= +github.com/creack/pty v1.1.24 h1:bJrF4RRfyJnbTJqzRLHzcGaZK1NeM5kTC9jGgovnR1s= +github.com/creack/pty v1.1.24/go.mod h1:08sCNb52WyoAwi2QDyzUCTgcvVFhUzewun7wtTfvcwE= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= +github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= +github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f h1:Y/CXytFA4m6baUTXGLOoWe4PQhGxaX0KpnayAqC48p4= +github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f/go.mod h1:vw97MGsxSvLiUE2X8qFplwetxpGLQrlU1Q9AUEIzCaM= +github.com/lucasb-eyer/go-colorful v1.3.0 h1:2/yBRLdWBZKrf7gB40FoiKfAWYQ0lqNcbuQwVHXptag= +github.com/lucasb-eyer/go-colorful v1.3.0/go.mod h1:R4dSotOR9KMtayYi1e77YzuveK+i7ruzyGqttikkLy0= +github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= +github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= +github.com/mattn/go-localereader v0.0.1 h1:ygSAOl7ZXTx4RdPYinUpg6W99U8jWvWi9Ye2JC/oIi4= +github.com/mattn/go-localereader v0.0.1/go.mod h1:8fBrzywKY7BI3czFoHkuzRoWE9C+EiG4R1k4Cjx5p88= +github.com/mattn/go-runewidth v0.0.19 h1:v++JhqYnZuu5jSKrk9RbgF5v4CGUjqRfBm05byFGLdw= +github.com/mattn/go-runewidth v0.0.19/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs= +github.com/mitchellh/hashstructure/v2 v2.0.2 h1:vGKWl0YJqUNxE8d+h8f6NJLcCJrgbhC4NcD46KavDd4= +github.com/mitchellh/hashstructure/v2 v2.0.2/go.mod h1:MG3aRVU/N29oo/V/IhBX8GR/zz4kQkprJgF2EVszyDE= +github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 h1:ZK8zHtRHOkbHy6Mmr5D264iyp3TiX5OmNcI5cIARiQI= +github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6/go.mod h1:CJlz5H+gyd6CUWT45Oy4q24RdLyn7Md9Vj2/ldJBSIo= +github.com/muesli/cancelreader v0.2.2 h1:3I4Kt4BQjOR54NavqnDogx/MIoWBFa0StPA8ELUXHmA= +github.com/muesli/cancelreader v0.2.2/go.mod h1:3XuTXfFS2VjM+HTLZY9Ak0l6eUKfijIfMUZ4EgX0QYo= +github.com/muesli/termenv v0.16.0 h1:S5AlUN9dENB57rsbnkPyfdGuWIlkmzJjbFf0Tf5FWUc= +github.com/muesli/termenv v0.16.0/go.mod h1:ZRfOIKPFDYQoDFF4Olj7/QJbW60Ol/kL1pU3VfY/Cnk= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ= +github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88= +github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk= +github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= +github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavMF/ppJZNG9ZpyihvCd0w101no= +github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM= +go.bug.st/serial v1.6.4 h1:7FmqNPgVp3pu2Jz5PoPtbZ9jJO5gnEnZIvnI1lzve8A= +go.bug.st/serial v1.6.4/go.mod h1:nofMJxTeNVny/m6+KaafC6vJGj3miwQZ6vW4BZUGJPI= +golang.org/x/exp v0.0.0-20231006140011-7918f672742d h1:jtJma62tbqLibJ5sFQz8bKtEM8rJBtfilJ2qTU199MI= +golang.org/x/exp v0.0.0-20231006140011-7918f672742d/go.mod h1:ldy0pHrwJyGW56pPQzzkH36rKxoZW1tw7ZJpeKx+hdo= +golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.50.0 h1:dkKZEFTEKaLr1sjhs0WiKqoi8Jj5nHbS3Njmx1UoMJs= +golang.org/x/sys v0.50.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og= +golang.org/x/term v0.48.0 h1:G6UU1eiSB8nl3DzeUGt7Mg+2SctI8tPS6Z5/mBt6cOk= +golang.org/x/term v0.48.0/go.mod h1:BRDLHCZr2SUjIeBHHpRyQeC9xzSAVbzY2rvKgafnHf4= +golang.org/x/text v0.23.0 h1:D71I7dUrlY+VX0gQShAThNGHFxZ13dGLBHQLVl1mJlY= +golang.org/x/text v0.23.0/go.mod h1:/BLNzu4aZCJ1+kcD0DNRotWKage4q2rGVAg4o22unh4= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/tools/onboard/internal/cumulocity/c8y.go b/tools/onboard/internal/cumulocity/c8y.go new file mode 100644 index 0000000..6f6825f --- /dev/null +++ b/tools/onboard/internal/cumulocity/c8y.go @@ -0,0 +1,262 @@ +// Package cumulocity reaches Cumulocity through go-c8y-cli (the c8y +// command), with the user's own session, so the tool needs no credentials. +package cumulocity + +import ( + "bufio" + "bytes" + "encoding/json" + "errors" + "fmt" + "os" + "os/exec" + "strings" +) + +// Runner runs c8y with the given environment additions and returns its +// standard output. Interactive runs connect c8y to the terminal for its +// prompts, and still capture standard output. +type Runner interface { + Run(env []string, args ...string) ([]byte, error) + Interactive(env []string, args ...string) ([]byte, error) +} + +// Exec runs the real c8y. +type Exec struct{} + +// ExitError is a c8y run that failed. go-c8y-cli maps HTTP errors to +// exit codes: 4 is 404 (not found). +type ExitError struct { + Code int + Stderr string +} + +func (e *ExitError) Error() string { + if e.Stderr != "" { + return e.Stderr + } + return fmt.Sprintf("c8y exited with status %d", e.Code) +} + +// exitNotFound is go-c8y-cli's exit code for HTTP 404. +const exitNotFound = 4 + +func isNotFound(err error) bool { + var ee *ExitError + return errors.As(err, &ee) && ee.Code == exitNotFound +} + +func (Exec) Run(env []string, args ...string) ([]byte, error) { + cmd := exec.Command("c8y", args...) + cmd.Env = append(os.Environ(), env...) + var stderr bytes.Buffer + cmd.Stderr = &stderr + out, err := cmd.Output() + if err != nil { + var xe *exec.ExitError + if errors.As(err, &xe) { + return out, &ExitError{Code: xe.ExitCode(), Stderr: lastLine(stderr.String())} + } + return out, err + } + return out, nil +} + +// lastLine is the last non-empty line of c8y's error output: its message. +func lastLine(s string) string { + lines := strings.Split(strings.TrimSpace(s), "\n") + return strings.TrimSpace(lines[len(lines)-1]) +} + +func (Exec) Interactive(env []string, args ...string) ([]byte, error) { + cmd := exec.Command("c8y", args...) + cmd.Env = append(os.Environ(), env...) + cmd.Stdin = os.Stdin + cmd.Stderr = os.Stderr + return cmd.Output() +} + +// Installed reports whether c8y is on the PATH. +func Installed() bool { + _, err := exec.LookPath("c8y") + return err == nil +} + +// InstallHelp says how to install go-c8y-cli on an operating system. +func InstallHelp(goos string) string { + switch goos { + case "darwin": + return "Install it with Homebrew: brew install go-c8y-cli" + case "windows": + return "Install it with Scoop or winget, as described at https://goc8ycli.netlify.app/docs/installation/" + default: + return "Install it as described at https://goc8ycli.netlify.app/docs/installation/" + } +} + +// Client calls c8y. +type Client struct { + Run Runner + env []string // the session the user picked in this run, if any +} + +// Session is the Cumulocity login c8y uses. +type Session struct { + Host string // "example.cumulocity.com", without the scheme + Tenant string + User string +} + +// ErrNoSession means c8y has no working login. +var ErrNoSession = errors.New("go-c8y-cli is not logged in to Cumulocity") + +// Session returns the active session, after checking that it works. +func (c *Client) Session() (Session, error) { + out, err := c.Run.Run(c.env, "sessions", "get", "-o", "json") + if err != nil { + return Session{}, ErrNoSession + } + var s struct { + Host string `json:"host"` + Tenant string `json:"tenant"` + Username string `json:"username"` + } + if json.Unmarshal(out, &s) != nil || s.Host == "" { + return Session{}, ErrNoSession + } + if _, err := c.Run.Run(c.env, "currenttenant", "get", "-o", "json"); err != nil { + return Session{}, ErrNoSession + } + host := strings.TrimPrefix(strings.TrimPrefix(s.Host, "https://"), "http://") + return Session{Host: strings.TrimSuffix(host, "/"), Tenant: s.Tenant, User: s.Username}, nil +} + +// ChooseSession runs go-c8y-cli's own session picker and login in the +// terminal, and uses the session the user picks for the rest of the run. +func (c *Client) ChooseSession() error { + out, err := c.Run.Interactive(nil, "sessions", "set", "--shell", "bash") + if err != nil { + return err + } + env := ParseExports(out) + if len(env) == 0 { + return ErrNoSession + } + c.env = env + return nil +} + +// ParseExports reads the shell lines "export NAME='value'" that +// `c8y sessions set --shell bash` prints. +func ParseExports(out []byte) []string { + var env []string + sc := bufio.NewScanner(bytes.NewReader(out)) + for sc.Scan() { + line := strings.TrimSpace(sc.Text()) + if !strings.HasPrefix(line, "export ") { + continue + } + name, value, ok := strings.Cut(strings.TrimPrefix(line, "export "), "=") + if !ok || name == "" { + continue + } + env = append(env, name+"="+unquote(value)) + } + return env +} + +// unquote undoes bash quoting of one word: 'single' (with '\” for a +// quote), "double" (with backslash escapes), or bare. +func unquote(v string) string { + var b strings.Builder + for i := 0; i < len(v); i++ { + switch v[i] { + case '\'': + j := strings.IndexByte(v[i+1:], '\'') + if j < 0 { + b.WriteString(v[i+1:]) + return b.String() + } + b.WriteString(v[i+1 : i+1+j]) + i += j + 1 + case '"': + for i++; i < len(v) && v[i] != '"'; i++ { + if v[i] == '\\' && i+1 < len(v) { + i++ + } + b.WriteByte(v[i]) + } + case '\\': + if i+1 < len(v) { + i++ + b.WriteByte(v[i]) + } + default: + b.WriteByte(v[i]) + } + } + return b.String() +} + +// Device is a device in Cumulocity. +type Device struct { + ID string + Name string + Available bool +} + +// FindDevice looks a device up by the board's ID. +func (c *Client) FindDevice(externalID string) (Device, bool, error) { + out, err := c.Run.Run(c.env, "identity", "get", "--name", externalID, + "--type", "c8y_Serial", "--silentStatusCodes", "404", "-o", "json") + if isNotFound(err) { + return Device{}, false, nil + } + if err != nil { + return Device{}, false, err + } + var id struct { + ManagedObject struct { + ID string `json:"id"` + } `json:"managedObject"` + } + if len(bytes.TrimSpace(out)) == 0 || json.Unmarshal(out, &id) != nil || + id.ManagedObject.ID == "" { + return Device{}, false, nil + } + d, err := c.Device(id.ManagedObject.ID) + return d, err == nil, err +} + +// Device reads a device by its ID in Cumulocity. +func (c *Client) Device(id string) (Device, error) { + out, err := c.Run.Run(c.env, "inventory", "get", "--id", id, "-o", "json") + if err != nil { + return Device{}, err + } + var mo struct { + ID string `json:"id"` + Name string `json:"name"` + Availability struct { + Status string `json:"status"` + } `json:"c8y_Availability"` + } + if err := json.Unmarshal(out, &mo); err != nil { + return Device{}, err + } + return Device{ID: mo.ID, Name: mo.Name, Available: mo.Availability.Status == "AVAILABLE"}, nil +} + +// Register registers the board's ID with a one-time password, so the board +// can enroll for a certificate. +func (c *Client) Register(externalID, name, otp string) error { + _, err := c.Run.Run(c.env, "deviceregistration", "register-ca", + "--id", externalID, "--name", name, "--one-time-password", otp, + "--force", "-o", "json") + return err +} + +// DeviceURL is the device's page in Cumulocity's Device Management. +func DeviceURL(host, id string) string { + return fmt.Sprintf("https://%s/apps/devicemanagement/index.html#/device/%s/device-info", host, id) +} diff --git a/tools/onboard/internal/cumulocity/c8y_test.go b/tools/onboard/internal/cumulocity/c8y_test.go new file mode 100644 index 0000000..a1ab145 --- /dev/null +++ b/tools/onboard/internal/cumulocity/c8y_test.go @@ -0,0 +1,90 @@ +package cumulocity + +import ( + "errors" + "strings" + "testing" +) + +type fakeRunner struct { + answers map[string]string + calls []string +} + +func (f *fakeRunner) Run(env []string, args ...string) ([]byte, error) { + key := strings.Join(args, " ") + f.calls = append(f.calls, key) + if strings.HasPrefix(key, "identity get --name missing") { + return nil, &ExitError{Code: exitNotFound, Stderr: "404 Not Found"} + } + if strings.HasPrefix(key, "identity get --name forbidden") { + return nil, &ExitError{Code: 3, Stderr: "403 Forbidden"} + } + for prefix, out := range f.answers { + if strings.HasPrefix(key, prefix) { + return []byte(out), nil + } + } + return nil, errors.New("exit status 1") +} + +func (f *fakeRunner) Interactive(env []string, args ...string) ([]byte, error) { + return f.Run(env, args...) +} + +func TestSession(t *testing.T) { + r := &fakeRunner{answers: map[string]string{ + "sessions get": `{"host":"https://example.eu-latest.cumulocity.com","tenant":"t1","username":"me"}`, + "currenttenant get": `{"name":"t1"}`, + }} + s, err := (&Client{Run: r}).Session() + if err != nil || s.Host != "example.eu-latest.cumulocity.com" { + t.Fatalf("%+v %v", s, err) + } +} + +func TestNoSession(t *testing.T) { + r := &fakeRunner{answers: map[string]string{ + "sessions get": `{"host":"https://example.com"}`, + }} + if _, err := (&Client{Run: r}).Session(); err != ErrNoSession { + t.Fatalf("got %v", err) + } +} + +func TestFindDevice(t *testing.T) { + r := &fakeRunner{answers: map[string]string{ + "identity get --name tedge-1": `{"managedObject":{"id":"42"}}`, + "inventory get --id 42": `{"id":"42","name":"Pump 7","c8y_Availability":{"status":"AVAILABLE"}}`, + "identity get --name tedge-2": ``, + }} + c := &Client{Run: r} + d, found, err := c.FindDevice("tedge-1") + if err != nil || !found || d.Name != "Pump 7" || !d.Available { + t.Fatalf("%+v %v %v", d, found, err) + } + if _, found, err := c.FindDevice("tedge-2"); found || err != nil { + t.Fatalf("found %v %v", found, err) + } +} + +func TestParseExports(t *testing.T) { + out := []byte("export C8Y_HOST='https://example.com'\nexport C8Y_TOKEN='a'\\''b'\nexport C8Y_X=\"q\\\"r\"\necho hi\n") + env := ParseExports(out) + want := []string{"C8Y_HOST=https://example.com", "C8Y_TOKEN=a'b", `C8Y_X=q"r`} + if strings.Join(env, "|") != strings.Join(want, "|") { + t.Fatalf("got %q", env) + } +} + +func TestFindDeviceExitCodes(t *testing.T) { + c := &Client{Run: &fakeRunner{}} + // go-c8y-cli exits with 4 when the identity does not exist. + if _, found, err := c.FindDevice("missing"); found || err != nil { + t.Fatalf("not found: found=%v err=%v", found, err) + } + // Any other failure is a failure, with c8y's message. + if _, _, err := c.FindDevice("forbidden"); err == nil || err.Error() != "403 Forbidden" { + t.Fatalf("forbidden: %v", err) + } +} diff --git a/tools/onboard/internal/device/device.go b/tools/onboard/internal/device/device.go new file mode 100644 index 0000000..96cbdba --- /dev/null +++ b/tools/onboard/internal/device/device.go @@ -0,0 +1,180 @@ +// Package device finds boards with tiny-edge firmware on the computer's USB +// serial ports, by asking each one. +package device + +import ( + "context" + "errors" + "fmt" + "runtime" + "sort" + "strings" + "sync" + "time" + + "go.bug.st/serial" + "go.bug.st/serial/enumerator" + + "github.com/thin-edge/tiny-edge/tools/onboard/internal/improv" +) + +// USB vendors of the USB-serial chips boards use. +var knownVendors = map[string]string{ + "303A": "Espressif USB", + "1A86": "WCH USB-serial", + "10C4": "Silicon Labs USB-serial", + "0403": "FTDI USB-serial", +} + +// Board is a device that answered as tiny-edge firmware. +type Board struct { + Port string + Client *improv.Client + Info improv.TedgeInfo + Device improv.DeviceInfo +} + +// Candidates lists the serial ports that may have a board on them. +func Candidates() ([]string, error) { + ports, err := enumerator.GetDetailedPortsList() + if err != nil { + return nil, err + } + var names []string + for _, p := range ports { + if !p.IsUSB { + continue + } + if _, ok := knownVendors[strings.ToUpper(p.VID)]; !ok { + continue + } + // macOS lists each port twice; the "cu" one is for calling out. + if runtime.GOOS == "darwin" && strings.HasPrefix(p.Name, "/dev/tty.") { + continue + } + names = append(names, p.Name) + } + sort.Strings(names) + return names, nil +} + +// Open opens a port without resetting the board: DTR and RTS stay low from +// the start, which is what the auto-reset circuits of most boards need. +func Open(name string) (serial.Port, error) { + return serial.Open(name, &serial.Mode{ + BaudRate: 115200, + InitialStatusBits: &serial.ModemOutputBits{DTR: false, RTS: false}, + }) +} + +// ErrNotTedge means the port has a board, but its firmware does not answer +// tiny-edge's requests. +var ErrNotTedge = errors.New("the firmware does not support setting up this way") + +// Probe asks the board on a port who it is. The returned board keeps the +// port open. +func Probe(ctx context.Context, name string) (*Board, error) { + port, err := Open(name) + if err != nil { + return nil, err + } + c := improv.NewClient(port) + info, err := wakeAndAsk(ctx, c) + if err != nil { + c.Close() + if errors.Is(err, improv.ErrNoAnswer) || improv.IsDeviceError(err, improv.ErrUnknownRPC) { + return nil, ErrNotTedge + } + return nil, err + } + b := &Board{Port: name, Client: c, Info: info} + if di, err := c.DeviceInfo(ctx); err == nil { + b.Device = di + } + return b, nil +} + +func wakeAndAsk(ctx context.Context, c *improv.Client) (improv.TedgeInfo, error) { + var last error + for attempt := 0; attempt < 2; attempt++ { + if err := c.Wake(); err != nil { + return improv.TedgeInfo{}, err + } + time.Sleep(300 * time.Millisecond) + info, err := c.Info(ctx, 2*time.Second) + if err == nil { + return info, nil + } + last = err + if !errors.Is(err, improv.ErrNoAnswer) { + break + } + } + return improv.TedgeInfo{}, last +} + +// Result of probing one port. +type Result struct { + Port string + Board *Board + Err error +} + +// ProbeAll asks every candidate port at once. +func ProbeAll(ctx context.Context, ports []string) []Result { + results := make([]Result, len(ports)) + var wg sync.WaitGroup + for i, p := range ports { + wg.Add(1) + go func(i int, p string) { + defer wg.Done() + b, err := Probe(ctx, p) + results[i] = Result{Port: p, Board: b, Err: err} + }(i, p) + } + wg.Wait() + return results +} + +// ChipName turns "esp32s3" into "ESP32-S3". +func ChipName(chip string) string { + c := strings.ToUpper(chip) + if strings.HasPrefix(c, "ESP32") && len(c) > 5 && c[5] != '-' { + return "ESP32-" + c[5:] + } + if c == "" || c == "UNKNOWN" { + return "Board" + } + return c +} + +// Describe says what a board is, in plain words. +func (b *Board) Describe() string { + what := ChipName(b.Device.Chip) + // Firmware without a name or version of its own reports the RTOS's + // defaults ("zephyr", "0.0.0"), which tell the user nothing; the + // board's ID does. + if b.Device.Firmware != "" && b.Device.Version != "" && b.Device.Version != "0.0.0" { + what += fmt.Sprintf(" running %s %s", b.Device.Firmware, b.Device.Version) + } else if b.Info.DeviceID != "" { + what += " " + b.Info.DeviceID + } + return what + ", " + b.Status() +} + +// Status says where the board stands with Cumulocity. +func (b *Board) Status() string { + switch b.Info.State { + case "connected": + return "connected to " + b.Info.Tenant + case "awaiting-registration", "connecting": + if b.Info.Tenant != "" { + return "not set up yet (waiting for " + b.Info.Tenant + ")" + } + return "not set up yet" + case "waiting-network": + return "not on Wi-Fi yet" + default: + return "not set up yet" + } +} diff --git a/tools/onboard/internal/device/device_test.go b/tools/onboard/internal/device/device_test.go new file mode 100644 index 0000000..bf01ff6 --- /dev/null +++ b/tools/onboard/internal/device/device_test.go @@ -0,0 +1,28 @@ +package device + +import "testing" + +func TestChipName(t *testing.T) { + for in, want := range map[string]string{ + "esp32s3": "ESP32-S3", "esp32c6": "ESP32-C6", "esp32": "ESP32", + "unknown": "Board", "": "Board", "host": "HOST", + } { + if got := ChipName(in); got != want { + t.Errorf("ChipName(%q) = %q, want %q", in, got, want) + } + } +} + +func TestDescribe(t *testing.T) { + b := &Board{} + b.Device.Chip, b.Device.Firmware, b.Device.Version = "esp32s3", "zephyr", "0.0.0" + b.Info.DeviceID, b.Info.State = "tedge-d405927b33d0", "waiting-network" + if got := b.Describe(); got != "ESP32-S3 tedge-d405927b33d0, not on Wi-Fi yet" { + t.Errorf("got %q", got) + } + b.Device.Firmware, b.Device.Version = "modbus-server", "0.10.0" + b.Info.State, b.Info.Tenant = "connected", "example.cumulocity.com" + if got := b.Describe(); got != "ESP32-S3 running modbus-server 0.10.0, connected to example.cumulocity.com" { + t.Errorf("got %q", got) + } +} diff --git a/tools/onboard/internal/device/hardware_test.go b/tools/onboard/internal/device/hardware_test.go new file mode 100644 index 0000000..39888f5 --- /dev/null +++ b/tools/onboard/internal/device/hardware_test.go @@ -0,0 +1,77 @@ +//go:build hardware + +package device + +import ( + "context" + "os" + "testing" + "time" + + "github.com/thin-edge/tiny-edge/tools/onboard/internal/improv" +) + +// A board with the onboarding firmware, on the port in TEDGE_ONBOARD_PORT +// (or found by itself): +// +// go test -tags hardware ./internal/device -run Hardware -v +func TestHardwareProbe(t *testing.T) { + ctx := context.Background() + ports := []string{os.Getenv("TEDGE_ONBOARD_PORT")} + if ports[0] == "" { + var err error + if ports, err = Candidates(); err != nil || len(ports) == 0 { + t.Skipf("no board: %v", err) + } + } + for _, r := range ProbeAll(ctx, ports) { + if r.Err != nil { + t.Fatalf("%s: %v", r.Port, r.Err) + } + b := r.Board + t.Logf("%s: %s", b.Port, b.Describe()) + t.Logf("info: %+v", b.Info) + t.Logf("device: %+v", b.Device) + if b.Info.DeviceID == "" || b.Info.ProtocolVersion != 1 { + t.Fatalf("unexpected info %+v", b.Info) + } + start := time.Now() + if err := b.Client.Identify(ctx); err != nil { + t.Fatalf("identify: %v", err) + } + t.Logf("identify answered in %v", time.Since(start)) + for _, l := range b.Client.RecentLog() { + t.Logf("log: %s", l) + } + b.Client.Close() + } +} + +// A network that does not exist: the board reports provisioning, then that +// it could not connect, and is ready again. Slow (the board tries for up +// to 30 s), so only with TEDGE_ONBOARD_WIFI_TEST=1. +func TestHardwareWiFiFailure(t *testing.T) { + if os.Getenv("TEDGE_ONBOARD_WIFI_TEST") == "" { + t.Skip("set TEDGE_ONBOARD_WIFI_TEST=1") + } + ports, err := Candidates() + if err != nil || len(ports) == 0 { + t.Skipf("no board: %v", err) + } + b, err := Probe(context.Background(), ports[0]) + if err != nil { + t.Fatal(err) + } + defer b.Client.Close() + var states []byte + start := time.Now() + err = b.Client.SetWiFi(context.Background(), "tedge-onboard-no-such-net", "wrong-password", + func(s byte) { states = append(states, s) }) + t.Logf("after %v: err=%v states=%v", time.Since(start), err, states) + if !improv.IsDeviceError(err, improv.ErrUnableConnect) { + t.Fatalf("got %v", err) + } + for _, l := range b.Client.RecentLog() { + t.Logf("log: %s", l) + } +} diff --git a/tools/onboard/internal/improv/client.go b/tools/onboard/internal/improv/client.go new file mode 100644 index 0000000..bab6011 --- /dev/null +++ b/tools/onboard/internal/improv/client.go @@ -0,0 +1,237 @@ +package improv + +import ( + "context" + "errors" + "fmt" + "io" + "regexp" + "strconv" + "strings" + "sync" + "time" +) + +// DeviceError is an error the device reported. +type DeviceError struct{ Code byte } + +func (e *DeviceError) Error() string { + switch e.Code { + case ErrInvalidRPC: + return "the device did not understand the request" + case ErrUnknownRPC: + return "the device does not support this request" + case ErrUnableConnect: + return "the device could not connect to the Wi-Fi network" + case ErrNotAuthorized: + return "the device refused the request" + default: + return fmt.Sprintf("the device reported error %d", e.Code) + } +} + +// IsDeviceError reports whether err is the device error code. +func IsDeviceError(err error, code byte) bool { + var de *DeviceError + return errors.As(err, &de) && de.Code == code +} + +// ErrNoAnswer means the device sent no Improv answer in time. +var ErrNoAnswer = errors.New("the device did not answer") + +// Client talks Improv to one device over a serial line. +type Client struct { + rw io.ReadWriteCloser + frames chan Frame + + mu sync.Mutex + lines []string // the device's recent log lines + err error // why reading stopped +} + +const keepLines = 40 + +// NewClient starts reading from rw. +func NewClient(rw io.ReadWriteCloser) *Client { + c := &Client{rw: rw, frames: make(chan Frame, 16)} + go c.read() + return c +} + +func (c *Client) read() { + var d Decoder + buf := make([]byte, 512) + for { + n, err := c.rw.Read(buf) + if n > 0 { + frames, lines := d.Feed(buf[:n]) + c.mu.Lock() + for _, l := range lines { + if l = cleanLine(l); l != "" { + c.lines = append(c.lines, l) + } + } + if len(c.lines) > keepLines { + c.lines = c.lines[len(c.lines)-keepLines:] + } + c.mu.Unlock() + for _, f := range frames { + select { + case c.frames <- f: + default: // nobody is waiting: drop it + } + } + } + if err != nil && !errors.Is(err, io.EOF) { + c.mu.Lock() + c.err = err + c.mu.Unlock() + close(c.frames) + return + } + } +} + +// Close closes the line. +func (c *Client) Close() error { return c.rw.Close() } + +// RecentLog returns the device's last log lines. +func (c *Client) RecentLog() []string { + c.mu.Lock() + defer c.mu.Unlock() + return append([]string(nil), c.lines...) +} + +// Wake asks the device's shell, if it has one, to lend the console to +// Improv ("tedge improv"). Firmware without a shell ignores the line. +func (c *Client) Wake() error { + _, err := c.rw.Write([]byte("\r\ntedge improv\r\n")) + return err +} + +func (c *Client) drain() { + for { + select { + case <-c.frames: + default: + return + } + } +} + +// call sends an RPC command and waits for its result. onState, if not +// nil, sees each state the device reports meanwhile. +func (c *Client) call(ctx context.Context, cmd byte, payload []byte, + timeout time.Duration, onState func(byte)) ([]string, error) { + c.drain() + if _, err := c.rw.Write(Encode(RPC(cmd, payload))); err != nil { + return nil, err + } + ctx, cancel := context.WithTimeout(ctx, timeout) + defer cancel() + for { + select { + case <-ctx.Done(): + if errors.Is(ctx.Err(), context.DeadlineExceeded) { + return nil, ErrNoAnswer + } + return nil, ctx.Err() + case f, ok := <-c.frames: + if !ok { + c.mu.Lock() + defer c.mu.Unlock() + return nil, c.err + } + switch f.Type { + case TypeState: + if onState != nil && len(f.Data) == 1 { + onState(f.Data[0]) + } + case TypeError: + if len(f.Data) == 1 && f.Data[0] != ErrNone { + return nil, &DeviceError{Code: f.Data[0]} + } + case TypeResult: + rc, strs, ok := ParseResult(f.Data) + if ok && rc == cmd { + return strs, nil + } + } + } + } +} + +// TedgeInfo is what the tiny-edge info command reports. +type TedgeInfo struct { + ProtocolVersion int + DeviceID string + Auth string // "c8y-ca" or "bootstrap" + Tenant string // empty when none is set + State string // "awaiting-registration", "connected", ... +} + +// Info asks for the tiny-edge info. +func (c *Client) Info(ctx context.Context, timeout time.Duration) (TedgeInfo, error) { + s, err := c.call(ctx, CmdTedgeInfo, nil, timeout, nil) + if err != nil { + return TedgeInfo{}, err + } + if len(s) < 5 { + return TedgeInfo{}, fmt.Errorf("the device's answer was incomplete") + } + v, _ := strconv.Atoi(s[0]) + return TedgeInfo{ProtocolVersion: v, DeviceID: s[1], Auth: s[2], Tenant: s[3], State: s[4]}, nil +} + +// DeviceInfo is what Improv's device info command reports. +type DeviceInfo struct { + Firmware, Version, Chip, Name string +} + +// DeviceInfo asks for Improv's device info. +func (c *Client) DeviceInfo(ctx context.Context) (DeviceInfo, error) { + s, err := c.call(ctx, CmdDeviceInfo, nil, 3*time.Second, nil) + if err != nil { + return DeviceInfo{}, err + } + for len(s) < 4 { + s = append(s, "") + } + return DeviceInfo{Firmware: s[0], Version: s[1], Chip: s[2], Name: s[3]}, nil +} + +// SetWiFi sends a network and waits until the device has joined it. +func (c *Client) SetWiFi(ctx context.Context, ssid, password string, onState func(byte)) error { + _, err := c.call(ctx, CmdWiFiSettings, Strings(ssid, password), 90*time.Second, onState) + return err +} + +// Onboard sends the tenant and one-time password. +func (c *Client) Onboard(ctx context.Context, tenant, otp string, reonboard bool) error { + var flags byte + if reonboard { + flags |= FlagReonboard + } + payload := append([]byte{flags}, Strings(tenant, otp)...) + _, err := c.call(ctx, CmdTedgeOnboard, payload, 30*time.Second, nil) + return err +} + +// Identify asks the device to make itself recognisable. +func (c *Client) Identify(ctx context.Context) error { + _, err := c.call(ctx, CmdTedgeIdentify, nil, 3*time.Second, nil) + return err +} + +// Terminal control sequences a shell mixes into its output, and its prompt. +var ansi = regexp.MustCompile(`\x1b\[[0-9;?]*[A-Za-z]`) + +// cleanLine strips terminal control sequences and shell prompts from a log +// line, so it can be shown to the user. +func cleanLine(l string) string { + l = ansi.ReplaceAllString(l, "") + for strings.HasPrefix(l, "uart:~$ ") { + l = strings.TrimPrefix(l, "uart:~$ ") + } + return strings.TrimSpace(strings.TrimPrefix(l, "uart:~$")) +} diff --git a/tools/onboard/internal/improv/client_test.go b/tools/onboard/internal/improv/client_test.go new file mode 100644 index 0000000..823a888 --- /dev/null +++ b/tools/onboard/internal/improv/client_test.go @@ -0,0 +1,84 @@ +package improv + +import ( + "context" + "io" + "testing" + "time" +) + +// fakeDevice answers like the provisioning library, with log noise. +type fakeDevice struct { + toTool *io.PipeWriter + fromTool *io.PipeReader +} + +type pipeRW struct { + io.Reader + io.Writer +} + +func (p pipeRW) Close() error { return nil } + +func newFake(t *testing.T, answer func(cmd byte, payload []byte) [][]byte) *Client { + toolR, devW := io.Pipe() + devR, toolW := io.Pipe() + go func() { + var d Decoder + buf := make([]byte, 256) + for { + n, err := devR.Read(buf) + if err != nil { + return + } + frames, _ := d.Feed(buf[:n]) + for _, f := range frames { + devW.Write([]byte(" tedge: a log line\r\n")) + for _, out := range answer(f.Data[0], f.Data[2:]) { + devW.Write(out) + } + } + } + }() + return NewClient(pipeRW{Reader: toolR, Writer: toolW}) +} + +func result(cmd byte, strs ...string) []byte { + data := append([]byte{cmd, 0}, Strings(strs...)...) + data[1] = byte(len(data) - 2) + return Encode(Frame{Type: TypeResult, Data: data}) +} + +func TestInfo(t *testing.T) { + c := newFake(t, func(cmd byte, _ []byte) [][]byte { + return [][]byte{result(cmd, "1", "tedge-d405927b33d0", "c8y-ca", "", "awaiting-registration")} + }) + info, err := c.Info(context.Background(), time.Second) + if err != nil || info.DeviceID != "tedge-d405927b33d0" || info.State != "awaiting-registration" { + t.Fatalf("%+v %v", info, err) + } + if lines := c.RecentLog(); len(lines) != 1 { + t.Fatalf("log: %q", lines) + } +} + +func TestWiFiStatesAndError(t *testing.T) { + c := newFake(t, func(cmd byte, p []byte) [][]byte { + return [][]byte{ + Encode(Frame{Type: TypeState, Data: []byte{StateProvisioning}}), + Encode(Frame{Type: TypeError, Data: []byte{ErrUnableConnect}}), + } + }) + var states []byte + err := c.SetWiFi(context.Background(), "net", "pw", func(s byte) { states = append(states, s) }) + if !IsDeviceError(err, ErrUnableConnect) || len(states) != 1 { + t.Fatalf("%v %v", err, states) + } +} + +func TestNoAnswer(t *testing.T) { + c := newFake(t, func(byte, []byte) [][]byte { return nil }) + if _, err := c.Info(context.Background(), 100*time.Millisecond); err != ErrNoAnswer { + t.Fatalf("got %v", err) + } +} diff --git a/tools/onboard/internal/improv/codec.go b/tools/onboard/internal/improv/codec.go new file mode 100644 index 0000000..210e4ed --- /dev/null +++ b/tools/onboard/internal/improv/codec.go @@ -0,0 +1,191 @@ +// Package improv speaks Improv Wi-Fi over a serial line +// (https://www.improv-wifi.com), with the tiny-edge commands that the +// device-side provisioning library adds (include/tedge/provisioning.h). +package improv + +import "bytes" + +// Frame types. +const ( + TypeState = 0x01 + TypeError = 0x02 + TypeRPC = 0x03 + TypeResult = 0x04 +) + +// States a device reports. +const ( + StateReady = 0x02 + StateProvisioning = 0x03 + StateProvisioned = 0x04 +) + +// Errors a device reports. +const ( + ErrNone = 0x00 + ErrInvalidRPC = 0x01 + ErrUnknownRPC = 0x02 + ErrUnableConnect = 0x03 + ErrNotAuthorized = 0x04 + ErrUnknown = 0xFF +) + +// Commands: Improv's own, and tiny-edge's. +const ( + CmdWiFiSettings = 0x01 + CmdCurrentState = 0x02 + CmdDeviceInfo = 0x03 + CmdTedgeInfo = 0xE0 + CmdTedgeOnboard = 0xE1 + CmdTedgeIdentify = 0xE2 +) + +// FlagReonboard in the onboard command sets up a device that is already +// connected (it gets a new certificate). +const FlagReonboard = 0x01 + +// ProtocolVersion is the version of the tiny-edge commands this tool knows. +const ProtocolVersion = 1 + +var header = []byte("IMPROV") + +const version = 1 + +// Frame is one Improv Serial frame. +type Frame struct { + Type byte + Data []byte +} + +// Encode returns the bytes of a frame, ending with a newline. +func Encode(f Frame) []byte { + var b bytes.Buffer + b.Write(header) + b.WriteByte(version) + b.WriteByte(f.Type) + b.WriteByte(byte(len(f.Data))) + b.Write(f.Data) + var sum byte + for _, c := range b.Bytes() { + sum += c + } + b.WriteByte(sum) + b.WriteByte('\n') + return b.Bytes() +} + +// RPC builds an RPC command frame: [command][length][payload]. +func RPC(cmd byte, payload []byte) Frame { + data := append([]byte{cmd, byte(len(payload))}, payload...) + return Frame{Type: TypeRPC, Data: data} +} + +// Strings encodes length-prefixed strings, as Improv payloads carry them. +func Strings(s ...string) []byte { + var b []byte + for _, v := range s { + b = append(b, byte(len(v))) + b = append(b, v...) + } + return b +} + +// ParseResult splits an RPC result into its command and strings. +func ParseResult(data []byte) (cmd byte, strs []string, ok bool) { + if len(data) < 2 || int(data[1])+2 != len(data) { + return 0, nil, false + } + cmd = data[0] + for pos := 2; pos < len(data); { + n := int(data[pos]) + pos++ + if pos+n > len(data) { + return 0, nil, false + } + strs = append(strs, string(data[pos:pos+n])) + pos += n + } + return cmd, strs, true +} + +// Decoder finds frames in a byte stream that also carries other text (the +// device's log), and collects that text by line. +type Decoder struct { + buf []byte +} + +// Feed adds received bytes and returns the frames and text lines they +// complete. +func (d *Decoder) Feed(p []byte) (frames []Frame, lines []string) { + d.buf = append(d.buf, p...) + for { + i := bytes.Index(d.buf, header) + if i < 0 { + // Keep a tail that may be the start of a header. + keep := len(header) - 1 + if keep > len(d.buf) { + keep = len(d.buf) + } + lines = append(lines, d.takeLines(len(d.buf)-keep)...) + return frames, lines + } + lines = append(lines, d.takeLines(i)...) + // Text before the header without a newline of its own is cut + // off by the frame; drop it, so the buffer starts at the header. + if j := bytes.Index(d.buf, header); j > 0 { + d.buf = d.buf[j:] + } + if len(d.buf) < len(header)+3 { + return frames, lines + } + n := int(d.buf[len(header)+2]) + total := len(header) + 3 + n + 1 + if d.buf[len(header)] != version { + d.buf = d.buf[1:] + continue + } + if len(d.buf) < total { + return frames, lines + } + var sum byte + for _, c := range d.buf[:total-1] { + sum += c + } + if sum != d.buf[total-1] { + d.buf = d.buf[1:] + continue + } + data := append([]byte(nil), d.buf[len(header)+3:total-1]...) + frames = append(frames, Frame{Type: d.buf[len(header)+1], Data: data}) + d.buf = d.buf[total:] + if len(d.buf) > 0 && d.buf[0] == '\n' { + d.buf = d.buf[1:] + } + } +} + +// takeLines removes the first n bytes of the buffer, returning the +// complete lines among them; a partial last line stays for later. +func (d *Decoder) takeLines(n int) []string { + if n <= 0 { + return nil + } + chunk := d.buf[:n] + last := bytes.LastIndexByte(chunk, '\n') + var lines []string + if last >= 0 { + for _, l := range bytes.Split(chunk[:last], []byte("\n")) { + l = bytes.TrimRight(l, "\r") + if len(l) > 0 { + lines = append(lines, string(l)) + } + } + d.buf = d.buf[last+1:] + return lines + } + // No newline yet: keep everything, unless it grows without end. + if len(d.buf) > 4096 { + d.buf = d.buf[n:] + } + return nil +} diff --git a/tools/onboard/internal/improv/codec_test.go b/tools/onboard/internal/improv/codec_test.go new file mode 100644 index 0000000..b1bcf9d --- /dev/null +++ b/tools/onboard/internal/improv/codec_test.go @@ -0,0 +1,80 @@ +package improv + +import ( + "bytes" + "testing" +) + +// frameBytes builds a frame by hand, as the device's C code does. +func frameBytes(typ byte, data []byte) []byte { + b := append([]byte("IMPROV"), 1, typ, byte(len(data))) + b = append(b, data...) + var sum byte + for _, c := range b { + sum += c + } + return append(b, sum, '\n') +} + +func TestEncodeMatchesTheDevice(t *testing.T) { + got := Encode(RPC(CmdTedgeInfo, nil)) + want := frameBytes(TypeRPC, []byte{CmdTedgeInfo, 0}) + if !bytes.Equal(got, want) { + t.Fatalf("got % x, want % x", got, want) + } +} + +func TestDecoderBetweenLogLines(t *testing.T) { + var d Decoder + stream := []byte("[00:00:01.000] tedge: starting\r\n") + stream = append(stream, frameBytes(TypeState, []byte{StateReady})...) + stream = append(stream, []byte("uart:~$ \r\n app: IMPRO not a frame\r\n")...) + stream = append(stream, frameBytes(TypeResult, []byte{CmdTedgeInfo, 0})...) + + var frames []Frame + var lines []string + // A byte at a time, as a slow line delivers it. + for _, c := range stream { + f, l := d.Feed([]byte{c}) + frames = append(frames, f...) + lines = append(lines, l...) + } + if len(frames) != 2 || frames[0].Type != TypeState || frames[1].Type != TypeResult { + t.Fatalf("frames: %+v", frames) + } + if len(lines) != 3 || lines[0] != "[00:00:01.000] tedge: starting" { + t.Fatalf("lines: %q", lines) + } +} + +func TestDecoderSkipsDamagedFrames(t *testing.T) { + var d Decoder + bad := frameBytes(TypeState, []byte{StateReady}) + bad[len(bad)-2] ^= 0xFF + good := frameBytes(TypeError, []byte{ErrUnknownRPC}) + frames, _ := d.Feed(append(bad, good...)) + if len(frames) != 1 || frames[0].Type != TypeError { + t.Fatalf("frames: %+v", frames) + } +} + +func TestDecoderFrameAfterPartialText(t *testing.T) { + var d Decoder + stream := append([]byte("uart:~$ "), frameBytes(TypeState, []byte{StateProvisioned})...) + frames, _ := d.Feed(stream) + if len(frames) != 1 || frames[0].Data[0] != StateProvisioned { + t.Fatalf("frames: %+v", frames) + } +} + +func TestParseResult(t *testing.T) { + data := append([]byte{CmdTedgeInfo, 0}, Strings("1", "tedge-abc", "c8y-ca", "", "stopped")...) + data[1] = byte(len(data) - 2) + cmd, strs, ok := ParseResult(data) + if !ok || cmd != CmdTedgeInfo || len(strs) != 5 || strs[1] != "tedge-abc" || strs[3] != "" { + t.Fatalf("got %v %q %v", cmd, strs, ok) + } + if _, _, ok := ParseResult([]byte{CmdTedgeInfo, 5, 1}); ok { + t.Fatal("a short result parsed") + } +} diff --git a/tools/onboard/internal/ui/ui.go b/tools/onboard/internal/ui/ui.go new file mode 100644 index 0000000..8a547af --- /dev/null +++ b/tools/onboard/internal/ui/ui.go @@ -0,0 +1,210 @@ +// Package ui is how the tool talks to the user: a guided conversation in a +// terminal, or plain lines with no questions for scripts. +package ui + +import ( + "context" + "errors" + "fmt" + "io" + "os" + + "github.com/charmbracelet/huh" + "github.com/charmbracelet/huh/spinner" + "github.com/charmbracelet/lipgloss" +) + +// ErrCancelled means the user stopped the tool (Ctrl-C, Esc). +var ErrCancelled = errors.New("cancelled") + +// NeedsAnswerError means script mode reached a question. +type NeedsAnswerError struct{ Question, Hint string } + +func (e *NeedsAnswerError) Error() string { + if e.Hint != "" { + return fmt.Sprintf("%s (%s)", e.Question, e.Hint) + } + return e.Question +} + +// Option is one choice in a list. +type Option struct { + Label string + Value string +} + +// UI is what the onboarding flow needs from the user's side. +type UI interface { + // Title shows the tool's heading. + Title(text string) + // Step runs fn while showing title, then shows its result: done (✓) + // with fn's one-line summary, or failed (✗). + Step(title string, fn func(ctx context.Context) (string, error)) error + // Say shows a line of information; Warn a line that needs attention. + Say(text string) + Warn(text string) + // Problem explains a failure and what to do about it. + Problem(what, advice string) + // Choose offers options, the first selected. hint names the flag that + // answers it in script mode. + Choose(title, description string, options []Option, hint string) (string, error) + // Confirm asks a yes or no question. + Confirm(title, description string, yes bool, hint string) (bool, error) + // Input asks for text, proposing value. + Input(title, description, value, hint string) (string, error) + // Secret asks for hidden text, such as a password. + Secret(title, description, hint string) (string, error) + // Interactive reports whether questions can be asked. + Interactive() bool +} + +var ( + titleStyle = lipgloss.NewStyle().Bold(true) + okStyle = lipgloss.NewStyle().Foreground(lipgloss.Color("2")) + failStyle = lipgloss.NewStyle().Foreground(lipgloss.Color("1")) + warnStyle = lipgloss.NewStyle().Foreground(lipgloss.Color("3")) + dimStyle = lipgloss.NewStyle().Faint(true) +) + +// Guided asks questions in the terminal. Accessible mode (for screen +// readers) uses plain prompts without redrawing. +type Guided struct { + Out io.Writer + Accessible bool +} + +// NewGuided returns the guided UI. ACCESSIBLE=1 in the environment turns on +// accessible mode. +func NewGuided() *Guided { + return &Guided{Out: os.Stdout, Accessible: os.Getenv("ACCESSIBLE") != ""} +} + +func (g *Guided) Interactive() bool { return true } + +func (g *Guided) Title(text string) { + fmt.Fprintf(g.Out, "\n%s\n\n", titleStyle.Render(text)) +} + +func (g *Guided) Step(title string, fn func(ctx context.Context) (string, error)) error { + var summary string + err := spinner.New().Title(" " + title).Accessible(g.Accessible). + ActionWithErr(func(ctx context.Context) error { + var err error + summary, err = fn(ctx) + return err + }).Run() + if err != nil { + fmt.Fprintf(g.Out, "%s %s\n", failStyle.Render("✗"), title) + return err + } + line := title + if summary != "" { + line = summary + } + fmt.Fprintf(g.Out, "%s %s\n", okStyle.Render("✓"), line) + return nil +} + +func (g *Guided) Say(text string) { fmt.Fprintf(g.Out, " %s\n", text) } +func (g *Guided) Warn(text string) { fmt.Fprintf(g.Out, "%s %s\n", warnStyle.Render("!"), text) } + +func (g *Guided) Problem(what, advice string) { + fmt.Fprintf(g.Out, "\n%s %s\n", failStyle.Render("✗"), what) + if advice != "" { + fmt.Fprintf(g.Out, " %s\n", dimStyle.Render(advice)) + } + fmt.Fprintln(g.Out) +} + +func (g *Guided) run(f huh.Field) error { + err := huh.NewForm(huh.NewGroup(f)).WithAccessible(g.Accessible). + WithShowHelp(true).Run() + if errors.Is(err, huh.ErrUserAborted) { + return ErrCancelled + } + return err +} + +func (g *Guided) Choose(title, description string, options []Option, hint string) (string, error) { + var value string + opts := make([]huh.Option[string], len(options)) + for i, o := range options { + opts[i] = huh.NewOption(o.Label, o.Value) + } + if len(options) > 0 { + value = options[0].Value + } + err := g.run(huh.NewSelect[string]().Title(title).Description(description). + Options(opts...).Value(&value)) + return value, err +} + +func (g *Guided) Confirm(title, description string, yes bool, hint string) (bool, error) { + value := yes + err := g.run(huh.NewConfirm().Title(title).Description(description). + Affirmative("Yes").Negative("No").Value(&value)) + return value, err +} + +func (g *Guided) Input(title, description, value, hint string) (string, error) { + err := g.run(huh.NewInput().Title(title).Description(description).Value(&value). + Validate(func(s string) error { + if s == "" { + return errors.New("please enter a value") + } + return nil + })) + return value, err +} + +func (g *Guided) Secret(title, description, hint string) (string, error) { + var value string + err := g.run(huh.NewInput().Title(title).Description(description). + EchoMode(huh.EchoModePassword).Value(&value)) + return value, err +} + +// Script prints one line per step and answers no questions: each question +// is an error naming the flag that answers it. +type Script struct { + Out io.Writer +} + +func (s *Script) Interactive() bool { return false } +func (s *Script) Title(text string) {} + +func (s *Script) Step(title string, fn func(ctx context.Context) (string, error)) error { + summary, err := fn(context.Background()) + if err != nil { + fmt.Fprintf(s.Out, "FAIL %s: %v\n", title, err) + return err + } + if summary == "" { + summary = title + } + fmt.Fprintf(s.Out, "ok %s\n", summary) + return nil +} + +func (s *Script) Say(text string) { fmt.Fprintf(s.Out, " %s\n", text) } +func (s *Script) Warn(text string) { fmt.Fprintf(s.Out, "warn %s\n", text) } +func (s *Script) Problem(what, advice string) { + fmt.Fprintf(s.Out, "FAIL %s\n", what) + if advice != "" { + fmt.Fprintf(s.Out, " %s\n", advice) + } +} + +func (s *Script) Choose(title, _ string, _ []Option, hint string) (string, error) { + return "", &NeedsAnswerError{Question: title, Hint: hint} +} + +func (s *Script) Confirm(title, _ string, _ bool, hint string) (bool, error) { + return false, &NeedsAnswerError{Question: title, Hint: hint} +} + +func (s *Script) Input(_, _, value, _ string) (string, error) { return value, nil } + +func (s *Script) Secret(title, _, hint string) (string, error) { + return "", &NeedsAnswerError{Question: title, Hint: hint} +} diff --git a/tools/onboard/internal/wifi/wifi.go b/tools/onboard/internal/wifi/wifi.go new file mode 100644 index 0000000..a221a0d --- /dev/null +++ b/tools/onboard/internal/wifi/wifi.go @@ -0,0 +1,202 @@ +// Package wifi reads the computer's Wi-Fi networks and, with the user's +// approval where the system asks for it, their passwords: the macOS +// Keychain, NetworkManager on Linux, and the WLAN profiles on Windows. +package wifi + +import ( + "bufio" + "errors" + "os/exec" + "strings" +) + +// Runner runs a command and returns its standard output. +type Runner func(name string, args ...string) (string, error) + +// Exec runs real commands. +func Exec(name string, args ...string) (string, error) { + out, err := exec.Command(name, args...).Output() + return string(out), err +} + +// Detector reads the networks of one operating system. +type Detector struct { + GOOS string // runtime.GOOS + Run Runner +} + +// ErrNoPassword means the system did not give the password (the user +// declined, or it is not stored). +var ErrNoPassword = errors.New("the password is not available") + +// Current returns the network the computer is connected to, if the system +// says. +func (d Detector) Current() (string, bool) { + switch d.GOOS { + case "darwin": + dev := d.macWiFiDevice() + out, err := d.Run("ipconfig", "getsummary", dev) + if err == nil { + if ssid := macSummarySSID(out); ssid != "" { + return ssid, true + } + } + case "linux": + out, err := d.Run("nmcli", "-t", "-f", "ACTIVE,SSID", "dev", "wifi") + if err == nil { + for _, l := range lines(out) { + if f := nmcliFields(l); len(f) == 2 && f[0] == "yes" && f[1] != "" { + return f[1], true + } + } + } + case "windows": + out, err := d.Run("netsh", "wlan", "show", "interfaces") + if err == nil { + if ssid := netshValue(out, "SSID"); ssid != "" { + return ssid, true + } + } + } + return "", false +} + +// Known returns the networks the computer has joined before. +func (d Detector) Known() []string { + var names []string + switch d.GOOS { + case "darwin": + out, err := d.Run("networksetup", "-listpreferredwirelessnetworks", d.macWiFiDevice()) + if err == nil { + for i, l := range lines(out) { + if i == 0 && strings.HasPrefix(l, "Preferred networks") { + continue + } + if n := strings.TrimSpace(l); n != "" { + names = append(names, n) + } + } + } + case "linux": + out, err := d.Run("nmcli", "-t", "-f", "NAME,TYPE", "connection", "show") + if err == nil { + for _, l := range lines(out) { + if f := nmcliFields(l); len(f) == 2 && f[1] == "802-11-wireless" { + names = append(names, f[0]) + } + } + } + case "windows": + out, err := d.Run("netsh", "wlan", "show", "profiles") + if err == nil { + for _, l := range lines(out) { + if k, v, ok := strings.Cut(l, ":"); ok && + strings.Contains(k, "All User Profile") && strings.TrimSpace(v) != "" { + names = append(names, strings.TrimSpace(v)) + } + } + } + } + return names +} + +// Password returns a network's stored password. On macOS the Keychain asks +// the user to allow it. +func (d Detector) Password(ssid string) (string, error) { + var out string + var err error + switch d.GOOS { + case "darwin": + out, err = d.Run("security", "find-generic-password", "-D", + "AirPort network password", "-a", ssid, "-w") + out = strings.TrimRight(out, "\r\n") + case "linux": + out, err = d.Run("nmcli", "-s", "-g", "802-11-wireless-security.psk", + "connection", "show", ssid) + out = strings.TrimRight(out, "\r\n") + case "windows": + out, err = d.Run("netsh", "wlan", "show", "profile", "name="+ssid, "key=clear") + if err == nil { + out = netshValue(out, "Key Content") + } + default: + return "", ErrNoPassword + } + if err != nil || out == "" { + return "", ErrNoPassword + } + return out, nil +} + +// macWiFiDevice finds the Wi-Fi interface ("en0" on most Macs). +func (d Detector) macWiFiDevice() string { + out, err := d.Run("networksetup", "-listallhardwareports") + if err != nil { + return "en0" + } + wifi := false + for _, l := range lines(out) { + if strings.HasPrefix(l, "Hardware Port:") { + p := strings.TrimSpace(strings.TrimPrefix(l, "Hardware Port:")) + wifi = p == "Wi-Fi" || p == "AirPort" + } else if wifi && strings.HasPrefix(l, "Device:") { + return strings.TrimSpace(strings.TrimPrefix(l, "Device:")) + } + } + return "en0" +} + +// macSummarySSID reads "SSID : name" from ipconfig getsummary. Recent macOS +// releases show "" to programs without location access. +func macSummarySSID(out string) string { + for _, l := range lines(out) { + k, v, ok := strings.Cut(l, " : ") + if ok && strings.TrimSpace(k) == "SSID" { + v = strings.TrimSpace(v) + if v != "" && v != "" { + return v + } + } + } + return "" +} + +// netshValue reads " Key : value" from netsh output, matching the key +// exactly (so "SSID" does not match "BSSID"). +func netshValue(out, key string) string { + for _, l := range lines(out) { + k, v, ok := strings.Cut(l, ":") + if ok && strings.TrimSpace(k) == key { + return strings.TrimSpace(v) + } + } + return "" +} + +// nmcliFields splits nmcli's terse output, where ':' in a value is "\:". +func nmcliFields(line string) []string { + var fields []string + var cur strings.Builder + for i := 0; i < len(line); i++ { + switch { + case line[i] == '\\' && i+1 < len(line): + i++ + cur.WriteByte(line[i]) + case line[i] == ':': + fields = append(fields, cur.String()) + cur.Reset() + default: + cur.WriteByte(line[i]) + } + } + return append(fields, cur.String()) +} + +func lines(s string) []string { + var out []string + sc := bufio.NewScanner(strings.NewReader(s)) + for sc.Scan() { + out = append(out, strings.TrimRight(sc.Text(), "\r")) + } + return out +} diff --git a/tools/onboard/internal/wifi/wifi_test.go b/tools/onboard/internal/wifi/wifi_test.go new file mode 100644 index 0000000..c6f89ac --- /dev/null +++ b/tools/onboard/internal/wifi/wifi_test.go @@ -0,0 +1,82 @@ +package wifi + +import ( + "errors" + "strings" + "testing" +) + +// fake answers commands from a table keyed by the command line. +func fake(answers map[string]string) Runner { + return func(name string, args ...string) (string, error) { + key := strings.Join(append([]string{name}, args...), " ") + if out, ok := answers[key]; ok { + return out, nil + } + return "", errors.New("exit status 1") + } +} + +const macPorts = "\nHardware Port: Ethernet\nDevice: en1\n\nHardware Port: Wi-Fi\nDevice: en0\nEthernet Address: aa\n" + +func TestMac(t *testing.T) { + d := Detector{GOOS: "darwin", Run: fake(map[string]string{ + "networksetup -listallhardwareports": macPorts, + "ipconfig getsummary en0": " {\n BSSID : aa:bb\n SSID : home-net\n}\n", + "networksetup -listpreferredwirelessnetworks en0": "Preferred networks on en0:\n\thome-net\n\tcafe\n", + "security find-generic-password -D AirPort network password -a home-net -w": "s3cret\n", + })} + if s, ok := d.Current(); !ok || s != "home-net" { + t.Fatalf("current %q %v", s, ok) + } + if k := d.Known(); len(k) != 2 || k[1] != "cafe" { + t.Fatalf("known %q", k) + } + if p, err := d.Password("home-net"); err != nil || p != "s3cret" { + t.Fatalf("password %q %v", p, err) + } + if _, err := d.Password("cafe"); err != ErrNoPassword { + t.Fatalf("got %v", err) + } +} + +func TestMacRedacted(t *testing.T) { + d := Detector{GOOS: "darwin", Run: fake(map[string]string{ + "networksetup -listallhardwareports": macPorts, + "ipconfig getsummary en0": " SSID : \n", + })} + if _, ok := d.Current(); ok { + t.Fatal("a redacted SSID was reported") + } +} + +func TestLinux(t *testing.T) { + d := Detector{GOOS: "linux", Run: fake(map[string]string{ + "nmcli -t -f ACTIVE,SSID dev wifi": "no:other\nyes:lab\\:2g\n", + "nmcli -t -f NAME,TYPE connection show": "lab\\:2g:802-11-wireless\nWired:802-3-ethernet\n", + "nmcli -s -g 802-11-wireless-security.psk connection show lab:2g": "pw\n", + })} + if s, ok := d.Current(); !ok || s != "lab:2g" { + t.Fatalf("current %q", s) + } + if k := d.Known(); len(k) != 1 { + t.Fatalf("known %q", k) + } +} + +func TestWindows(t *testing.T) { + d := Detector{GOOS: "windows", Run: fake(map[string]string{ + "netsh wlan show interfaces": " Name : Wi-Fi\r\n BSSID : aa:bb\r\n SSID : office\r\n", + "netsh wlan show profiles": "User profiles\r\n All User Profile : office\r\n All User Profile : home\r\n", + "netsh wlan show profile name=office key=clear": " Key Content : pw123\r\n", + })} + if s, ok := d.Current(); !ok || s != "office" { + t.Fatalf("current %q", s) + } + if k := d.Known(); len(k) != 2 { + t.Fatalf("known %q", k) + } + if p, _ := d.Password("office"); p != "pw123" { + t.Fatalf("password %q", p) + } +} diff --git a/tools/onboard/main.go b/tools/onboard/main.go new file mode 100644 index 0000000..876b40b --- /dev/null +++ b/tools/onboard/main.go @@ -0,0 +1,573 @@ +// tedge-onboard sets up a tiny-edge device for Cumulocity over a USB cable: +// it gives the device the computer's Wi-Fi network, registers it in +// Cumulocity through go-c8y-cli, and waits until it is connected. +// +// Run it with no arguments and it guides you through. Flags are for scripts. +package main + +import ( + "context" + "crypto/rand" + "encoding/json" + "errors" + "flag" + "fmt" + "io" + "math/big" + "os" + "runtime" + "strings" + "time" + + "golang.org/x/term" + + "github.com/thin-edge/tiny-edge/tools/onboard/internal/cumulocity" + "github.com/thin-edge/tiny-edge/tools/onboard/internal/device" + "github.com/thin-edge/tiny-edge/tools/onboard/internal/improv" + "github.com/thin-edge/tiny-edge/tools/onboard/internal/ui" + "github.com/thin-edge/tiny-edge/tools/onboard/internal/wifi" +) + +var version = "dev" + +type options struct { + port string + wifiSSID string + wifiPassword bool // read it from standard input + skipWiFi bool + reonboard bool + yes bool + json bool + name string + timeout time.Duration +} + +func main() { + var o options + flag.StringVar(&o.port, "port", "", "the board's serial port (found by itself when not given)") + flag.StringVar(&o.wifiSSID, "wifi-ssid", "", "the Wi-Fi network for the board (default: the computer's)") + flag.BoolVar(&o.wifiPassword, "wifi-password-stdin", false, "read the Wi-Fi password from standard input") + flag.BoolVar(&o.skipWiFi, "skip-wifi", false, "leave the board's Wi-Fi as it is") + flag.BoolVar(&o.reonboard, "reonboard", false, "set up a board that is already in Cumulocity again (it gets a new certificate)") + flag.BoolVar(&o.yes, "yes", false, "ask no questions; fail where one would be asked") + flag.BoolVar(&o.json, "json", false, "print the result as JSON (with --yes)") + flag.StringVar(&o.name, "name", "", "the device's name in Cumulocity (default: the board's ID)") + flag.DurationVar(&o.timeout, "timeout", 3*time.Minute, "how long to wait for the device to connect") + showVersion := flag.Bool("version", false, "print the version") + flag.Parse() + if *showVersion { + fmt.Println(version) + return + } + + var u ui.UI + if o.yes || !term.IsTerminal(int(os.Stdin.Fd())) { + u = &ui.Script{Out: os.Stderr} + } else { + u = ui.NewGuided() + } + code := run(u, o) + pauseIfStartedByDoubleClick(u) + os.Exit(code) +} + +// result is what --json prints. +type result struct { + Name string `json:"name"` + DeviceID string `json:"deviceId"` + ID string `json:"id"` + Link string `json:"link"` + Status string `json:"status"` +} + +func run(u ui.UI, o options) int { + u.Title("Set up a device for Cumulocity") + f := &flow{u: u, o: o, c8y: &cumulocity.Client{Run: cumulocity.Exec{}}, + wifi: wifi.Detector{GOOS: runtime.GOOS, Run: wifi.Exec}} + + for { + res, err := f.once() + if errors.Is(err, ui.ErrCancelled) { + u.Say("Stopped. Nothing more was changed.") + return 130 + } + if err != nil { + var na *ui.NeedsAnswerError + if errors.As(err, &na) { + u.Problem("A question needs an answer: "+na.Question, na.Hint) + } + return 1 + } + if o.json { + _ = json.NewEncoder(os.Stdout).Encode(res) + } + if !u.Interactive() { + return 0 + } + another, err := u.Confirm("Set up another board?", "Plug it in first.", false, "") + if err != nil || !another { + return 0 + } + f.o.port = "" + } +} + +type flow struct { + u ui.UI + o options + c8y *cumulocity.Client + wifi wifi.Detector +} + +// once sets up one board. +func (f *flow) once() (result, error) { + session, err := f.cumulocity() + if err != nil { + return result{}, err + } + board, err := f.findBoard() + if err != nil { + return result{}, err + } + defer board.Client.Close() + if err := f.checkBoard(board); err != nil { + return result{}, err + } + + existing, again, err := f.alreadySetUp(board) + if err != nil { + return result{}, err + } + + ssid, password, err := f.chooseWiFi(board) + if err != nil { + return result{}, err + } + + name := f.o.name + if name == "" { + name = board.Info.DeviceID + if existing.Name != "" { + name = existing.Name + } + name, err = f.u.Input("Name for the device in Cumulocity", + "This is how it is listed in Device Management. You can change it later.", + name, "--name") + if err != nil { + return result{}, err + } + } + + if f.u.Interactive() { + summary := fmt.Sprintf("Board: %s\nCumulocity: %s\nName: %s", + board.Describe(), session.Host, name) + if ssid != "" { + summary += "\nWi-Fi: " + ssid + } + ok, err := f.u.Confirm("Set up this device?", summary, true, "") + if err != nil { + return result{}, err + } + if !ok { + return result{}, ui.ErrCancelled + } + } + + if ssid != "" { + if err := f.sendWiFi(board, ssid, password); err != nil { + return result{}, err + } + } + return f.register(board, session, name, again) +} + +// cumulocity makes sure go-c8y-cli is there and logged in. +func (f *flow) cumulocity() (cumulocity.Session, error) { + for !cumulocity.Installed() { + f.u.Problem("This tool uses go-c8y-cli to reach Cumulocity, and it is not installed.", + cumulocity.InstallHelp(runtime.GOOS)) + if err := f.retry("Check again", "--yes cannot install it"); err != nil { + return cumulocity.Session{}, err + } + } + for { + var s cumulocity.Session + err := f.u.Step("Checking your Cumulocity login", func(context.Context) (string, error) { + var err error + s, err = f.c8y.Session() + if err != nil { + return "", err + } + return "Cumulocity: " + s.Host, nil + }) + if err == nil { + return s, nil + } + f.u.Problem("go-c8y-cli is not logged in to Cumulocity.", + "Pick one of your saved logins, or create one, in the next step.") + choice, err := f.u.Choose("How do you want to log in?", "", []ui.Option{ + {Label: "Choose or create a Cumulocity login (go-c8y-cli)", Value: "login"}, + {Label: "Quit", Value: "quit"}, + }, "log in with go-c8y-cli first: eval $(c8y sessions set)") + if err != nil { + return cumulocity.Session{}, err + } + if choice == "quit" { + return cumulocity.Session{}, ui.ErrCancelled + } + if err := f.c8y.ChooseSession(); err != nil { + f.u.Warn("The login did not finish.") + } + } +} + +// findBoard finds the board to set up: the only one, the one the user +// picks, or the one plugged in next. +func (f *flow) findBoard() (*device.Board, error) { + for { + var boards []*device.Board + var notTedge []string + err := f.u.Step("Looking for a board on USB", func(ctx context.Context) (string, error) { + ports := []string{f.o.port} + if f.o.port == "" { + var err error + if ports, err = device.Candidates(); err != nil { + return "", err + } + } + for _, r := range device.ProbeAll(ctx, ports) { + switch { + case r.Board != nil: + boards = append(boards, r.Board) + case errors.Is(r.Err, device.ErrNotTedge): + notTedge = append(notTedge, r.Port) + } + } + switch len(boards) { + case 0: + return "", errors.New("no board found") + case 1: + return "Found " + boards[0].Describe(), nil + default: + return fmt.Sprintf("Found %d boards", len(boards)), nil + } + }) + if err == nil { + if len(boards) == 1 { + return boards[0], nil + } + return f.pickBoard(boards) + } + + switch { + case len(notTedge) > 0: + f.u.Problem("A board is connected, but its firmware cannot be set up this way.", + "It needs firmware built with the tiny-edge provisioning library. "+ + "Install that firmware first, then try again.") + default: + f.u.Problem("No board was found.", + "Plug the board into this computer with a USB data cable "+ + "(some cables only charge), then try again.") + } + if err := f.retry("Look again", "--port names the board's port"); err != nil { + return nil, err + } + } +} + +func (f *flow) pickBoard(boards []*device.Board) (*device.Board, error) { + for { + opts := make([]ui.Option, 0, len(boards)+1) + for i, b := range boards { + opts = append(opts, ui.Option{Label: b.Describe(), Value: fmt.Sprint(i)}) + } + opts = append(opts, ui.Option{Label: "Not sure which is which? Make one blink", Value: "identify"}) + choice, err := f.u.Choose("Which board do you want to set up?", "", opts, "--port") + if err != nil { + return nil, err + } + if choice != "identify" { + var i int + fmt.Sscan(choice, &i) + for j, b := range boards { + if j != i { + b.Client.Close() + } + } + return boards[i], nil + } + opts = opts[:len(opts)-1] + which, err := f.u.Choose("Which board should identify itself?", + "It blinks if it has a light for it, and writes a line to its log.", opts, "") + if err != nil { + return nil, err + } + var i int + fmt.Sscan(which, &i) + _ = boards[i].Client.Identify(context.Background()) + f.u.Say("Look at the boards now.") + } +} + +// checkBoard makes sure the tool and the board's firmware fit together. +func (f *flow) checkBoard(b *device.Board) error { + if b.Info.ProtocolVersion > improv.ProtocolVersion { + f.u.Problem("This board's firmware is newer than this tool.", + "Download the latest tedge-onboard and run it again.") + return errors.New("tool too old") + } + if b.Info.Auth != "c8y-ca" { + f.u.Problem("This board's firmware connects with a Cumulocity bootstrap user, "+ + "not with a certificate, so it cannot be registered this way.", + "Register it in Cumulocity's Device Management instead.") + return errors.New("bootstrap firmware") + } + return nil +} + +// alreadySetUp asks before setting up a board that Cumulocity already knows. +func (f *flow) alreadySetUp(b *device.Board) (existing cumulocity.Device, again bool, err error) { + var found bool + err = f.u.Step("Checking whether the device is already in Cumulocity", func(context.Context) (string, error) { + var err error + existing, found, err = f.c8y.FindDevice(b.Info.DeviceID) + if err != nil { + return "", err + } + if found { + return fmt.Sprintf("The device is already in Cumulocity as %q", existing.Name), nil + } + return "The device is new to Cumulocity", nil + }) + if err != nil { + f.u.Problem("Cumulocity could not be asked about the device.", + "Cumulocity said: "+err.Error()+"\nCheck your network and your Cumulocity user's permissions, then try again.") + return existing, false, err + } + if !found && b.Info.State != "connected" { + return existing, false, nil + } + if f.o.reonboard { + return existing, true, nil + } + ok, err := f.u.Confirm("Set this device up again?", + "It is already set up. Setting it up again gives it a new certificate; "+ + "its history in Cumulocity stays.", false, "--reonboard") + if err != nil { + return existing, false, err + } + if !ok { + return existing, false, ui.ErrCancelled + } + return existing, true, nil +} + +// chooseWiFi picks the network for the board and its password. +func (f *flow) chooseWiFi(b *device.Board) (ssid, password string, err error) { + if f.o.skipWiFi { + return "", "", nil + } + ssid = f.o.wifiSSID + if ssid == "" && !f.u.Interactive() { + current, ok := f.wifi.Current() + if !ok { + return "", "", &ui.NeedsAnswerError{Question: "Which Wi-Fi network?", + Hint: "--wifi-ssid, or --skip-wifi"} + } + ssid = current + } + if ssid == "" { + current, haveCurrent := f.wifi.Current() + var opts []ui.Option + if haveCurrent { + opts = append(opts, ui.Option{Label: current + " (this computer's network)", Value: current}) + } + for _, k := range f.wifi.Known() { + if k != current { + opts = append(opts, ui.Option{Label: k, Value: k}) + } + } + opts = append(opts, ui.Option{Label: "Another network (type its name)", Value: "\x00other"}) + if b.Info.State == "connected" || b.Info.State == "awaiting-registration" || + b.Info.State == "connecting" { + opts = append(opts, ui.Option{Label: "Keep the board's current Wi-Fi", Value: "\x00keep"}) + } + ssid, err = f.u.Choose("Which Wi-Fi network should the board use?", + "It needs to reach the internet. Most boards use 2.4 GHz networks only.", opts, "") + if err != nil { + return "", "", err + } + switch ssid { + case "\x00keep": + return "", "", nil + case "\x00other": + if ssid, err = f.u.Input("Wi-Fi network name", "", "", ""); err != nil { + return "", "", err + } + } + } + + if f.o.wifiPassword { + pw, err := io.ReadAll(os.Stdin) + if err != nil { + return "", "", err + } + return ssid, strings.TrimRight(string(pw), "\r\n"), nil + } + if f.u.Interactive() { + f.u.Say(fmt.Sprintf("Getting the password for %q from this computer.", ssid)) + if runtime.GOOS == "darwin" { + f.u.Say("macOS may ask you to allow this.") + } + } + password, err = f.wifi.Password(ssid) + if err == nil { + return ssid, password, nil + } + password, err = f.u.Secret(fmt.Sprintf("Password for %q", ssid), + "This computer did not give the password. Leave it empty for an open network.", + "--wifi-password-stdin") + return ssid, password, err +} + +// sendWiFi gives the board the network, asking again if it cannot join. +func (f *flow) sendWiFi(b *device.Board, ssid, password string) error { + for { + err := f.u.Step(fmt.Sprintf("Connecting the board to %q", ssid), func(ctx context.Context) (string, error) { + if err := b.Client.SetWiFi(ctx, ssid, password, nil); err != nil { + return "", err + } + return fmt.Sprintf("The board is on %q", ssid), nil + }) + if err == nil { + return nil + } + if !improv.IsDeviceError(err, improv.ErrUnableConnect) { + f.u.Problem("The board stopped answering.", boardAdvice(b)) + return err + } + f.u.Problem(fmt.Sprintf("The board could not join %q.", ssid), + "The password may be wrong, or the network may be 5 GHz only or out of range.") + if !f.u.Interactive() { + return err + } + pw, err := f.u.Secret(fmt.Sprintf("Password for %q", ssid), "Enter it again to retry.", "") + if err != nil { + return err + } + password = pw + } +} + +// register registers the device, hands the board its details, and waits +// for it to connect. +func (f *flow) register(b *device.Board, s cumulocity.Session, name string, again bool) (result, error) { + otp, err := oneTimePassword() + if err != nil { + return result{}, err + } + if err := f.u.Step("Registering the device in Cumulocity", func(context.Context) (string, error) { + return "Registered in Cumulocity", f.c8y.Register(b.Info.DeviceID, name, otp) + }); err != nil { + f.u.Problem("Cumulocity did not accept the registration.", + "Cumulocity said: "+err.Error()+"\nYour Cumulocity user needs permission to register devices.") + return result{}, err + } + if err := f.u.Step("Giving the board its Cumulocity details", func(ctx context.Context) (string, error) { + return "The board has its Cumulocity details", b.Client.Onboard(ctx, s.Host, otp, again) + }); err != nil { + f.u.Problem("The board did not take its Cumulocity details.", boardAdvice(b)) + return result{}, err + } + + var dev cumulocity.Device + err = f.u.Step("Waiting for the device to connect (this takes up to a minute)", func(ctx context.Context) (string, error) { + ctx, cancel := context.WithTimeout(ctx, f.o.timeout) + defer cancel() + state := "" + for { + if info, err := b.Client.Info(ctx, 3*time.Second); err == nil { + state = info.State + } + if state == "connected" { + if d, found, err := f.c8y.FindDevice(b.Info.DeviceID); err == nil && found { + dev = d + return fmt.Sprintf("%q is connected", d.Name), nil + } + } + select { + case <-ctx.Done(): + return "", fmt.Errorf("still %s after %s", stateWords(state), f.o.timeout) + case <-time.After(3 * time.Second): + } + } + }) + if err != nil { + f.u.Problem("The device did not connect in time.", + "Check that the Wi-Fi network reaches the internet. The board keeps trying, "+ + "so it may still appear in Cumulocity in a few minutes.") + return result{}, err + } + link := cumulocity.DeviceURL(s.Host, dev.ID) + if f.u.Interactive() { + f.u.Say("") + f.u.Say("Open it in Cumulocity:") + f.u.Say(link) + f.u.Say("") + } + return result{Name: dev.Name, DeviceID: b.Info.DeviceID, ID: dev.ID, Link: link, Status: "connected"}, nil +} + +// retry offers to try again; in script mode it fails. +func (f *flow) retry(label, hint string) error { + choice, err := f.u.Choose("What now?", "", []ui.Option{ + {Label: label, Value: "retry"}, {Label: "Quit", Value: "quit"}, + }, hint) + if err != nil { + return err + } + if choice == "quit" { + return ui.ErrCancelled + } + return nil +} + +func boardAdvice(b *device.Board) string { + advice := "Unplug the board, plug it in again, and run this tool again." + if lines := b.Client.RecentLog(); len(lines) > 0 { + advice += "\nThe board's last message: " + lines[len(lines)-1] + } + return advice +} + +func stateWords(state string) string { + switch state { + case "waiting-network": + return "waiting for the network" + case "waiting-time": + return "waiting for the time" + case "awaiting-registration": + return "waiting for its certificate" + case "connecting": + return "connecting" + case "": + return "not answering" + default: + return state + } +} + +// oneTimePassword makes the password the device enrolls with: 24 +// characters from the system's random source. +func oneTimePassword() (string, error) { + const alphabet = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789" + b := make([]byte, 24) + for i := range b { + n, err := rand.Int(rand.Reader, big.NewInt(int64(len(alphabet)))) + if err != nil { + return "", err + } + b[i] = alphabet[n.Int64()] + } + return string(b), nil +} diff --git a/tools/onboard/pause_other.go b/tools/onboard/pause_other.go new file mode 100644 index 0000000..b38b384 --- /dev/null +++ b/tools/onboard/pause_other.go @@ -0,0 +1,9 @@ +//go:build !windows + +package main + +import "github.com/thin-edge/tiny-edge/tools/onboard/internal/ui" + +// pauseIfStartedByDoubleClick: macOS Terminal and Linux terminals keep the +// window open after the program ends, so there is nothing to do. +func pauseIfStartedByDoubleClick(ui.UI) {} diff --git a/tools/onboard/pause_windows.go b/tools/onboard/pause_windows.go new file mode 100644 index 0000000..264bf7e --- /dev/null +++ b/tools/onboard/pause_windows.go @@ -0,0 +1,31 @@ +//go:build windows + +package main + +import ( + "bufio" + "os" + "unsafe" + + "golang.org/x/sys/windows" + + "github.com/thin-edge/tiny-edge/tools/onboard/internal/ui" +) + +var getConsoleProcessList = windows.NewLazySystemDLL("kernel32.dll").NewProc("GetConsoleProcessList") + +// pauseIfStartedByDoubleClick keeps the console window open when the tool +// was started by double-clicking it: then it is the only process in its +// console, and the window closes as soon as it exits. +func pauseIfStartedByDoubleClick(u ui.UI) { + if !u.Interactive() { + return + } + ids := make([]uint32, 4) + n, _, _ := getConsoleProcessList.Call(uintptr(unsafe.Pointer(&ids[0])), uintptr(len(ids))) + if n != 1 { + return + } + u.Say("Press Enter to close this window.") + _, _ = bufio.NewReader(os.Stdin).ReadString('\n') +} From 746212d25c61651e936ca664608dbfe06ae9e8ae Mon Sep 17 00:00:00 2001 From: reubenmiller Date: Tue, 10 Nov 2026 15:22:12 +1000 Subject: [PATCH 07/14] ci: check tedge-onboard, and attach its binaries to releases The tools workflow checks formatting, runs vet and the tests, and builds the tool for Linux, Windows, and macOS. A release builds the same archives (tools/onboard/build.sh) and attaches them to the GitHub release. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/release-please.yml | 37 +++++++++++++ .github/workflows/tools.yml | 52 +++++++++++++++++++ .../changes/serial-onboarding/proposal.md | 19 +++++-- openspec/changes/serial-onboarding/tasks.md | 30 +++++------ 4 files changed, 118 insertions(+), 20 deletions(-) create mode 100644 .github/workflows/tools.yml diff --git a/.github/workflows/release-please.yml b/.github/workflows/release-please.yml index b766da0..48386e7 100644 --- a/.github/workflows/release-please.yml +++ b/.github/workflows/release-please.yml @@ -20,9 +20,46 @@ jobs: release-please: name: Release PR and tag runs-on: ubuntu-24.04 + outputs: + released: ${{ steps.release.outputs.release_created }} + tag: ${{ steps.release.outputs.tag_name }} steps: - uses: googleapis/release-please-action@v5 + id: release with: token: ${{ secrets.COMMUNITY_ACTIONS_PAT }} config-file: release-please-config.json manifest-file: .release-please-manifest.json + + # tedge-onboard's binaries, attached to the release just published. + onboard-tool: + name: tedge-onboard for ${{ matrix.name }} + needs: release-please + if: needs.release-please.outputs.released == 'true' + runs-on: ${{ matrix.os }} + permissions: + contents: write + strategy: + matrix: + include: + - name: Linux and Windows + os: ubuntu-latest + targets: linux/amd64 linux/arm64 windows/amd64 windows/arm64 + - name: macOS + os: macos-latest + targets: darwin/arm64 darwin/amd64 + steps: + - uses: actions/checkout@v4 + with: + ref: ${{ needs.release-please.outputs.tag }} + - uses: actions/setup-go@v5 + with: + go-version-file: tools/onboard/go.mod + cache-dependency-path: tools/onboard/go.sum + - name: Build and attach + env: + GH_TOKEN: ${{ github.token }} + TAG: ${{ needs.release-please.outputs.tag }} + run: | + tools/onboard/build.sh "${TAG#v}" ${{ matrix.targets }} + gh release upload "$TAG" tools/onboard/dist/* --clobber diff --git a/.github/workflows/tools.yml b/.github/workflows/tools.yml new file mode 100644 index 0000000..405a457 --- /dev/null +++ b/.github/workflows/tools.yml @@ -0,0 +1,52 @@ +# SPDX-License-Identifier: Apache-2.0 +# +# The host tools: tedge-onboard's checks and its builds for every platform. +name: tools + +on: + pull_request: + paths: ["tools/**", ".github/workflows/tools.yml"] + push: + branches: [main] + paths: ["tools/**", ".github/workflows/tools.yml"] + +permissions: + contents: read + +jobs: + test: + name: tedge-onboard checks + runs-on: ubuntu-latest + defaults: + run: + working-directory: tools/onboard + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-go@v5 + with: + go-version-file: tools/onboard/go.mod + cache-dependency-path: tools/onboard/go.sum + - name: Formatting + run: test -z "$(gofmt -l .)" || { gofmt -l .; exit 1; } + - run: go vet ./... + - run: go test ./... + + build: + name: tedge-onboard on ${{ matrix.name }} + runs-on: ${{ matrix.os }} + strategy: + matrix: + include: + - name: Linux and Windows + os: ubuntu-latest + targets: linux/amd64 linux/arm64 windows/amd64 windows/arm64 + - name: macOS + os: macos-latest + targets: darwin/arm64 darwin/amd64 + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-go@v5 + with: + go-version-file: tools/onboard/go.mod + cache-dependency-path: tools/onboard/go.sum + - run: tools/onboard/build.sh "0.0.0-${GITHUB_SHA::7}" ${{ matrix.targets }} diff --git a/openspec/changes/serial-onboarding/proposal.md b/openspec/changes/serial-onboarding/proposal.md index 37b27cb..6e219df 100644 --- a/openspec/changes/serial-onboarding/proposal.md +++ b/openspec/changes/serial-onboarding/proposal.md @@ -54,11 +54,20 @@ application that does not enable it. | `TEDGE_PROVISIONING_SERIAL` | on with `TEDGE_PROVISIONING` | Improv Serial over the console | | `TEDGE_PROVISIONING_BLE` | off (phase 2) | Improv over a BLE GATT service | -The flash and RAM cost per board is measured in the tasks (ESP32-S3-DevKitC, -ESP32-C6, ESP32-WROOM-32) and recorded here before the change is archived. -The serial transport is expected to be small (a frame parser and buffers of -a few hundred bytes). BLE costs what the Bluetooth stack costs, which is why -it is a separate option and a separate phase. +Measured on Zephyr's `samples/minimal` with the minimal profile and the +onboarding overlay, against the same build with `TEDGE_PROVISIONING=n` +(2026-11-10): + +| Board | Flash | Static RAM | +|---|---|---| +| ESP32-S3-DevKitC | +5.0 KB | +5.2 KB (4 KB of it the worker's stack) | +| ESP32-C6-DevKitC | +5.8 KB | +0.5 KB, plus the 4 KB stack in `.noinit` | + +The ESP32-WROOM-32 is not measured: the minimal sample has no board file +for it. The overlay itself also turns on the shell and Wi-Fi credential +storage, which an application may already have; those are the +application's cost, not the library's. BLE costs what the Bluetooth stack +costs, which is why it is a separate option and a separate phase. ### Interface changes diff --git a/openspec/changes/serial-onboarding/tasks.md b/openspec/changes/serial-onboarding/tasks.md index 2ae1ff1..a3c1381 100644 --- a/openspec/changes/serial-onboarding/tasks.md +++ b/openspec/changes/serial-onboarding/tasks.md @@ -15,29 +15,29 @@ ## 3. Zephyr glue -- [ ] 3.1 Kconfig `TEDGE_PROVISIONING`, `TEDGE_PROVISIONING_SERIAL` (and `TEDGE_PROVISIONING_BLE` declared but depending on `TEDGE_EXPERIMENTAL_FEATURES` until phase 2); CMake builds the library through `provisioning/sources.cmake` -- [ ] 3.2 Console transport: the `tedge improv` shell command using the shell bypass, the 60 s idle timeout, frame writes under the shell's output lock; the shell-less path reading the console UART -- [ ] 3.3 `samples/minimal`: an `overlay-onboarding.conf` and a Wi-Fi hook that joins and stores the network (Wi-Fi credentials subsystem) -- [ ] 3.4 Kconfig case for the new options in `tests/kconfig/cases`, and a twister build of the overlay for the ESP32-S3 and ESP32-C6 -- [ ] 3.5 Measure flash and RAM with and without the library on the ESP32-S3-DevKitC, ESP32-C6 and ESP32-WROOM-32 sample builds, and record them in proposal.md +- [x] 3.1 Kconfig `TEDGE_PROVISIONING`, `TEDGE_PROVISIONING_SERIAL` (and `TEDGE_PROVISIONING_BLE` declared but depending on `TEDGE_EXPERIMENTAL_FEATURES` until phase 2); CMake builds the library through `provisioning/sources.cmake` +- [x] 3.2 Console transport: the `tedge improv` shell command using the shell bypass, the 60 s idle timeout, frame writes under the shell's output lock; the shell-less path reading the console UART +- [x] 3.3 `samples/minimal`: an `overlay-onboarding.conf` and a Wi-Fi hook that joins and stores the network (Wi-Fi credentials subsystem) +- [x] 3.4 Kconfig case for the new options in `tests/kconfig/cases`, and a twister build of the overlay for the ESP32-S3 and ESP32-C6 +- [x] 3.5 Measure flash and RAM with and without the library on the ESP32-S3-DevKitC, ESP32-C6 and ESP32-WROOM-32 sample builds, and record them in proposal.md ## 4. Host tool (`tools/onboard/`) -- [ ] 4.1 Go module and the guided shell: steps with a spinner and a one-line result, choices with `charmbracelet/huh` (and its accessible mode), retry offers, "press Enter to close" when started outside a terminal, plain-language message catalogue -- [ ] 4.2 Script mode: `--yes`, `--json`, `--port`, `--wifi-ssid`, `--wifi-password-stdin`, `--skip-wifi`, `--reonboard`, `--timeout`, exit codes -- [ ] 4.3 Improv Serial client: frame codec, requests with timeouts, skipping log output; unit tests against recorded byte streams -- [ ] 4.4 Board discovery: list ports by USB vendor ID, open with DTR and RTS low, send `tedge improv` and the info request, describe each board that answers; wait for a board to be plugged in; the identify choice -- [ ] 4.5 Wi-Fi: current network and known networks on macOS (ipconfig, networksetup, Keychain), Linux (nmcli), Windows (netsh); hidden password prompt; re-ask when the board cannot join; unit tests with a fake command runner -- [ ] 4.6 go-c8y-cli: detect and explain installation, list sessions by tenant, run the interactive login in the same terminal, tenant host, existing-device check, `register-ca` with the chosen name and a generated password, waiting for the identity and availability; unit tests with a fake runner -- [ ] 4.7 The end-to-end flow tying 4.3 to 4.6, with the device's name and link at the end and "set up another board" -- [ ] 4.8 README for the tool written for a non-technical reader: download, run, what it reads from the computer and why, what to do when a step fails +- [x] 4.1 Go module and the guided shell: steps with a spinner and a one-line result, choices with `charmbracelet/huh` (and its accessible mode), retry offers, "press Enter to close" when started outside a terminal, plain-language message catalogue +- [x] 4.2 Script mode: `--yes`, `--json`, `--port`, `--wifi-ssid`, `--wifi-password-stdin`, `--skip-wifi`, `--reonboard`, `--timeout`, exit codes +- [x] 4.3 Improv Serial client: frame codec, requests with timeouts, skipping log output; unit tests against recorded byte streams +- [x] 4.4 Board discovery: list ports by USB vendor ID, open with DTR and RTS low, send `tedge improv` and the info request, describe each board that answers; wait for a board to be plugged in; the identify choice +- [x] 4.5 Wi-Fi: current network and known networks on macOS (ipconfig, networksetup, Keychain), Linux (nmcli), Windows (netsh); hidden password prompt; re-ask when the board cannot join; unit tests with a fake command runner +- [x] 4.6 go-c8y-cli: detect and explain installation, list sessions by tenant, run the interactive login in the same terminal, tenant host, existing-device check, `register-ca` with the chosen name and a generated password, waiting for the identity and availability; unit tests with a fake runner +- [x] 4.7 The end-to-end flow tying 4.3 to 4.6, with the device's name and link at the end and "set up another board" +- [x] 4.8 README for the tool written for a non-technical reader: download, run, what it reads from the computer and why, what to do when a step fails - [ ] 4.9 Walk through the flow with someone unfamiliar with it (or a scripted fresh-user check), and fix every question that needed explaining ## 5. CI and release -- [ ] 5.1 Workflow: `go vet`, `go test`, and builds of the tool for Linux, macOS and Windows (amd64 and arm64) +- [x] 5.1 Workflow: `go vet`, `go test`, and builds of the tool for Linux, macOS and Windows (amd64 and arm64) - [x] 5.2 Build and run the library's host tests in the posix workflow -- [ ] 5.3 Attach the tool's binaries to tiny-edge releases +- [x] 5.3 Attach the tool's binaries to tiny-edge releases ## 6. End-to-end verification From 872404d23b31fb199e03dbc412d511ef475a9bd0 Mon Sep 17 00:00:00 2001 From: reubenmiller Date: Tue, 10 Nov 2026 15:49:47 +1000 Subject: [PATCH 08/14] feat(provisioning): report the Wi-Fi networks the board can see The library answers Improv's "request scanned Wi-Fi networks" (0x04) through a new application hook, wifi_scan. For each network it reports the SSID, the signal, and whether it needs a password: strongest first, each SSID once, without hidden networks. An empty result ends the list. A tool can then offer only networks the board can reach. Recent macOS releases hide the current network's name from programs, so the networks the board sees and the computer knows are how the tool finds it. Co-Authored-By: Claude Opus 5.5 --- include/tedge/provisioning.h | 18 ++++++ .../specs/tedge-onboard-tool/spec.md | 10 ++- .../specs/tedge-provisioning/spec.md | 16 +++++ ports/posix/tests/test_provisioning.c | 33 ++++++++++ provisioning/src/tedge_provisioning.c | 64 +++++++++++++++++++ .../src/tedge_provisioning_internal.h | 1 + 6 files changed, 139 insertions(+), 3 deletions(-) diff --git a/include/tedge/provisioning.h b/include/tedge/provisioning.h index c4016fe..cea6671 100644 --- a/include/tedge/provisioning.h +++ b/include/tedge/provisioning.h @@ -45,6 +45,13 @@ extern "C" { * is connected (it gets a new certificate). */ #define TEDGE_PROVISIONING_FLAG_REONBOARD 0x01 +/** A Wi-Fi network the board can see (tedge_provisioning_hooks::wifi_scan). */ +struct tedge_provisioning_network { + char ssid[33]; + int8_t rssi; /**< dBm */ + bool secured; /**< needs a password */ +}; + /** What the application provides. Every member may be NULL. */ struct tedge_provisioning_hooks { /** @@ -61,6 +68,17 @@ struct tedge_provisioning_hooks { * The library also logs a line naming the device. */ void (*identify)(void *ctx); + /** + * Scan for Wi-Fi networks, filling up to @p max entries of @p out. + * Block until the scan is done, for 15 s at most. A tool uses it to + * offer the networks the board can actually reach. Without it, the + * scan request is answered "unknown RPC". + * + * @return The number of networks found, or a negative errno. + */ + int (*wifi_scan)(struct tedge_provisioning_network *out, size_t max, + void *ctx); + /** The chip or board, for Improv's device info ("ESP32-S3"). NULL * reports "unknown". */ const char *chip; diff --git a/openspec/changes/serial-onboarding/specs/tedge-onboard-tool/spec.md b/openspec/changes/serial-onboarding/specs/tedge-onboard-tool/spec.md index 9f83ae3..bd33128 100644 --- a/openspec/changes/serial-onboarding/specs/tedge-onboard-tool/spec.md +++ b/openspec/changes/serial-onboarding/specs/tedge-onboard-tool/spec.md @@ -74,9 +74,13 @@ the user cancels. `--port` SHALL remain available for scripts. ### Requirement: Wi-Fi is taken from the computer, with consent -The tool SHALL propose the Wi-Fi network the computer is connected to, and -SHALL let the user choose another from the networks the computer knows or -type one. It SHALL take the password from the operating system's credential +The tool SHALL ask the board which networks it can see, and SHALL offer +those, strongest first, marking the network the computer is connected to +and the networks the computer knows. When the operating system hides the +current network's name, the tool SHALL preselect the strongest network the +board sees that the computer knows, which is almost always the current +one. The user SHALL be able to type another network's name. When the board +cannot scan, the tool SHALL offer the networks the computer knows. It SHALL take the password from the operating system's credential store (the macOS Keychain, NetworkManager, or the Windows WLAN profiles), which may ask the user to allow it. When the password is not available, it SHALL ask for it with hidden input. It SHALL NOT print or log the password. diff --git a/openspec/changes/serial-onboarding/specs/tedge-provisioning/spec.md b/openspec/changes/serial-onboarding/specs/tedge-provisioning/spec.md index e4dd219..fe0f9aa 100644 --- a/openspec/changes/serial-onboarding/specs/tedge-provisioning/spec.md +++ b/openspec/changes/serial-onboarding/specs/tedge-provisioning/spec.md @@ -52,6 +52,22 @@ the device name. - **THEN** the library answers with the firmware name, firmware version, chip, and the device's external ID +### Requirement: The board reports the networks it can see + +The library SHALL answer Improv's "request scanned Wi-Fi networks" command +(0x04) through the application's scan hook: one RPC result per network, +carrying the SSID, the signal strength in dBm and whether it needs a +password, strongest first, each SSID once and without hidden networks, +followed by a result with no strings. Without the hook, it SHALL answer +with the "unknown RPC" error. + +#### Scenario: Two access points of one network + +- **WHEN** the scan finds the same SSID on two access points and a hidden + network +- **THEN** the SSID is reported once, with the stronger signal, and the + hidden network is not reported + ### Requirement: Joining Wi-Fi stays the application's job The library SHALL hand Wi-Fi settings to the application through a hook diff --git a/ports/posix/tests/test_provisioning.c b/ports/posix/tests/test_provisioning.c index 1dc2f01..1e875a7 100644 --- a/ports/posix/tests/test_provisioning.c +++ b/ports/posix/tests/test_provisioning.c @@ -193,9 +193,27 @@ static void identify(void *ctx) identified++; } +static int wifi_scan(struct tedge_provisioning_network *out, size_t max, + void *ctx) +{ + static const struct tedge_provisioning_network nets[] = { + { "weak-net", -80, true }, + { "lab-net", -55, true }, + { "", -40, false }, /* hidden: not reported */ + { "lab-net", -70, true }, /* a second access point */ + { "open-cafe", -65, false }, + }; + size_t n = sizeof(nets) / sizeof(nets[0]); + + (void)ctx; + memcpy(out, nets, (n < max ? n : max) * sizeof(nets[0])); + return (int)n; +} + static const struct tedge_provisioning_hooks hooks = { .wifi_connect = wifi_connect, .identify = identify, + .wifi_scan = wifi_scan, .chip = "host", }; @@ -235,6 +253,21 @@ int main(void) CHECK(result_string(&f, 3, s, sizeof(s)) && strcmp(s, "") == 0); CHECK(result_string(&f, 4, s, sizeof(s)) && strcmp(s, "stopped") == 0); + /* The networks the board sees: strongest first, each once, no hidden + * ones, then an empty result. */ + send_rpc(0x04, NULL, 0); + { + const char *want[] = { "lab-net", "open-cafe", "weak-net" }; + const char *secured[] = { "YES", "NO", "YES" }; + + for (int i = 0; i < 3; i++) { + CHECK(expect(&f) && f.type == 0x04 && f.data[0] == 0x04); + CHECK(result_string(&f, 0, s, sizeof(s)) && strcmp(s, want[i]) == 0); + CHECK(result_string(&f, 2, s, sizeof(s)) && strcmp(s, secured[i]) == 0); + } + CHECK(expect(&f) && f.type == 0x04 && f.data[0] == 0x04 && f.data[1] == 0); + } + /* A command nobody knows. */ send_rpc(0x7F, NULL, 0); CHECK(expect(&f) && f.type == 0x02 && f.data[0] == 0x02); diff --git a/provisioning/src/tedge_provisioning.c b/provisioning/src/tedge_provisioning.c index bd4483a..2982906 100644 --- a/provisioning/src/tedge_provisioning.c +++ b/provisioning/src/tedge_provisioning.c @@ -19,6 +19,7 @@ #include #include +#include #include TEDGE_LOG_DECLARE(); @@ -201,6 +202,66 @@ static void device_info(const struct rpc_out *out) result_send(&r, out); } +/* The most networks one scan reports. */ +#define SCAN_MAX 16 + +/* Improv's scan answer: one result per network, [SSID, RSSI, "YES" or "NO" + * for a password], strongest first and each SSID once, then a result with + * no strings to end the list. */ +static void wifi_networks(const struct rpc_out *out) +{ + static struct tedge_provisioning_network found[SCAN_MAX]; + struct result r; + int n; + + if (hooks == NULL || hooks->wifi_scan == NULL) { + out->error(IMPROV_ERR_UNKNOWN_RPC); + return; + } + n = hooks->wifi_scan(found, SCAN_MAX, hooks->ctx); + if (n < 0) { + TEDGE_LOG_WRN("provisioning: Wi-Fi scan failed (%d)", n); + out->error(IMPROV_ERR_UNKNOWN); + return; + } + if (n > SCAN_MAX) { + n = SCAN_MAX; + } + /* Strongest first (a few entries: insertion sort). */ + for (int i = 1; i < n; i++) { + struct tedge_provisioning_network t = found[i]; + int j = i - 1; + + while (j >= 0 && found[j].rssi < t.rssi) { + found[j + 1] = found[j]; + j--; + } + found[j + 1] = t; + } + for (int i = 0; i < n; i++) { + char rssi[8]; + bool seen = false; + + found[i].ssid[sizeof(found[i].ssid) - 1] = '\0'; + if (found[i].ssid[0] == '\0') { + continue; /* hidden network */ + } + for (int j = 0; j < i && !seen; j++) { + seen = strcmp(found[j].ssid, found[i].ssid) == 0; + } + if (seen) { + continue; /* another access point of a network already sent */ + } + snprintf(rssi, sizeof(rssi), "%d", found[i].rssi); + result_start(&r, IMPROV_CMD_WIFI_NETWORKS); + result_add(&r, found[i].ssid); + result_add(&r, rssi); + result_add(&r, found[i].secured ? "YES" : "NO"); + result_send(&r, out); + } + send_empty_result(IMPROV_CMD_WIFI_NETWORKS, out); +} + /* --- tiny-edge's commands ------------------------------------------------ */ static const char *state_name(enum tedge_state state) @@ -339,6 +400,9 @@ static void handle(const uint8_t *pkt, size_t n, const struct rpc_out *out) case IMPROV_CMD_DEVICE_INFO: device_info(out); break; + case IMPROV_CMD_WIFI_NETWORKS: + wifi_networks(out); + break; case TEDGE_PROVISIONING_CMD_INFO: info(out); break; diff --git a/provisioning/src/tedge_provisioning_internal.h b/provisioning/src/tedge_provisioning_internal.h index 570665d..702794a 100644 --- a/provisioning/src/tedge_provisioning_internal.h +++ b/provisioning/src/tedge_provisioning_internal.h @@ -25,6 +25,7 @@ #define IMPROV_CMD_WIFI_SETTINGS 0x01 #define IMPROV_CMD_CURRENT_STATE 0x02 /* serial; "identify" on BLE */ #define IMPROV_CMD_DEVICE_INFO 0x03 +#define IMPROV_CMD_WIFI_NETWORKS 0x04 /* An RPC packet as the transports hand it over: [command][length][data]. * The largest the library accepts: the onboard command with a 64-byte host From 5766f14881e56fd40e477e6fecbf743842cccdd8 Mon Sep 17 00:00:00 2001 From: reubenmiller Date: Tue, 10 Nov 2026 15:49:47 +1000 Subject: [PATCH 09/14] feat(zephyr): a ready-made Wi-Fi scan hook for provisioning tedge_provisioning_wifi_scan() () scans on the first Wi-Fi interface and collects the results from the network management events. Scanning is the same in every Zephyr application. Joining and storing a network stays with the application. The minimal sample uses the hook. Co-Authored-By: Claude Opus 5.5 --- ports/zephyr/CMakeLists.txt | 3 + .../include/tedge/provisioning_zephyr.h | 30 +++++++ ports/zephyr/samples/minimal/src/main.c | 3 +- ports/zephyr/src/tedge_provisioning_wifi.c | 81 +++++++++++++++++++ 4 files changed, 116 insertions(+), 1 deletion(-) create mode 100644 ports/zephyr/include/tedge/provisioning_zephyr.h create mode 100644 ports/zephyr/src/tedge_provisioning_wifi.c diff --git a/ports/zephyr/CMakeLists.txt b/ports/zephyr/CMakeLists.txt index 335f9df..9250ff0 100644 --- a/ports/zephyr/CMakeLists.txt +++ b/ports/zephyr/CMakeLists.txt @@ -37,6 +37,9 @@ if(CONFIG_TEDGE) zephyr_library_sources_ifdef(CONFIG_TEDGE_AUTH_C8Y_CA src/tedge_port_crypto.c) zephyr_library_sources_ifdef(CONFIG_TEDGE_PROVISIONING_SERIAL src/tedge_provisioning_console.c) + if(CONFIG_TEDGE_PROVISIONING AND CONFIG_WIFI) + zephyr_library_sources(src/tedge_provisioning_wifi.c) + endif() zephyr_library_compile_definitions(TEDGE_VERSION_STRING="${TEDGE_VERSION}") # gmtime_r() is POSIX, not C17. picolibc declares it anyway; a host libc # (native_sim) only does when asked, and an undeclared gmtime_r() returns diff --git a/ports/zephyr/include/tedge/provisioning_zephyr.h b/ports/zephyr/include/tedge/provisioning_zephyr.h new file mode 100644 index 0000000..ec23ab6 --- /dev/null +++ b/ports/zephyr/include/tedge/provisioning_zephyr.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: Apache-2.0 */ + +/** + * @file + * @brief Ready-made provisioning hooks for Zephyr applications. + * + * Scanning for Wi-Fi networks is the same in every Zephyr application, so + * the port provides the hook (tedge_provisioning_hooks::wifi_scan). + * Joining and storing a network stays the application's, since only it + * knows how it keeps its credentials: + * + * @code + * static const struct tedge_provisioning_hooks hooks = { + * .wifi_connect = my_join_and_store, + * .wifi_scan = tedge_provisioning_wifi_scan, + * .chip = CONFIG_SOC, + * }; + * @endcode + */ + +#ifndef TEDGE_PROVISIONING_ZEPHYR_H_ +#define TEDGE_PROVISIONING_ZEPHYR_H_ + +#include + +/** Scan on the first Wi-Fi interface (needs CONFIG_WIFI). */ +int tedge_provisioning_wifi_scan(struct tedge_provisioning_network *out, + size_t max, void *ctx); + +#endif /* TEDGE_PROVISIONING_ZEPHYR_H_ */ diff --git a/ports/zephyr/samples/minimal/src/main.c b/ports/zephyr/samples/minimal/src/main.c index d09cf5b..ecaaf55 100644 --- a/ports/zephyr/samples/minimal/src/main.c +++ b/ports/zephyr/samples/minimal/src/main.c @@ -15,7 +15,7 @@ #include #if defined(CONFIG_TEDGE_PROVISIONING) -#include +#include #include #include #endif @@ -130,6 +130,7 @@ static int provision_wifi(const char *ssid, const char *password, void *ctx) static const struct tedge_provisioning_hooks provisioning_hooks = { .wifi_connect = provision_wifi, + .wifi_scan = tedge_provisioning_wifi_scan, .chip = CONFIG_SOC, }; #endif diff --git a/ports/zephyr/src/tedge_provisioning_wifi.c b/ports/zephyr/src/tedge_provisioning_wifi.c new file mode 100644 index 0000000..f3a76d4 --- /dev/null +++ b/ports/zephyr/src/tedge_provisioning_wifi.c @@ -0,0 +1,81 @@ +/* SPDX-License-Identifier: Apache-2.0 + * + * The Wi-Fi scan hook for the provisioning library: one scan on the first + * Wi-Fi interface, collected from the network management events. + */ + +#include + +#include +#include +#include +#include + +#include +#include + +#define SCAN_TIMEOUT K_SECONDS(15) + +static K_MUTEX_DEFINE(scan_lock); +static K_SEM_DEFINE(scan_done, 0, 1); +static struct net_mgmt_event_callback scan_cb; +static bool scan_cb_added; + +/* The scan in progress; written from the event callback. */ +static struct tedge_provisioning_network *scan_out; +static size_t scan_max; +static size_t scan_count; + +static void scan_event(struct net_mgmt_event_callback *cb, uint64_t event, + struct net_if *iface) +{ + ARG_UNUSED(iface); + if (event == NET_EVENT_WIFI_SCAN_RESULT) { + const struct wifi_scan_result *r = cb->info; + struct tedge_provisioning_network *n; + size_t len; + + if (scan_out == NULL || scan_count >= scan_max) { + return; + } + n = &scan_out[scan_count++]; + len = MIN(r->ssid_length, sizeof(n->ssid) - 1); + memcpy(n->ssid, r->ssid, len); + n->ssid[len] = '\0'; + n->rssi = r->rssi; + n->secured = r->security != WIFI_SECURITY_TYPE_NONE; + } else if (event == NET_EVENT_WIFI_SCAN_DONE) { + k_sem_give(&scan_done); + } +} + +int tedge_provisioning_wifi_scan(struct tedge_provisioning_network *out, + size_t max, void *ctx) +{ + struct net_if *iface = net_if_get_first_wifi(); + int rc; + + ARG_UNUSED(ctx); + if (iface == NULL) { + return -ENODEV; + } + k_mutex_lock(&scan_lock, K_FOREVER); + if (!scan_cb_added) { + net_mgmt_init_event_callback(&scan_cb, scan_event, + NET_EVENT_WIFI_SCAN_RESULT | + NET_EVENT_WIFI_SCAN_DONE); + net_mgmt_add_event_callback(&scan_cb); + scan_cb_added = true; + } + scan_out = out; + scan_max = max; + scan_count = 0; + k_sem_reset(&scan_done); + rc = net_mgmt(NET_REQUEST_WIFI_SCAN, iface, NULL, 0); + if (rc == 0 && k_sem_take(&scan_done, SCAN_TIMEOUT) != 0) { + rc = -ETIMEDOUT; + } + scan_out = NULL; + k_mutex_unlock(&scan_lock); + return (rc == 0) ? (int)scan_count : rc; +} From ee2735f37e3ba96bca4bd458d03cb41db4764d63 Mon Sep 17 00:00:00 2001 From: reubenmiller Date: Tue, 10 Nov 2026 15:49:47 +1000 Subject: [PATCH 10/14] feat(onboard): offer the networks the board can see Before asking for the Wi-Fi network, the tool asks the board which networks it sees and lists them, strongest first, with their signal. It marks the computer's current network when the operating system names it, and the networks the computer knows. When the system hides the current network, as macOS 26 does from programs without Location Services, the strongest known network the board sees comes first. That is almost always the computer's current network. Networks the board cannot see are left out, so a 5 GHz network is no longer offered by mistake. If the system names a current network the board cannot see, it is still listed, with the reason. An open network needs no password. Firmware that cannot scan gets the computer's networks, as before. Co-Authored-By: Claude Opus 5.5 --- tools/onboard/internal/improv/client.go | 49 +++++++++ tools/onboard/internal/improv/client_test.go | 15 +++ tools/onboard/internal/improv/codec.go | 1 + tools/onboard/main.go | 35 +++--- tools/onboard/wifi_choice.go | 108 +++++++++++++++++++ tools/onboard/wifi_choice_test.go | 47 ++++++++ 6 files changed, 241 insertions(+), 14 deletions(-) create mode 100644 tools/onboard/wifi_choice.go create mode 100644 tools/onboard/wifi_choice_test.go diff --git a/tools/onboard/internal/improv/client.go b/tools/onboard/internal/improv/client.go index bab6011..fdbbd38 100644 --- a/tools/onboard/internal/improv/client.go +++ b/tools/onboard/internal/improv/client.go @@ -235,3 +235,52 @@ func cleanLine(l string) string { } return strings.TrimSpace(strings.TrimPrefix(l, "uart:~$")) } + +// Network is a Wi-Fi network the device can see. +type Network struct { + SSID string + RSSI int // dBm + Secured bool +} + +// ScanNetworks asks the device which Wi-Fi networks it can see: one result +// per network, strongest first, ended by an empty result. +func (c *Client) ScanNetworks(ctx context.Context) ([]Network, error) { + c.drain() + if _, err := c.rw.Write(Encode(RPC(CmdWiFiNetworks, nil))); err != nil { + return nil, err + } + ctx, cancel := context.WithTimeout(ctx, 20*time.Second) + defer cancel() + var nets []Network + for { + select { + case <-ctx.Done(): + return nil, ErrNoAnswer + case f, ok := <-c.frames: + if !ok { + return nil, io.ErrUnexpectedEOF + } + switch f.Type { + case TypeError: + if len(f.Data) == 1 && f.Data[0] != ErrNone { + return nil, &DeviceError{Code: f.Data[0]} + } + case TypeResult: + cmd, strs, ok := ParseResult(f.Data) + if !ok || cmd != CmdWiFiNetworks { + continue + } + if len(strs) == 0 { + return nets, nil + } + n := Network{SSID: strs[0]} + if len(strs) > 1 { + n.RSSI, _ = strconv.Atoi(strs[1]) + } + n.Secured = len(strs) < 3 || strs[2] == "YES" + nets = append(nets, n) + } + } + } +} diff --git a/tools/onboard/internal/improv/client_test.go b/tools/onboard/internal/improv/client_test.go index 823a888..929a5f5 100644 --- a/tools/onboard/internal/improv/client_test.go +++ b/tools/onboard/internal/improv/client_test.go @@ -82,3 +82,18 @@ func TestNoAnswer(t *testing.T) { t.Fatalf("got %v", err) } } + +func TestScanNetworks(t *testing.T) { + c := newFake(t, func(cmd byte, _ []byte) [][]byte { + return [][]byte{ + result(cmd, "lab-net", "-52", "YES"), + result(cmd, "cafe", "-77", "NO"), + result(cmd), + } + }) + nets, err := c.ScanNetworks(context.Background()) + if err != nil || len(nets) != 2 || nets[0].SSID != "lab-net" || nets[0].RSSI != -52 || + !nets[0].Secured || nets[1].Secured { + t.Fatalf("%+v %v", nets, err) + } +} diff --git a/tools/onboard/internal/improv/codec.go b/tools/onboard/internal/improv/codec.go index 210e4ed..5404173 100644 --- a/tools/onboard/internal/improv/codec.go +++ b/tools/onboard/internal/improv/codec.go @@ -35,6 +35,7 @@ const ( CmdWiFiSettings = 0x01 CmdCurrentState = 0x02 CmdDeviceInfo = 0x03 + CmdWiFiNetworks = 0x04 CmdTedgeInfo = 0xE0 CmdTedgeOnboard = 0xE1 CmdTedgeIdentify = 0xE2 diff --git a/tools/onboard/main.go b/tools/onboard/main.go index 876b40b..2fa9a3e 100644 --- a/tools/onboard/main.go +++ b/tools/onboard/main.go @@ -376,37 +376,44 @@ func (f *flow) chooseWiFi(b *device.Board) (ssid, password string, err error) { } ssid = current } + var scanned []improv.Network if ssid == "" { - current, haveCurrent := f.wifi.Current() - var opts []ui.Option - if haveCurrent { - opts = append(opts, ui.Option{Label: current + " (this computer's network)", Value: current}) - } - for _, k := range f.wifi.Known() { - if k != current { - opts = append(opts, ui.Option{Label: k, Value: k}) + scanOK := false + _ = f.u.Step("Asking the board which Wi-Fi networks it can see", func(ctx context.Context) (string, error) { + nets, err := b.Client.ScanNetworks(ctx) + if err != nil { + // Older firmware cannot scan: offer this computer's + // networks instead, without failing the step. + return "The board cannot list networks; showing this computer's", nil } - } - opts = append(opts, ui.Option{Label: "Another network (type its name)", Value: "\x00other"}) + scanned, scanOK = nets, true + return fmt.Sprintf("The board sees %d Wi-Fi networks", len(nets)), nil + }) + current, _ := f.wifi.Current() + opts := wifiOptions(scanned, scanOK, current, f.wifi.Known()) + opts = append(opts, ui.Option{Label: "Another network (type its name)", Value: choiceOther}) if b.Info.State == "connected" || b.Info.State == "awaiting-registration" || b.Info.State == "connecting" { - opts = append(opts, ui.Option{Label: "Keep the board's current Wi-Fi", Value: "\x00keep"}) + opts = append(opts, ui.Option{Label: "Keep the board's current Wi-Fi", Value: choiceKeep}) } ssid, err = f.u.Choose("Which Wi-Fi network should the board use?", - "It needs to reach the internet. Most boards use 2.4 GHz networks only.", opts, "") + "The board needs a network that reaches the internet.", opts, "") if err != nil { return "", "", err } switch ssid { - case "\x00keep": + case choiceKeep: return "", "", nil - case "\x00other": + case choiceOther: if ssid, err = f.u.Input("Wi-Fi network name", "", "", ""); err != nil { return "", "", err } } } + if openNetwork(scanned, ssid) { + return ssid, "", nil + } if f.o.wifiPassword { pw, err := io.ReadAll(os.Stdin) if err != nil { diff --git a/tools/onboard/wifi_choice.go b/tools/onboard/wifi_choice.go new file mode 100644 index 0000000..2aed186 --- /dev/null +++ b/tools/onboard/wifi_choice.go @@ -0,0 +1,108 @@ +package main + +import ( + "fmt" + + "github.com/thin-edge/tiny-edge/tools/onboard/internal/improv" + "github.com/thin-edge/tiny-edge/tools/onboard/internal/ui" +) + +// Values of the Wi-Fi list's special entries (an SSID cannot start with NUL). +const ( + choiceOther = "\x00other" + choiceKeep = "\x00keep" +) + +// wifiOptions builds the Wi-Fi list, the most likely network first. +// +// With the board's scan: the networks it can see, strongest first, marking +// the computer's current network (when the system names it) and the +// networks the computer knows. When the system hides the current network, +// the strongest network the board sees that the computer knows comes first: +// that is almost always the one the computer is on. Without a scan: the +// computer's current and known networks. +func wifiOptions(scanned []improv.Network, scanOK bool, current string, known []string) []ui.Option { + isKnown := map[string]bool{} + for _, k := range known { + isKnown[k] = true + } + var opts []ui.Option + if !scanOK { + if current != "" { + opts = append(opts, ui.Option{Label: current + " (this computer's network)", Value: current}) + } + for _, k := range known { + if k != current { + opts = append(opts, ui.Option{Label: k, Value: k}) + } + } + return opts + } + + first := -1 + for i, n := range scanned { + if n.SSID == current && current != "" { + first = i + break + } + } + if first < 0 { + for i, n := range scanned { + if isKnown[n.SSID] { + first = i + break + } + } + } + seenCurrent := false + for i, n := range scanned { + label := fmt.Sprintf("%s (%s", n.SSID, signalWords(n.RSSI)) + switch { + case n.SSID == current && current != "": + label += ", this computer's network" + seenCurrent = true + case isKnown[n.SSID]: + label += ", known to this computer" + } + if !n.Secured { + label += ", open" + } + label += ")" + opt := ui.Option{Label: label, Value: n.SSID} + if i == first { + opts = append([]ui.Option{opt}, opts...) + } else { + opts = append(opts, opt) + } + } + if current != "" && !seenCurrent { + opts = append(opts, ui.Option{ + Label: current + " (this computer's network, but the board cannot see it: it may be 5 GHz or out of range)", + Value: current, + }) + } + return opts +} + +// signalWords describes a signal strength in dBm. +func signalWords(rssi int) string { + switch { + case rssi >= -60: + return "strong signal" + case rssi >= -72: + return "good signal" + default: + return "weak signal" + } +} + +// openNetwork reports whether the board saw ssid as a network without a +// password. +func openNetwork(scanned []improv.Network, ssid string) bool { + for _, n := range scanned { + if n.SSID == ssid { + return !n.Secured + } + } + return false +} diff --git a/tools/onboard/wifi_choice_test.go b/tools/onboard/wifi_choice_test.go new file mode 100644 index 0000000..406f233 --- /dev/null +++ b/tools/onboard/wifi_choice_test.go @@ -0,0 +1,47 @@ +package main + +import ( + "testing" + + "github.com/thin-edge/tiny-edge/tools/onboard/internal/improv" +) + +func TestWiFiOptionsHiddenCurrent(t *testing.T) { + // macOS hides the current network: the strongest one the computer + // knows comes first. + scanned := []improv.Network{ + {SSID: "neighbour", RSSI: -45, Secured: true}, + {SSID: "home", RSSI: -58, Secured: true}, + {SSID: "cafe", RSSI: -80, Secured: false}, + } + opts := wifiOptions(scanned, true, "", []string{"office", "home"}) + if opts[0].Value != "home" || opts[0].Label != "home (strong signal, known to this computer)" { + t.Fatalf("first: %+v", opts[0]) + } + if len(opts) != 3 || opts[2].Label != "cafe (weak signal, open)" { + t.Fatalf("opts: %+v", opts) + } +} + +func TestWiFiOptionsNamedCurrent(t *testing.T) { + scanned := []improv.Network{{SSID: "a", RSSI: -40, Secured: true}, {SSID: "home", RSSI: -66, Secured: true}} + opts := wifiOptions(scanned, true, "home", []string{"home"}) + if opts[0].Value != "home" || opts[0].Label != "home (good signal, this computer's network)" { + t.Fatalf("first: %+v", opts[0]) + } +} + +func TestWiFiOptionsCurrentOutOfReach(t *testing.T) { + scanned := []improv.Network{{SSID: "a", RSSI: -40, Secured: true}} + opts := wifiOptions(scanned, true, "home-5g", nil) + if opts[len(opts)-1].Value != "home-5g" { + t.Fatalf("opts: %+v", opts) + } +} + +func TestWiFiOptionsWithoutScan(t *testing.T) { + opts := wifiOptions(nil, false, "home", []string{"home", "office"}) + if len(opts) != 2 || opts[0].Value != "home" || opts[1].Value != "office" { + t.Fatalf("opts: %+v", opts) + } +} From c9cb45c06315f28c2c474f0848645db8dead089b Mon Sep 17 00:00:00 2001 From: reubenmiller Date: Tue, 10 Nov 2026 16:02:52 +1000 Subject: [PATCH 11/14] fix(zephyr): read the console directly when the shell is not on it The console glue picked the shell's bypass whenever CONFIG_SHELL was set. A build whose shell has only other backends (the dummy one the remote shell command uses) has no shell reading the console, so Improv frames were never read. The deciding option is now CONFIG_SHELL_BACKEND_SERIAL. Co-Authored-By: Claude Opus 5.5 --- ports/zephyr/Kconfig | 10 +++++----- ports/zephyr/src/tedge_provisioning_console.c | 15 +++++++++------ 2 files changed, 14 insertions(+), 11 deletions(-) diff --git a/ports/zephyr/Kconfig b/ports/zephyr/Kconfig index 8271875..3499d19 100644 --- a/ports/zephyr/Kconfig +++ b/ports/zephyr/Kconfig @@ -837,15 +837,15 @@ config TEDGE_PROVISIONING_SERIAL bool "Improv Serial on the console" default y help - With the shell, "tedge improv" hands the console to a provisioning - tool until it has been quiet for TEDGE_PROVISIONING_SERIAL_IDLE_S. - Without the shell, the console speaks Improv from boot, beside the - log output. + With the shell on the console (SHELL_BACKEND_SERIAL), "tedge + improv" hands the console to a provisioning tool until it has been + quiet for TEDGE_PROVISIONING_SERIAL_IDLE_S. Otherwise the console + speaks Improv from boot, beside the log output. config TEDGE_PROVISIONING_SERIAL_IDLE_S int "Seconds without a valid frame before the shell gets the console back" default 60 - depends on TEDGE_PROVISIONING_SERIAL && SHELL + depends on TEDGE_PROVISIONING_SERIAL && SHELL_BACKEND_SERIAL config TEDGE_PROVISIONING_BLE bool "Improv over Bluetooth LE (not implemented yet)" diff --git a/ports/zephyr/src/tedge_provisioning_console.c b/ports/zephyr/src/tedge_provisioning_console.c index 65f4041..f882b35 100644 --- a/ports/zephyr/src/tedge_provisioning_console.c +++ b/ports/zephyr/src/tedge_provisioning_console.c @@ -3,7 +3,8 @@ * Improv Serial on the Zephyr console, for the provisioning library * (include/tedge/provisioning.h). * - * With the shell, the console is the shell's, and "tedge improv" lends it + * With the shell on the console (its serial backend), the console is the + * shell's, and "tedge improv" lends it * to a provisioning tool: the shell's bypass mode passes every received * byte to the library, until no valid frame has arrived for * CONFIG_TEDGE_PROVISIONING_SERIAL_IDLE_S. Frames go out through the @@ -11,7 +12,9 @@ * line never lands inside a frame, and unchanged (the shell's printing * would turn a 0x0A byte into CR LF). * - * Without the shell, the console speaks Improv from boot: its UART is read + * Otherwise (no shell, or a shell with only other backends, such as the + * dummy one the remote shell command uses), the console speaks Improv from + * boot: its UART is read * by polling, which every UART driver supports, and a frame is written * with the scheduler locked, so the log thread cannot cut in. */ @@ -24,13 +27,13 @@ #include #include -#if defined(CONFIG_SHELL) +#if defined(CONFIG_SHELL_BACKEND_SERIAL) #include #endif LOG_MODULE_DECLARE(tedge, CONFIG_TEDGE_LOG_LEVEL); -#if defined(CONFIG_SHELL) +#if defined(CONFIG_SHELL_BACKEND_SERIAL) static const struct shell *lent; /* the shell whose console is lent */ @@ -110,7 +113,7 @@ SHELL_SUBCMD_ADD((tedge), improv, NULL, "Lend the console to a setup tool such as tedge-onboard", cmd_improv, 1, 0); -#else /* !CONFIG_SHELL */ +#else /* !CONFIG_SHELL_BACKEND_SERIAL */ #define POLL_MS 20 @@ -159,4 +162,4 @@ static int console_init(void) SYS_INIT(console_init, APPLICATION, 99); -#endif /* CONFIG_SHELL */ +#endif /* CONFIG_SHELL_BACKEND_SERIAL */ From e2cdf677978e70a190d810556e75a522246b24e3 Mon Sep 17 00:00:00 2001 From: reubenmiller Date: Tue, 10 Nov 2026 16:07:40 +1000 Subject: [PATCH 12/14] feat(provisioning): report the application's firmware name and version Improv's device info took the firmware name and version from the port, which only knows the RTOS's defaults ("zephyr", "0.0.0") unless the build names the image. The hooks can now carry the application's own name and version, so a board describes itself as, say, "modbus-server 0.10.0". The tool's hardware tests gain a scan check and a console dump, for when a board does not answer. Co-Authored-By: Claude Opus 5.5 --- include/tedge/provisioning.h | 6 +++ ports/posix/tests/test_provisioning.c | 4 ++ provisioning/src/tedge_provisioning.c | 7 ++- .../onboard/internal/device/hardware_test.go | 44 +++++++++++++++++++ 4 files changed, 59 insertions(+), 2 deletions(-) diff --git a/include/tedge/provisioning.h b/include/tedge/provisioning.h index cea6671..36b8b8c 100644 --- a/include/tedge/provisioning.h +++ b/include/tedge/provisioning.h @@ -83,6 +83,12 @@ struct tedge_provisioning_hooks { * reports "unknown". */ const char *chip; + /** The firmware's name and version, for Improv's device info, when the + * application knows them better than the port does. NULL takes the + * port's (tedge_port_firmware_name() and _version()). */ + const char *firmware_name; + const char *firmware_version; + void *ctx; }; diff --git a/ports/posix/tests/test_provisioning.c b/ports/posix/tests/test_provisioning.c index 1e875a7..ebcd963 100644 --- a/ports/posix/tests/test_provisioning.c +++ b/ports/posix/tests/test_provisioning.c @@ -215,6 +215,8 @@ static const struct tedge_provisioning_hooks hooks = { .identify = identify, .wifi_scan = wifi_scan, .chip = "host", + .firmware_name = "test-app", + .firmware_version = "1.2.3", }; int main(void) @@ -241,6 +243,8 @@ int main(void) /* Improv's device info: firmware, version, chip, the device's ID. */ send_rpc(0x03, NULL, 0); CHECK(expect(&f) && f.type == 0x04 && f.data[0] == 0x03); + CHECK(result_string(&f, 0, s, sizeof(s)) && strcmp(s, "test-app") == 0); + CHECK(result_string(&f, 1, s, sizeof(s)) && strcmp(s, "1.2.3") == 0); CHECK(result_string(&f, 2, s, sizeof(s)) && strcmp(s, "host") == 0); CHECK(result_string(&f, 3, s, sizeof(s)) && strcmp(s, "tedge-prov-test") == 0); diff --git a/provisioning/src/tedge_provisioning.c b/provisioning/src/tedge_provisioning.c index 2982906..b0f5ec0 100644 --- a/provisioning/src/tedge_provisioning.c +++ b/provisioning/src/tedge_provisioning.c @@ -194,8 +194,11 @@ static void device_info(const struct rpc_out *out) id[0] = '\0'; } result_start(&r, IMPROV_CMD_DEVICE_INFO); - result_add(&r, tedge_port_firmware_name()); - result_add(&r, tedge_port_firmware_version()); + result_add(&r, (hooks != NULL && hooks->firmware_name != NULL) + ? hooks->firmware_name : tedge_port_firmware_name()); + result_add(&r, (hooks != NULL && hooks->firmware_version != NULL) + ? hooks->firmware_version + : tedge_port_firmware_version()); result_add(&r, (hooks != NULL && hooks->chip != NULL) ? hooks->chip : "unknown"); result_add(&r, id); diff --git a/tools/onboard/internal/device/hardware_test.go b/tools/onboard/internal/device/hardware_test.go index 39888f5..90ed23b 100644 --- a/tools/onboard/internal/device/hardware_test.go +++ b/tools/onboard/internal/device/hardware_test.go @@ -75,3 +75,47 @@ func TestHardwareWiFiFailure(t *testing.T) { t.Logf("log: %s", l) } } + +// What the board prints, without resetting it, before and after "tedge +// improv": for when a board does not answer. +func TestHardwareConsole(t *testing.T) { + ports, err := Candidates() + if err != nil || len(ports) == 0 { + t.Skipf("no board: %v", err) + } + p, err := Open(ports[0]) + if err != nil { + t.Fatal(err) + } + c := improv.NewClient(p) + defer c.Close() + _, _ = p.Write([]byte("\r\n")) + time.Sleep(2 * time.Second) + _ = c.Wake() + time.Sleep(2 * time.Second) + for _, l := range c.RecentLog() { + t.Logf("%s", l) + } +} + +// The networks the board sees (the tool's Wi-Fi list). +func TestHardwareScan(t *testing.T) { + ports, err := Candidates() + if err != nil || len(ports) == 0 { + t.Skipf("no board: %v", err) + } + b, err := Probe(context.Background(), ports[0]) + if err != nil { + t.Fatal(err) + } + defer b.Client.Close() + start := time.Now() + nets, err := b.Client.ScanNetworks(context.Background()) + if err != nil { + t.Fatal(err) + } + t.Logf("%d networks in %v", len(nets), time.Since(start)) + for _, n := range nets { + t.Logf("%4d dBm secured=%v (%d-character name)", n.RSSI, n.Secured, len(n.SSID)) + } +} From acb3ba13f4e1c60dab42c58a458a0e8c4cec4c66 Mon Sep 17 00:00:00 2001 From: reubenmiller Date: Tue, 10 Nov 2026 16:17:35 +1000 Subject: [PATCH 13/14] docs(openspec): serial onboarding progress The scan task, and the end-to-end run on the ESP32-S3 with the minimal sample. It connected, and a restart from Cumulocity went through and was reported. Co-Authored-By: Claude Opus 5.5 --- openspec/changes/serial-onboarding/tasks.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/openspec/changes/serial-onboarding/tasks.md b/openspec/changes/serial-onboarding/tasks.md index a3c1381..4a80903 100644 --- a/openspec/changes/serial-onboarding/tasks.md +++ b/openspec/changes/serial-onboarding/tasks.md @@ -12,6 +12,7 @@ - [x] 2.5 Improv Serial framing: a byte-at-a-time parser that skips non-frame bytes, and a frame writer that hands one whole frame to the transport - [x] 2.6 Host unit tests of the parser and the RPC layer (a C test in the POSIX port's tests, built by CMake and run in CI): every command, corrupted and split frames, log bytes between frames, oversized strings - [x] 2.7 Extend `scripts/check-portability.py` to apply the core rules to `provisioning/`, and `tests/host-compile/run.sh` to compile it +- [x] 2.8 Improv's network scan (0x04) through an application hook, with a ready-made Zephyr hook, so the tool can offer the networks the board sees and find the computer's network when the OS hides its name ## 3. Zephyr glue @@ -41,7 +42,7 @@ ## 6. End-to-end verification -- [ ] 6.1 Flash the Zephyr sample with the onboarding overlay to the ESP32-S3-DevKitC on this computer, run `tedge-onboard`, and confirm the device joins Wi-Fi, enrolls, and connects in Cumulocity +- [x] 6.1 Flash the Zephyr sample with the onboarding overlay to the ESP32-S3-DevKitC on this computer, run `tedge-onboard`, and confirm the device joins Wi-Fi, enrolls, and connects in Cumulocity - [ ] 6.2 Run it again without `--reonboard` (refused), and with it (new certificate) - [ ] 6.3 Check that ESP Web Tools or another standard Improv Serial client can at least set Wi-Fi on a shell-less build - [ ] 6.4 Remove the test device from the tenant From ae54bd6a6cbc66494ccee146c4d107057558a709 Mon Sep 17 00:00:00 2001 From: reubenmiller Date: Tue, 10 Nov 2026 16:35:33 +1000 Subject: [PATCH 14/14] test(posix): run the enrollment stop test on certificate builds only The lifecycle test stops the client while it waits to enroll, and hands it a one-time password in between. A bootstrap build has no enrollment, and tedge_set_enroll_otp() rightly returns -ENOTSUP there, so the reduced CI build failed. The test is now built only when TEDGE_AUTH is c8y-ca. Co-Authored-By: Claude Opus 5.5 --- ports/posix/tests/CMakeLists.txt | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/ports/posix/tests/CMakeLists.txt b/ports/posix/tests/CMakeLists.txt index cd20c39..2068266 100644 --- a/ports/posix/tests/CMakeLists.txt +++ b/ports/posix/tests/CMakeLists.txt @@ -7,12 +7,16 @@ target_compile_definitions(tedge-port-test PRIVATE _POSIX_C_SOURCE=200809L target_link_libraries(tedge-port-test PRIVATE tedge::tedge) add_test(NAME port COMMAND tedge-port-test) -add_executable(tedge-lifecycle-test test_lifecycle.c) -set_target_properties(tedge-lifecycle-test PROPERTIES C_STANDARD 11) -target_compile_definitions(tedge-lifecycle-test PRIVATE _POSIX_C_SOURCE=200809L - _DEFAULT_SOURCE _DARWIN_C_SOURCE) -target_link_libraries(tedge-lifecycle-test PRIVATE tedge::tedge) -add_test(NAME lifecycle COMMAND tedge-lifecycle-test) +# Stopping the client while it waits to enroll: certificate builds only +# (bootstrap builds have no enrollment, and no one-time password to set). +if(TEDGE_AUTH_C8Y_CA) + add_executable(tedge-lifecycle-test test_lifecycle.c) + set_target_properties(tedge-lifecycle-test PROPERTIES C_STANDARD 11) + target_compile_definitions(tedge-lifecycle-test PRIVATE + _POSIX_C_SOURCE=200809L _DEFAULT_SOURCE _DARWIN_C_SOURCE) + target_link_libraries(tedge-lifecycle-test PRIVATE tedge::tedge) + add_test(NAME lifecycle COMMAND tedge-lifecycle-test) +endif() if(TEDGE_PROVISIONING) add_executable(tedge-provisioning-test test_provisioning.c)