Skip to content

feat: set devices up over USB with tedge-onboard - #8

Merged
reubenmiller merged 14 commits into
mainfrom
feat/serial-onboarding
Nov 10, 2026
Merged

reubenmiller merged 14 commits into
mainfrom
feat/serial-onboarding

Conversation

@reubenmiller

Copy link
Copy Markdown
Contributor

Setting a device up is now one guided command on the computer it's plugged into. tedge-onboard gives the device the computer's Wi-Fi network, registers it in Cumulocity through the user's go-c8y-cli session, and waits until it's connected. It's written for users who aren't technical: no arguments, no ports or IDs to type, choices with the likely answer selected, and plain-language messages that say what to do next.

The design and specs are in openspec/changes/serial-onboarding/.

What's in it

  • provisioning/ is an optional library beside the client (TEDGE_PROVISIONING, include/tedge/provisioning.h). It speaks Improv Wi-Fi, so ESP Web Tools and Improv phone apps can at least set Wi-Fi, plus three tiny-edge commands:

    • info (0xE0);
    • onboard (0xE1): the tenant and one-time password, applied through the public API;
    • identify (0xE2).

    It also answers Improv's network scan (0x04). Wi-Fi stays the application's job, through a hook. The client doesn't know the library exists, as the tedge-c8y-onboarding spec requires.

  • The Zephyr port connects Improv Serial to the console. With a shell on the console, tedge improv lends it to the tool until it goes quiet. Otherwise the console speaks Improv from boot. The port also provides a ready-made Wi-Fi scan hook. The minimal sample has overlay-onboarding.conf.

  • tools/onboard/ is the tool, in Go, for macOS, Linux, and Windows:

    • It finds the board by asking each USB serial port, without resetting it.
    • It lists the networks the board can see. It marks the computer's network, and preselects it when the OS hides its name, as macOS 26 does.
    • It reads the Wi-Fi password from the Keychain, NetworkManager, or the WLAN profiles.
    • It registers the device with a fresh one-time password.
    • --yes turns it into a script tool.
  • Core:

    • tedge_stop() now interrupts the wait between enrollment attempts. Before, it gave up after 10 s and reported success with the thread still running. It returns -EBUSY if the thread really is stuck.
    • New read-only getters: tedge_get_device_id() and tedge_get_c8y_url().
  • CI:

    • The POSIX workflow builds and tests the library.
    • A new tools workflow checks and builds the tool.
    • Releases attach the tool's archives.

Interfaces

  • Public API: the two getters, and tedge_stop() may now return -EBUSY.
  • Port interface: unchanged.
  • Messages to Cumulocity: unchanged.

Cost

Zephyr minimal sample, with and without the library:

Board Flash RAM
ESP32-S3-DevKitC +5 KB about 5 KB (4 KB of it the worker's stack)
ESP32-C6-DevKitC +6 KB about 5 KB

Testing

  • POSIX host tests:
    • Every command, damaged and split frames, log text between frames, the scan, and the hand-over to a real client.
    • A stop and start while enrolling, with new onboarding data in between.
  • Go tests:
    • The codec, and the client against a fake device.
    • Wi-Fi parsing per OS, the go-c8y-cli wrapper (including its exit code 4 for "not found"), and the Wi-Fi list.
  • Zephyr:
    • All 31 Kconfig cases pass, 2 of them new.
    • The onboarding overlay builds for the ESP32-S3 and ESP32-C6.
    • The native_sim build is unchanged.
  • On an ESP32-S3-DevKitC, with the minimal sample:
    • The tool set the board up from scratch and it connected.
    • A restart from Cumulocity went through and was reported.
    • It came back on its stored Wi-Fi and certificate.
    • A network that doesn't exist ends in "could not connect".
  • On the same board, with microcontroller-devices' modbus-server-tedge-full:
    • Built against this branch (follow-up PR there).
    • The board describes itself by its app name and version.
    • The board answers in milliseconds.
    • The scan found the network the board can reach.

Not in this PR

  • Improv over BLE: phase 2, task group 7.
  • Still open from the task list:
    • A walk-through with a new user (4.9).
    • The re-onboarding checks (6.2).
    • A standard Improv client against a shell-less build (6.3).
    • Removing the test devices (6.4).
  • Unsigned macOS binaries: they aren't signed or notarized, so the README explains Gatekeeper's "Open Anyway". Signing would remove that step.

Please merge with "Create a merge commit"; the commits are conventional commits.

🤖 Generated with Claude Code

reubenmiller and others added 14 commits November 10, 2026 14:34
A host tool, tedge-onboard, that onboards a device on USB with the
computer's Wi-Fi and the user's go-c8y-cli session. On the device, an
optional provisioning library beside the client speaks Improv Wi-Fi,
plus two tiny-edge commands, over serial first and BLE later.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The enrollment loop slept up to 60 s between attempts, and nothing woke
it. tedge_stop() gave up its 10 s join and reported success with the
thread still running, so a following tedge_start() put a second thread
on the same stack. The wait now ends when a stop is requested.
tedge_stop() returns -EBUSY when the thread does not end in time, and
the client stays marked as running.

A second tedge_start() in the same boot no longer declares the
client's own parameters and log type again.

A POSIX test stops and starts the client while it is enrolling, with
new onboarding data in between, as a provisioning library will.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
tedge_get_device_id() returns the external ID that tedge_init()
settled on. tedge_get_c8y_url() returns the tenant host the client
uses. Both are read-only. A provisioning tool needs them to tell the
user which device it is talking to and where it reports.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
An optional library beside the client (TEDGE_PROVISIONING) that answers
the Improv Wi-Fi protocol. It hands the Wi-Fi network to the
application through a hook, and the Cumulocity tenant and one-time
password to the client through the public API. The client stays
unaware of it.

It answers Improv's Wi-Fi settings, current state, and device info
commands, plus three of tiny-edge's own:
- info (0xE0): the protocol version, the device ID, the authentication,
  the tenant, and the client's state;
- onboard (0xE1): the tenant and password; the library stops the
  client, sets them, and starts it again, and refuses a connected
  device unless the re-onboard flag is set;
- identify (0xE2): a hook to blink the board, and a log line.

The RPC layer is shared by every transport. Improv Serial framing is
here too: the port feeds the console's bytes, and the library skips
anything that is not a valid frame, so log lines and typed text can
share the console. Every answer is written from one worker thread.

The library follows the core's portability rules, is compiled by the
host-compile check, and is tested on the POSIX port. The tests cover
every command, damaged and split frames, log text between frames, and
the hand-over to a real client.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
With TEDGE_PROVISIONING, the provisioning library is built.
TEDGE_PROVISIONING_SERIAL connects it to the console:
- With the shell, "tedge improv" lends the console through the shell's
  bypass mode, until no valid frame has come for
  TEDGE_PROVISIONING_SERIAL_IDLE_S (60 s). Each frame is written raw,
  through the shell's transport, while holding the lock its log output
  takes, so a log line never lands inside a frame.
- Without the shell, the console is polled from boot.

The minimal sample's overlay-onboarding.conf builds it with the shell
and Zephyr's Wi-Fi credentials store. The sample's Wi-Fi hook joins the
network, stores it, and joins it again on the next boot.

On an ESP32-S3-DevKitC, the library answers info, device info, and
identify within milliseconds. A network that does not exist ends in
"unable to connect". The library costs 5 to 6 KB of flash and about
5 KB of RAM, 4 KB of it the worker's stack. Twister builds the overlay
for the ESP32-S3 and ESP32-C6. Two Kconfig cases cover the new options.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
A guided tool for people who are not technical. Run with no arguments,
it:
- checks the go-c8y-cli login, and offers go-c8y-cli's own session
  picker when there is none;
- finds the board by asking each USB serial port of a known vendor,
  opened without resetting it, and offers a list (with "make one
  blink") when there are several;
- proposes the computer's Wi-Fi network and reads its password from
  the Keychain, NetworkManager, or the WLAN profiles, asking only when
  that fails;
- asks for the device's name and one confirmation, then registers the
  device through go-c8y-cli with a fresh one-time password, gives the
  board its Wi-Fi and Cumulocity details over Improv Serial, and waits
  until the device is connected.

Every failure says what to do next and offers to retry. With --yes,
questions become errors that name the option that answers them, for
scripts. It builds for macOS, Linux, and Windows; macOS needs cgo for
the USB port list. Go tests cover the codec, the client against a fake
device, the Wi-Fi parsing per operating system, and the go-c8y-cli
wrapper. The hardware tag runs checks against a real board.

The repository ignored all of /tools/, for the west modules a
workspace may put there; tools/onboard is now excepted.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The tools workflow checks formatting, runs vet and the tests, and
builds the tool for Linux, Windows, and macOS. A release builds the
same archives (tools/onboard/build.sh) and attaches them to the GitHub
release.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The library answers Improv's "request scanned Wi-Fi networks" (0x04)
through a new application hook, wifi_scan. For each network it reports
the SSID, the signal, and whether it needs a password: strongest
first, each SSID once, without hidden networks. An empty result ends
the list.

A tool can then offer only networks the board can reach. Recent macOS
releases hide the current network's name from programs, so the
networks the board sees and the computer knows are how the tool finds
it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
tedge_provisioning_wifi_scan() (<tedge/provisioning_zephyr.h>) scans
on the first Wi-Fi interface and collects the results from the network
management events. Scanning is the same in every Zephyr application.
Joining and storing a network stays with the application. The minimal
sample uses the hook.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Before asking for the Wi-Fi network, the tool asks the board which
networks it sees and lists them, strongest first, with their signal.
It marks the computer's current network when the operating system
names it, and the networks the computer knows. When the system hides
the current network, as macOS 26 does from programs without Location
Services, the strongest known network the board sees comes first.
That is almost always the computer's current network.

Networks the board cannot see are left out, so a 5 GHz network is no
longer offered by mistake. If the system names a current network the
board cannot see, it is still listed, with the reason. An open network
needs no password. Firmware that cannot scan gets the computer's
networks, as before.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The console glue picked the shell's bypass whenever CONFIG_SHELL was
set. A build whose shell has only other backends (the dummy one the
remote shell command uses) has no shell reading the console, so Improv
frames were never read. The deciding option is now
CONFIG_SHELL_BACKEND_SERIAL.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Improv's device info took the firmware name and version from the port,
which only knows the RTOS's defaults ("zephyr", "0.0.0") unless the
build names the image. The hooks can now carry the application's own
name and version, so a board describes itself as, say, "modbus-server
0.10.0".

The tool's hardware tests gain a scan check and a console dump, for
when a board does not answer.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The scan task, and the end-to-end run on the ESP32-S3 with the
minimal sample. It connected, and a restart from Cumulocity went
through and was reported.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The lifecycle test stops the client while it waits to enroll, and
hands it a one-time password in between. A bootstrap build has no
enrollment, and tedge_set_enroll_otp() rightly returns -ENOTSUP there,
so the reduced CI build failed. The test is now built only when
TEDGE_AUTH is c8y-ca.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@reubenmiller
reubenmiller merged commit 07a8559 into main Nov 10, 2026
14 checks passed
@reubenmiller
reubenmiller deleted the feat/serial-onboarding branch November 10, 2026 06:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant