What's Changed
-
Fix the permanent-failure marker never reaching the provision-failed
hook from the signing paths. mark_permanent_failure() recorded its state
in a shell variable, but the two callers that most need it --
get_openssl_sign_args() and get_sign_bootcode_key_args() -- are invoked
as$(...), and a variable set inside a command substitution dies with
that subshell. A station with no signing key configured therefore
reported PROVISION_FAILED_PERMANENT=0, so a hook filtering transient
bootstrap-phase failures discarded it as churn and left the head on its
in-progress indicator indefinitely -- exactly the case 2.3.2 set out to
fix. The marker is now a file keyed on the invoking shell's PID, which
subshells inherit unchanged, so it survives the capture while keeping
concurrent devices independent. It is cleared once the outcome has been
reported, so a later run reusing the PID cannot inherit it. -
Stop the UI service aborting at startup when it cannot establish its own
package version. The GitHub release check parsed that version with
std::stoi, which throws on a component holding no leading digits, and
nothing on the path catches it. getPackageVersion() falls back to
"unknown" whenever dpkg-query cannot answer -- which is any binary run
from a build tree rather than an installed package -- so the service
terminated during startup, before it bound its listener, whenever GitHub
was reachable enough to return a release tag to compare against. An
unreachable GitHub returns early and never reaches the comparison, which
is why this went unnoticed on stations without network access. A version
that will not parse is now treated as "no update known"; Debian revisions
and pre-release suffixes order exactly as before, so non-native builds
keep receiving update notifications. -
Make the device cards the default view on the Devices page, and move
them to the left of the view selector. The topology map led, so an
operator watching a flash had to switch to the cards on every visit --
and the tab the page remembered was written on every page load,
including the one that had merely applied the old default, so a browser
that had never expressed a preference still had one recorded. Only an
explicit click is remembered now, under a new storage key, and the graph
refits itself when its tab is revealed rather than laying out against a
hidden container. Reported by Patrick Groothuizen (#345). -
Show what a device is actually doing on its card, rather than only which
of the four stages it has reached. Each card now carries a
plain-language activity line -- "Writing root filesystem", "Updating
bootloader EEPROM" -- with a live counter for how long that step has
been running, so a stalled device is visible without opening its log.
Where few enough devices are connected for it to fit, the card also
lists that device's recent state transitions with timestamps, which is
the per-device form of the state-transition table earlier releases
showed. Those rows are keyed by USB path, so the list stops at the end
of the previous board's run rather than presenting it as this one's. The
wired MAC is shown alongside the serial once the provisioner has read it
from the device, so a finished module can be identified without
cross-referencing. -
Report provisioning progress in much finer detail. The provisioners
recorded three states per stage -- started, finished, aborted -- so
everything between them was invisible: a device spent minutes on
IMAGE-PREPARING or WRITING-ROOTFS indistinguishably from one that had
hung. Every stage now records the step it is entering, via a
record_progress() helper that keeps the stage name on the front of each
state, so the UI places a step it has never seen in the right stage
without needing to know it. That covers image preparation, signing,
storage erase and partitioning, each partition write (including one
state per partition in the IDP flash loop), and finalisation; the
customisation hooks report themselves too, so an operator script that
blocks is no longer mistaken for the provisioner stalling; and a device
queued behind another head's shared image build now says so instead of
appearing to sit still. -
Make the Devices page ignore state.db rows written before the service
started, as it was always meant to. The freshness filter bound the start
time as an integer against a text timestamp column, and SQLite sorts
every integer before any text, so the comparison was true for every row
ever recorded -- a freshly connected board inherited whatever state the
previous occupant of that USB port had finished in. Consequently a
device left connected across a service restart now shows no state until
it next changes, which is the honest answer: the service cannot know the
board still in the socket is the one those rows describe. -
Stop offering boot package generation on stations with no signing key.
Boot packages are signed, so without key material the button started
rpi-sb-image-bootimg-generator.sh, which logs "No signing key
configured", exits 0, and produces nothing -- leaving the page polling
for a package that would never appear and reporting the result as
"Not generated", the same wording as an image nobody had asked to
build. The Options page now reads the configured signing material from
a new GET /options/signing-status, classified exactly as
init_signing_context() classifies it, and withholds the control when
there is nothing to sign with; the Boot Package row says which of the
reasons applies -- no key, a PEM key naming a file that is not there,
or both a PEM and a PKCS#11 key configured at once, which the scripts
refuse to choose between -- and links to the signing key settings where
those are on screen. The status is re-read whenever a key is added,
activated or removed, so it follows the configuration without a reload.
/generate-boot-package refuses the request with 409 NO_SIGNING_KEY
rather than relying on the button being absent. The PKCS#11 token is
not opened to answer this, so a configured URI still reports a key
present while its token is unplugged. -
Check for a usable device key where secrets are stored, not at install
time. Uploading a signing key to a station whose OTP key slot had never
been programmed failed with a bare HTTP 500, the reason ("Failed to
device-wrap uploaded key") reaching only the journal. The 2.3.x
bootstrap that was supposed to prevent this ran solely from postinst,
which for any host built from an image executes in the build chroot
where the firmware crypto service is unreachable -- so it reported
"firmware crypto service unavailable; skipping key setup", exited 0,
and never ran again on the hardware. A host installed straight from a
build tree never ran it at all.The service now classifies the device key at startup and again at every
point it is asked to store a secret -- the PIN, an uploaded PEM key,
and the plaintext migration path -- so the check cannot be bypassed by
how the software arrived. The probe ends in an actual HMAC rather than
a public key read, because a slot that has never been programmed
answers for its status while holding all zeros, which is exactly the
case being detected. GET /options/device-key-status reports the
classification and the Options page explains it in place of the
signing key controls; the paths that store secrets refuse with 409
DEVICE_KEY_BLANK or DEVICE_KEY_UNAVAILABLE, naming the remedy where
there is one. -
Stop generating a device key during package installation. The previous
bootstrap did this on the premise that the key would land in a spare,
non-DEVICE slot and so cost nothing. That is not true on BCM2712: there
is exactly one key slot, id 1, and it is the device-unique slot, so
what the postinst performed was an irreversible OTP write as a silent
side effect of apt. It now only reports. Where the slot is blank the
Options page offers to generate one through a new POST
/options/provision-device-key, which requires an explicit confirmation
token in the body and is refused for any state other than blank -- a
host that already has a key returns idempotent success with no write
attempted. Note that config.txt's lock_device_private_key=1 sets
READ_LOCKED, which does not prevent generation; that is gated by the
separate GEN_LOCKED flag, so a locked-but-blank slot can still be
provisioned. -
Record why the device refused a fastboot command during IDP
provisioning. fastboot reports a device-side refusal as FAILED (remote:
'...'), but timeout_fatal_secs ran each command bare, so that line
reached the journal while provisioner.log -- the per-device log the
Devices page renders -- recorded only that a command had started.
die() echoes rather than logs, so not even the exit code landed there.
The output is now captured as well as streamed, the remote message
extracted and logged, and carried into the failure message, so the
recorded failure names its cause. Where there is no remote message the
last lines of output are logged instead. Streaming is unchanged, so
flash progress still appears as it happens. -
Validate IDP image descriptions against the schema rpi-image-gen
publishes, rather than a hand-written approximation of it. The embedded
kImageJsonSchema required only IGversion and layout -- IGmeta,
attributes, partitiontable and provisionmap were all optional, and
image-size and image-palign-bytes accepted any type at all -- so a
description missing every block the device needs was accepted at
upload and refused on the bench as "IDP:invalid description". The
schemas are now generated at configure time from a pinned
rpi-image-gen (RPI_IMAGE_GEN_TAG, v2.7.0), so an update to them
arrives as a reviewable diff instead of a failed provisioning run. -
Check the parts of a description a JSON schema cannot express, matching
what the device checks: image-palign-bytes must read as a genimage size
and be a whole number of mebibytes (no schema can tell 1M from 1MB),
IGversion must name major version 2, and every partition reference in
the provisionmap must name an entry in partitionimages. Verified
against a real rpi-image-gen artefact, which validates unchanged. -
Select the fastboot gadget for the connected device's family. A station
may now carry one gadget per family -- fastboot-gadget-pi5-family.img
and so on, built by pi-gen-micro's matching target -- and the gadget is
chosen at stage time from the rpiboot USB product ID, falling back to
the configured RPI_DEVICE_FAMILY. This matters because the gadget is
signed, cached and then pushed over USB to every board, so anything in
it the board cannot load is paid for per device: an all-devices gadget
is ~35MB against ~27MB for pi5-family, and none of the difference is
loadable on a BCM2712 part, which takes its firmware from EEPROM.
Backward compatible -- an unsuffixed image under /etc still overrides
everything, and a station carrying only the unsuffixed image behaves
exactly as before. Three byte-identical copies of get_fastboot_gadget()
are collapsed into rpi-sb-common.sh; two of them were never called. -
Record an abort when a provisioning stage dies without reaching die().
die() writes the ABORTED state, but aset -edeath exits at the
failing command and never reaches it, and cleanup() -- trapped on EXIT
in all six scripts -- ran the provision-failed hook without writing any
state. Any unguarded command that failed therefore left the device
recorded in whatever transitional state it died in, which is what the
Devices page shows, so a dead run looked like a running one
indefinitely. Observed with a truncated OS image, where a failed mount
left SB-PROVISIONER-IMAGE-PREPARING as the last word on the device. A
new record_abort_once() writes ${STATE_PREFIX}-ABORTED from every
cleanup(), guarded so die() and cleanup cannot record it twice. Triage
keeps its existing suppression: a triage that loses a race with
bootstrap during USB re-enumeration stays silent, as it did before. -
Name the reason on every path that stores a secret, not just the PEM
upload. The device-key classification described above reached
/options/upload-key, but the HSM PIN, the wrap-a-saved-key action
and the plaintext migration all flattened the same failure into a
bool on the way out of utils, so each answered with a flat 500 --
"Failed to save PIN securely", "Could not encrypt key at rest on
this device", "One or more secrets could not be wrapped" -- with the
cause reaching only the journal. An operator configuring an HSM on a
jig whose OTP slot had never been programmed was told to suspect the
PIN, or the secrets, rather than the station. All three now classify
before they write and refuse with the same 409 DEVICE_KEY_BLANK or
DEVICE_KEY_UNAVAILABLE the upload path uses. -
Carry the remedy as far as the operator. It travels separately from
the reason, as the error response's details field, and the Options
page read only the message -- so the half of the answer saying what
to do about a blank device key was discarded before it was
displayed. The PIN controls read errorData.message, a field
createErrorResponse never sets (it emits error), so those two
replaced every server-side reason with a generic fallback outright. -
Stop a device-key classification sticking for the life of the service.
deviceKeyStatus() cached its answer permanently, so a station whose
slot was programmed after the UI started -- by Raspberry Pi Connect
registering, or genkey from the command line -- kept reporting no
device key from every secret path and from the status endpoint behind
the Options banner until it was restarted, with no way to re-probe.
Failures are now cached for five seconds. A usable key is still not
re-probed on a timer, since it sits on the hot path of every wrap and
unwrap, but neither is it trusted indefinitely: HMAC_LOCKED is
settable at runtime and persists until reboot, so a slot that refuses
an operation now invalidates the classification that promised it
would not, at the point of failure rather than at the next poll. -
Name the device the firmware crypto service actually uses. The remedy
offered for an unreachable service said to check that /dev/vcio
exists. librpifwcrypto opens /dev/vcio_crypto, a different node with
different ownership, so an operator following that advice would find
/dev/vcio present and correctly permissioned and conclude the host was
fine.
PRs
- Fixup secure-boot EEPROM signing behaviours by @tdewey-rpi in #340
- 2.3.3: Better failure reasons, IDP validation, UI elements and explicit host unique secrets by @tdewey-rpi in #347
- 2.3.3: Further host unique secret improvements by @tdewey-rpi in #348
Full Changelog: v2.3.2...v2.3.3