miloco v2026.9.11

Since v2026.8.6: 181 commits, 15 PRs, 267 files changed (+36,522 / −3,028).

Camera IP direct-connect is back. v2026.8.6 reverted it in full because the native library shipped alongside it could crash the camera process with SIGSEGV on some older camera models. The library is fixed (miss 3.2.14) and direct-connect returns in this release, together with a silent-stream watchdog that reconnects a camera whose connection is alive but producing no frames.

🎯 New Features

• Rules and tasks re-architected — a task can now carry several rules, and entering and leaving no longer have to be opposites (@Zirconi): the biggest change in this release. A rule used to judge the condition, hold the state, decide entry/exit, and run the action all at once, so one task meant exactly one rule and the exit condition had to be the negation of the entry condition. Responsibility is now split by output: rules produce edges only, while state, actions, ledger, and notification belong to the task. mode ∈ {event, state} is replaced by direction ∈ {enter, exit, session, milestone}, which says which action slot an edge feeds. New capabilities: non-reciprocal tasks (a V-sign enters, a wave exits — two unrelated rules on one task), OR aggregation (several enter rules on one task; the second is absorbed as already_in_state and the action fires once), exit-type rules whose own entry edge is mapped to a leave signal, and milestones decided server-side from the record source against a threshold instead of an in-memory flag. Milestone rules are derived, generated by the server from on_target action + duration accounting + target_minutes, so the agent's existing rule create --on-target-desc path keeps working unchanged. Database user_version 3 → 4 as expand-contract phase A: columns are added, none removed; old rows migrate event → enter and state → session, with actions moved from the rule onto the task. Tasks that would become invalid (an exit with no way in, a session mixed with other rules, an incomplete on_target setup) are suspended during migration with the reason recorded.
• A single in-process view of what the home looks like right now (@Zirconi): a layered key-value state container at iot/device/<did>/…, with subscription by path pattern (* for one segment, ** for any depth), writes allowed from any thread and callbacks delivered on the main event loop. Device state is now kept live in it: the current home's readable properties are pulled once at startup, and after that online/offline pushes and property-change pushes keep it current, with a re-pull when a device comes back online to fill in leaves it was missing. The scope is the currently enabled home only — switching account, unbinding, or switching home bumps a scope generation, cancels the in-flight alignment, clears the container, and starts over, so writes from the old scope can no longer land. Offline devices get only their online flag written, never cached properties, so the container never reports stale readings as fresh. Consumers such as agent context and rule triggers come later.
• Rule actions can trigger Mi Home scenes directly (@Zirconi): previously a scene could only be described in on_enter_desc and handed to the agent. Measured on a live rule (6 triggers), the first action landed in 7.0–13.5 s via the agent versus about 0.5 s through the engine, at 240k–310k tokens per trigger versus zero. Scenes also get proper attribution now — the agent path recorded them as source='cli' with no rule id, indistinguishable from someone typing the command by hand. Scene actions are forced non-idempotent with a cooldown of at least one minute, since a scene has no readable current value to compare against.
• Account-wide device online/offline subscriptions (@HCl8): device list now reflects online state in real time instead of only scope camera list. Bind and unbind events subscribe and unsubscribe directly, subscription reconciliation for meta / state / scene shares one concurrency limit and one coalesced background loop, and after an mips instance is rebuilt (re-OAuth or token refresh) the subscription mirror is rebuilt from what the SDK actually replayed — leaving it stale in either direction would either lose a device's pushes permanently or leak a subscription on the broker.
• Token usage accounted by model name plus Base URL, and the Models page rebuilt around it (@ExWang): the same model name can be configured against different endpoints, and usage used to merge them into one row — worse, daily rollup keyed on (date, model, call type) summed them and then deleted the source rows, silently and unrecoverably. Identity is now the (model, endpoint) pair across recording, reading, grouping and clearing, with a migration that runs in one transaction and fails backend startup rather than leaving a half-converted table. Rows from before the migration are labelled in the UI as not having recorded a URL rather than being back-filled with a guess. The page becomes two columns side by side, with collapsible cards, a configurable refresh interval, a chart colour set that passes contrast and colour-blind separation in both themes, and clearing by time range or by individual entry instead of all-or-nothing.
• Camera IP direct-connect, with a silent-stream watchdog and cross-NAT diagnostics (@HCl8): the native library is updated to miss 3.2.14 across all four platforms, fixing the heap-corruption crash that forced the v2026.8.6 revert. Direct-connect works across subnets — the LAN IP is passed to the camera at creation, unicast probes are sent to every camera with a known LAN IP rather than relying on broadcast, and a camera already connected counts as reachable. A camera whose connection is alive but emits no frames is now detected and reconnected, using two thresholds: 30 s since the last frame for a channel that has produced frames, and 90 s since subscription for one whose first frame never arrived.
• HEIC/HEIF photo upload (@ExWang): iPhones have stored HEIC by default since iOS 11, and every upload path rejected it outright. Decoding now goes through one entry point with a Pillow/libheif fallback, wired into all user-supplied-bytes endpoints. This also closes a silently broken path that had nothing to do with HEIC support: HEIF and MP4 share the ISO BMFF container, so a still photo was being classified as a video, and ffmpeg exposes a HEIC as dozens of separate tiles rather than stitching them — the pipeline would run to completion on a single 512×512 tile and report "no valid subject" with no error anywhere. Format is now decided by container brand at the CLI, at pet observe, and when re-checking the client-declared media kind.
• Smart Crop can be turned off per camera (@Molly-3000): whether cropping helps is a per-viewpoint judgement — a narrow doorway camera gains little, a wide living-room camera gains a lot. A third per-camera gate joins the two global ones, stored as a deny-list keyed by composite did so the two channels of a dual-lens camera can differ. Defaults are unchanged, the engine reads it per perception window so changes take effect on the next window without a restart, and scope camera list now reports both the stored preference and whether all three gates are actually open.

🧠 Perception Stability

• Empty windows no longer reach the model (@Zirconi): when a video track had no data for a window — a decoder waiting for a keyframe, a buffer overflow flush, a reconnect — the window still went to omni, the encoder added no video block, and the model was effectively asked what was happening in the scene with nothing but a schema to go on. The fabricated caption then persisted, pushed over SSE, and reached the agent indistinguishably from a real one. In one 7.5-hour production log covering 12,158 windows, 6 had zero frames, 3 of those passed the hold gate, and all 3 results were invented — one produced a suggestion that was dispatched to the agent. Snapshots with neither frames nor audio are now dropped per device rather than per batch, so one blind camera in a room with several does not slip through.
• Per-frame detection state now crosses the identity boundary, and two coasting gates were added (@ExWang): the tracking layer correctly marked whether a frame had a real detection hit and at what confidence, but the semantic track type had nowhere to put it, so the identity engine fell back to defaults — every active track was treated as freshly detected at full confidence. A coasting box sits at the last true match and grows stale, so this meant crops from stale boxes could be written into the identity library, names could be pinned to the wrong person when several people share a frame, real people could be judged occluded and denied library entry, and recognition calls were spent on background. The fields are now carried across the seam, the stranger pool and the offline-video sampler both check for a real hit before taking a crop, and identity-drift self-check compares the evidence itself so an unchanged window no longer votes twice toward revoking a confirmed identity.

⚙️ Improvements: CLI / Web

• OpenClaw plugin works on OpenClaw 2026.8.x (@HCl8): the host moved from a session store to session entries with a delivery route, so notify could not find a session; the plugin now picks a path by runtime capability and reads the delivery target from either shape. Published artifacts no longer carry devDependencies.openclaw, which made the host crash while resolving plugin dependencies, and both install.py and sync-to-remote.sh probe for --accept-capabilities and pass it when the host requires consent. Verified against 2026.4.29 (correctly refused by the version gate), 2026.5.2, 2026.6.11, 2026.8.2 and 2026.9.1.
• install-hermes.sh syncs perception models by content (@Zirconi): the step skipped any model file that already existed, and an outer gate abandoned the search entirely if any .onnx was present, so re-running the installer after models changed in the source tree wrote nothing while the log said it had skipped existing files. Files are now compared by content and written through a temp file plus rename, each written filename is logged, and the closing check verifies each required model is present by name rather than asking whether the directory contains any .onnx at all.
• Task detail shows runtime state, and configured pushes no longer display as "no action" (@Zirconi): the Web UI consumes direction and task-level actions.
• Brace-quote shell variables that precede non-ASCII text (@idootop): Apple's /bin/bash 3.2.57 in a UTF-8 locale reads the first byte of a following non-ASCII character as part of the variable name. With set -u the script dies with a misleading unbound variable, usually on the branch that was about to print the real error; without it, the variable silently expands to nothing and eats a byte. Two of the five occurrences were on normal, non-error branches, so bash scripts/build.sh --packages <subset> failed every time on macOS. A CI gate now scans tracked shell scripts for the bare form.
• time-compute reads the local clock when --now is omitted (@Zirconi).
• Web fixes across the usage and perf views (@ExWang): a shared width-measuring hook lacking a fallback guard and flashing on the first frame, an endpoint rendered as two different abbreviations in the tooltip and the table, confirmation dialogs describing deletions that would not happen, y-axis tick labels overlapped by grid lines in a narrow container, and seven opacity utility classes that compiled to nothing.
• Rule and admin counts unified on "effectively enabled" across the list, the total and the enabled count (@Zirconi).

🔒 Security / Supply Chain

• Image decoding hardened along with the format work (@ExWang): every decode of user-supplied bytes runs in a worker thread — a source-scanning test fails on any bare call — since HEIC decoding takes hundreds of milliseconds on a single-process asyncio backend that is also transcoding live video. cv2.imdecode raises on images that exceed OpenCV's own size limits, which a 29-byte crafted GIF header can trigger, and that path now returns a 400 rather than a 500. Stored files are verified by full decode rather than magic bytes alone, and re-encoded output is capped by purpose so the format relaxation does not remove the implicit size ceiling.

📚 Docs

• create-task skill updated for the direction model and scene triggering (@Zirconi), and feature docs rewritten to describe behaviour by direction rather than the retired mode and in-memory milestone flag.
• Comments describing removed mechanisms cleaned out of the rule, perception, task and state modules (@Zirconi).

🙏 Acknowledgements

Special thanks to @Zirconi for landing the rule/task re-architecture — the biggest change in this release — and the state container it builds on.

Thanks as well to everyone else who contributed: @ExWang, @HCl8, @Molly-3000, @idootop