v0.5.5 — Advanced Sensing, WiFlow Architecture, Camera-Free Pose Training
TL;DR
Your $27 sensor kit now has spiking neural networks that learn your room in 30 seconds, min-cut person counting that actually works (fixes #348), CNN spectrogram embeddings for environment fingerprinting, multi-frequency mesh scanning across 6 WiFi channels, and the WiFlow SOTA architecture (1.8M params) for 17-keypoint pose estimation — all trainable without a camera.
What Changed (v0.5.4 → v0.5.5)
| Feature | v0.5.4 | v0.5.5 |
|---|---|---|
| Person counting | Broken (always 4) | MinCut: correct count (fixes #348) |
| WiFi channels | 1 (ch 5 only) | 6 channels (hopping 1/3/5/6/9/11) |
| Pose model | Simple 8→64→128 encoder | WiFlow: TCN + axial attention (1.8M params) |
| Pose keypoints | 5 proxy → 17 interpolated | 17 COCO keypoints (WiFlow decoder) |
| Online learning | None | SNN with STDP (adapts in <30s) |
| Environment fingerprint | 8-dim features | 128-dim CNN spectrogram |
| Multi-node fusion | Average | Graph transformer (GATv2 attention) |
| RF scanning | None | Live spectrum visualization |
| Neighbor WiFi | Ignored | Used as passive radar illuminators |
| Training pipeline | ruvllm basic | + camera-free + WiFlow + GCloud + Mac Mini |
| ADRs | 069-071 | 069-076 (8 total) |
New Capabilities
ADR-073: Multi-Frequency Mesh Scanning
ESP32 nodes hop across channels 1/3/5/6/9/11 at 200ms dwell. Neighbor WiFi networks (your printer, router, neighbors) become free RF illuminators. Null subcarriers dropped from 19% to 16%.
# Provision channel hopping
python firmware/esp32-csi-node/provision.py --port COM9 \
--hop-channels "1,6,11" --hop-dwell 200
# Live RF scan
node scripts/rf-scan.js --port 5006 --duration 30ADR-074: Spiking Neural Network
128→64→8 SNN with STDP unsupervised learning. Adapts to room in <30s without labels. 16-160x less compute than the FC encoder (event-driven, only processes changes).
node scripts/snn-csi-processor.js --port 5006ADR-075: MinCut Person Separation (fixes #348)
Stoer-Wagner min-cut on subcarrier correlation graph. Correctly counts 1 person on all 24 test windows where old firmware showed 4. <5ms per window.
node scripts/mincut-person-counter.js --port 5006
# Or replay: node scripts/mincut-person-counter.js --replay data/recordings/*.csi.jsonlADR-076: CNN Spectrogram Embeddings + Graph Transformer
CSI 64×20 matrix → 224×224 grayscale → CNN → 128-dim embedding. Same-node similarity 0.95+. GATv2 multi-head attention fuses multi-node features.
node scripts/csi-spectrogram.js --replay data/recordings/*.csi.jsonl
node scripts/mesh-graph-transformer.js --port 5006ADR-072: WiFlow SOTA Architecture
Full reimplementation of WiFlow (arXiv:2602.08661) in pure JS:
- TCN temporal encoder (dilated causal conv, k=7)
- Asymmetric spatial encoder (1×3 residual blocks)
- Axial self-attention (8 heads, width + height)
- Pose decoder → 17 COCO keypoints
- SmoothL1 + bone constraint loss (14 skeleton connections)
- 1.8M parameters (881 KB at 4-bit quantization)
Camera-Free Training (ADR-071 extended)
10 sensor signals replace cameras: PIR, BME280 temp/humidity, RSSI triangulation, subcarrier asymmetry, vibration, reed switch, kNN clusters, boundary fragility.
5-phase pipeline: multi-modal collection → weak labels → 5-keypoint proxy → 17-keypoint interpolation → self-refinement.
Validated Benchmarks
| Metric | Value |
|---|---|
| Rust tests | 1,463 passed |
| Presence accuracy | 100% |
| MinCut person count | 24/24 correct (was 0/24) |
| Inference latency | 0.012 ms (M4 Pro) |
| Throughput | 171,472 emb/s |
| WiFlow PCK@20 | 2.5% (camera-free baseline) |
| WiFlow parameters | 1,804,962 |
| WiFlow model (4-bit) | 881 KB |
| CNN spectrogram similarity | 0.95+ (same-node) |
| SNN adaptation time | <30s (STDP online) |
| Channel hopping | 6 channels, 200ms dwell |
| Null reduction | 19% → 16% |
Flash Instructions
Same firmware binary as v0.5.4 (channel hopping is NVS-configured, no reflash needed for existing v0.5.4 users):
python -m esptool --chip esp32s3 --port COM9 --baud 460800 \
write_flash --flash-mode dio --flash-size 8MB --flash-freq 80m \
0x0 bootloader.bin 0x8000 partition-table.bin \
0xf000 ota_data_initial.bin 0x20000 esp32-csi-node.bin
# Enable channel hopping (new in v0.5.5)
python firmware/esp32-csi-node/provision.py --port COM9 \
--hop-channels "1,6,11" --hop-dwell 200New Scripts (15+)
| Script | Purpose |
|---|---|
rf-scan.js
| Live RF spectrum scanner |
rf-scan-multifreq.js
| Multi-channel wideband view |
snn-csi-processor.js
| Spiking neural network processor |
mincut-person-counter.js
| Correct person counting |
csi-graph-visualizer.js
| Correlation graph visualization |
csi-spectrogram.js
| CNN spectrogram embeddings |
mesh-graph-transformer.js
| GATv2 multi-node fusion |
train-wiflow.js
| WiFlow SOTA training |
train-camera-free.js
| Camera-free 17-keypoint training |
train-ruvllm.js
| ruvllm contrastive + LoRA + TurboQuant |
benchmark-ruvllm.js
| Model benchmarking |
benchmark-wiflow.js
| WiFlow benchmarking |
benchmark-rf-scan.js
| RF scan benchmarking |
wiflow-model.js
| WiFlow architecture (pure JS) |
seed_csi_bridge.py
| ESP32 → Cognitum Seed ingest |
Learn more: Cognitum.one · Tutorial · User Guide
Full Changelog: v0.5.4-esp32...v0.5.5-esp32
Session Statistics
| Metric | Value |
|---|---|
| Commits | 24 |
| Files changed | 74 |
| Lines added | 22,231 |
| ADRs | 8 (069-076) |
| Research docs | 26 |
| New scripts | 20+ |
| PRs merged | 2 (#350, #352) |
| Issues addressed | 4 (#348, #188, #190, #268) |
| Rust tests | 1,463 passed |
| WiFlow PCK@20 | 2.5% (camera-free, improving with more data) |
| Overnight data | 65,938+ frames collecting |
| Session cost | ~$0.50 (Mac Mini electricity) |