Run on the Raspberry Pi Pico 2 W (hub) and Pico W (node): capability gates, RTC-less clock, BLE advertisement transport

[tyeth]

Summary

Makes the two trees run on the Raspberry Pi Pico 2 W (hub) and Pico W (node) under CircuitPython's Zephyr port — firmware tyeth/circuitpython ci/pico2w-ble-assets @ 94bfd47b5f, binaries on prerelease zephyr-cp-ble-ci-20260909. Umbrella: tyeth/circuitpython#15; why the reviewable firmware PRs cannot link in CI: tyeth/circuitpython#16.

Nothing in this PR was run on hardware. It was verified by compiling every source with the matching mpy-cross, walking both import graphs against the boards' generated module table, measuring import costs on the unix CircuitPython build, and executing the pure-logic modules and both entry points there with stubbed radios. The bench plan in pico_w_zephyr.md is what turns "should" into "does", with the REPL lines to expect and the failure signatures that would falsify the design.

What runs where

	Pico W (RP2040)	Pico 2 W (RP2350)
heap left after the firmware's static RAM (from the .elf)	42,292 B	303,336 B
role	node only: node_lite	hub (hub_main), or a node
node → hub	BLE advertisement broadcast (new)	receives BLE advertisements; POST /api/ingest over WiFi
user access	none	BLE UART (web-BLE) + HTTP portal on your home WiFi
deep sleep / ESP-NOW / eInk / SD / softAP	none of them	none of them

The repo's name promises deep sleep and ESP-NOW; on these boards it delivers neither, and the code now says so at boot instead of dying at import.

What the firmware turned out to lack — beyond espnow/alarm

Verified in the port sources and the linked ELF, not assumed:

• No time source. rtc = false means time.time() and zero-arg time.localtime() raise RuntimeError: RTC is not supported on this board (the weak rtc_get_time_source_time() is linked; nm shows W). The old code called time.time() in 34 places. caps.py keeps an epoch offset against time.monotonic_ns(); time.localtime(secs) / time.mktime() are pure conversions and keep working.
• wifi.radio.start_ap() is an empty stub — it returns without starting anything and ap_active stays False (common-hal/wifi/Radio.c, body commented out). The old code would have believed it was serving a captive portal on 192.168.4.1. Both call sites now check the radio and raise if nothing came up; where caps.AP_SUPPORTED is False they skip the attempt and say to use CIRCUITPY_WIFI_SSID.
• No custom-pin busio: busio.SPI(...)/busio.I2C(...) raise NotImplementedError; only devicetree buses exist and board.SPI is the CYW43439 radio's own PIO bus. So no eInk, no SD card, decided before their modules import. Sensor I2C is board.I2C0 (SDA GP4, SCL GP5).
• CONFIG_NET_MAX_CONTEXTS=6: six sockets total including the listener; net_wifi caps clients at 4 here (caps.MAX_SOCKETS - 2).
• _bleio: legacy advertising only, CONFIG_BT_MAX_CONN=1 — which is why the node→hub BLE path is a broadcast, not a connection.

Every import in both trees against the module table (tools/board_budget.py):

collector/hub_main.py   HARD: none.  soft: analogio, espidf, espnow, rtc (all guarded/conditional/deferred)
node/node_lite.py       HARD: none.  soft: analogio, rtc
node/node_full.py       HARD: alarm, espnow, rtc  -> the ESP32 node; never loaded on a board without espnow+alarm

Changes

• Entry points become capability gates (collector/code.py, node/code.py, 2 KB each). CircuitPython loads a module's whole bytecode before running its first line, so the check has to live in a file small enough to load anywhere. The bodies moved verbatim: collector/code.py → hub_main.py, node/code.py → node_full.py (pure renames in the first commit). The collector refuses below 128 KB free or without wifi/socketpool, prints why, and idles with the REPL reachable. The node loads node_full where espnow+alarm exist and node_lite otherwise. Capability and free memory, not board ids: an ESP32 build missing a module gets the same treatment.
• caps.py (identical in both trees): the probes and the RTC-less clock. Every time.time()/time.localtime()/rtc.RTC().datetime = in the collector goes through it; unchanged behaviour on boards with an RTC.
• BLE advertisement transport: envadv.py (18-byte reading in one Manufacturer Specific Data structure, 25 bytes on air), node/net_bleadv.py (raw _bleio, non-connectable), collector/net_blescan.py (1 s passive scan every 5 s with the AD prefix as filter, de-dup by address+seq, feeds the existing take_node_packet() with mac=None). No confirmation and no cfg push over this path — stated in both files. Nodes appear as ble-XXXX; zones maps them.
• node/node_lite.py: read → advertise ble_adv_s → optional WiFi POST (gated on wifi_min_free, so off on a Pico W) → time.sleep() the rest of interval_s. An awake loop, because that is all the port offers; the power cost is written down (tens of mA vs tens of µA).
• net_espnow is imported only when espnow exists (a _NoHub keeps the counters/API); no-SPI boards skip display and SD before importing them; net_wifi syncs NTP through caps.set_epoch(time.mktime(ntp.datetime)).
• tools/board_budget.py (import graph + heap floor per board), tools/test_envadv.py; pico_w_zephyr.md; README pointers. bugs_issues_and_todos.md deliberately untouched.

RAM: why the Pico W gets node_lite and not the existing node or the BLE UART portal

Measured on the 64-bit unix build (gc.mem_free() delta on import; ~2.0× a module's .mpy size there, so ~1.2× on 32-bit — the factor board_budget.py uses):

	.mpy	64-bit measured	32-bit estimate
adafruit_ble chain that net_ble needs	17.8 KB	50.8 KB	~30 KB — most of the Pico W
old node/code.py alone	15.3 KB	—	~18 KB, before a driver or net_ble
node_lite + caps + envadv + node_sensors + net_bleadv	12.4 KB	—	~15 KB
adafruit_scd4x + bus_device	6.1 KB	15.1 KB	~9 KB
adafruit_sen6x	9.2 KB	22.0 KB	~13 KB
adafruit_requests + connection_manager	10.6 KB	19.5 KB	~12 KB
hub_main tree (17 modules)	71.2 KB	—	~85 KB (+ BLE ~30, drivers, stores) vs 303 KB

So: an SCD4x node_lite should fit a Pico W with ~15 KB to spare; a SEN6x node is marginal; anything with adafruit_ble or the WiFi POST path on top does not fit, and node/code.py says so below 24 KB free. The 42 KB is the ceiling — the supervisor's own allocations come out of the same pool, so step 0 of the bench plan is reading gc.mem_free() on a flashed Pico W. Deploy .mpy (PR #10's tools/build_bundle.sh): compiling a 10 KB source on a 40 KB heap is the peak that fails.

Firmware

One change, on its own branch, built locally, not run: zephyr-cp-pico2w-net-contexts (one commit on ci/pico2w-ble-assets) raises the Pico 2 W to CONFIG_NET_MAX_CONTEXTS=12 / CONFIG_NET_MAX_CONN=16 — FLASH unchanged 1,146,096 B, RAM 229,144 → 231,112 B (+1,968 B). Pair it with "max_sockets": 12 in config.json. The prerelease UF2s do not include it. The existing WiFi+BLE stack sizing (SYSTEM_WORKQUEUE_STACK_SIZE=4096, BT_TX_PROCESSOR_STACK_SIZE=4096, HW_STACK_PROTECTION=y) was left alone: nothing off-hardware can show it insufficient, and guessing larger numbers would hide the fault that would prove it. west update was never run; the module clones are on their feature branches, 0 dirty.

Verification (no hardware)

• mpy-cross (10.3.0-50-g94bfd47b5f, same tree as the firmware): all 32 device sources compile; PR Build node/ and collector/ with mpy-cross + circup; record results #10's tools/build_bundle.sh run against this branch stages every library and compiles the source-only SEN5x driver, same result as Build node/ and collector/ with mpy-cross + circup; record results #10 recorded.
• tools/test_envproto.py: 21/21 under CPython and under the unix CircuitPython build (first time envproto has run under MicroPython semantics off-board). tools/test_envadv.py: 26/26 on both.
• Executed on the unix CircuitPython build with stubbed _bleio/wifi/board (the wifi stub's start_ap() is a no-op exactly like zephyr-cp's): node/code.py → node_lite runs cycles with the sim sensor and a flat heap; collector/code.py → hub_main brings up — _NoHub, the AP check firing, scanner on — and enters its main loop.
• native_sim was not usable here (no i386 toolchain on the host, no container runtime, and the port's container init runs west update); the unix build stood in for the pure-logic paths. It proves logic, not radios.

Untested, explicitly

Scanning while the hub advertises its own UART service (the receiver reports a failing start_scan once and keeps trying); WiFi station + HTTP + BLE on the shared gSPI bus at the same time; whether a Pico W's supervisor leaves the ~25 KB node_lite needs; the CYW43 controller's behaviour on a long advertising duty cycle. Each has a REPL signature in the bench plan.

Also corrects the record on #10 (comment there): both board builds are green with UF2 artifacts and zephyr-tests / zephyr is green on a branch off current main.

🤖 Generated with Claude Code

[tyeth]

Hardware results and corrections to this PR's premises

Everything below was measured on a Pico 2 W (raspberrypi_rpi_pico2_w_zephyr) and, where stated, a Pico W, after this PR's doc was written. Five of its premises have moved. Nothing here was reachable from mpy-cross and stub-radio runs, so this is not a criticism of how the PR was arrived at — but parts of pico_w_zephyr.md are now wrong, and one of them means the hub loop as written could not have run.

1. start_scan(timeout=) was silently ignored — the hub loop could not have worked

Zephyr's start_le_scan_legacy() never reads bt_le_scan_param.timeout; only the extended-scan path passes a duration to the controller, and CONFIG_BT_EXT_ADV=n is forced because the CYW43439 has no extended advertising. Scans never ended. A timeout=3 scan was still yielding reports at 57 s / 1600 reports.

collector/net_blescan.py scans for 1 s every 5 s, so on the firmware this PR targets the hub's first scan would have run forever and the loop would never have reached a second iteration. Confirmed by gdb: main thread parked in scanresults_next() with done=false, bt_dev.flags showing SCANNING, ncmd_sem.count=1, gSPI bus mutex free — nothing faulted, the scan just never stopped.

Fixed in tyeth/circuitpython#20 (deadline enforced from the main thread's background task): timeout=3 → 3.0 s, 1 → 1.0 s, 0.5 → 0.53 s. Re-verify the scan/idle cadence first when testing this PR.

2. BT_MAX_CONN=1 was Zephyr's default, not a controller limit

This PR's connectionless advertisement-only node→hub transport (node/net_bleadv.py, 25 bytes on air, no delivery confirmation, no config push) was chosen because only one connection appeared available. Nothing in the port had set CONFIG_BT_MAX_CONN — it was Zephyr's default of 1, and the CYW43439 is not the constraint. It is now 4 (tyeth/circuitpython#19).

So the premise for dropping the message-id + CRC-16 delivery confirmation from #2 is gone and the transport choice can be revisited. Caveat: multiple simultaneous BLE connections have not yet been tested on this controller — that is now an open question rather than a known limit.

3. start_ap was not a missing binding — the driver was broken. It works now, but APSTA does not

The doc reads the stubbed common_hal_wifi_radio_start_ap as the whole story. Underneath, AIROC's ap_enable composed band and bandwidth bits into a chanspec that WHD's whd_wifi_init_ap() re-wraps in CH20MHZ_CHSPEC() — channel 1 became 0xd001 (5 GHz) and every AP start failed with WLAN_BADCHAN/EAGAIN. Fixed on the zephyr fork, plus a second defect where a station's DEAUTH took the AP's own interface dormant.

softAP is now implemented (tyeth/circuitpython#22): 192.168.4.1/24, DHCPv4 server auto-started, RFC 8910 option 114. Verified with the ESP32-C6 as station — join in 7 s, lease 192.168.4.2, ping 5–8 ms, join/leave/rejoin tracked, survives a soft reboot. TLS 1.2 with an RSA-2048 PEM cert returns HTTP/1.0 200 OK in 2.0 s.

New constraint this PR does not know about: AIROC runs AP and STA on a single net_if — APSTA returns -EBUSY. The hub cannot serve a setup AP while joined to home WiFi; the two modes are mutually exclusive. max_connections is controller-fixed and not honoured as a parameter.

4. CONFIG_NET_TCP had never been enabled — every HTTP/TLS claim was untested by construction

ssl and socketpool being present in the board info implied a working TCP stack. CONFIG_NET_TCP was off for both Pico W boards, so no SOCK_STREAM socket had ever succeeded: socket() failed with EPROTOTYPE, which the port misreported as "Out of sockets". Three further defects had to be fixed before a handshake completed, including a failed-socket finaliser jumping into cdc_acm_1's data through a short vtable (usage fault, halt), and mbedTLS sizing the TLS 1.2 premaster from a dummy ECP_MAX_BITS 1 when p256-m serves P-256. All in tyeth/circuitpython#21.

This affects collector/net_wifi.py and collector/net_captive.py directly: the socket ceiling and portal paths in this PR have never actually executed on this hardware.

5. The heap table is wrong in both directions

this PR's doc	measured
303,336 B free (520 KB − static)	too high — ignores supervisor + GC bookkeeping
—	gc.mem_free() at a bare REPL: ~70,700 B (the initial heap; the split heap grows on demand, rising to 103,808 B mid-allocation)
—	212,992 B cumulative bytearray(8192) to MemoryError — the real capacity
—	largest single contiguous allocation: between 50,000 and 100,000 B

port_heap_init() skips its TLSF pool when picolibc's CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE=-1 arena covers the same region, routing allocations through Zephyr malloc. Capacity is therefore chunked: object count is fine, single large buffers are not. Budget against ~208 KB with nothing over ~50 KB — that rules out whole-page HTML buffers and large TLS records, not hub_main's object graph.

Also: frozen modules landed (tyeth/circuitpython#23) — adafruit_ble + 2 submodules cost 10,384 B frozen vs 21,792 B from .mpy, for +28,336 B flash and zero static RAM. That reopens this PR's "no BLE UART portal on the Pico W, ever" conclusion, though Pico W firmware RAM has since risen to 88.41 % because TCP costs ~9.8 KB, so the budget moved both ways.

What held up

• rtc = false: time.time() → RuntimeError('RTC is not supported on this board'). The caps.now() epoch-offset approach is right.
• start_ap() returning without error while ap_active stayed False — exactly as described (cause was deeper, see 3).
• busio.I2C(scl, sda) → NotImplementedError('Use device tree to define I2C devices'); board.I2C0/I2C1/SPI/UART exist as callables.
• espnow absent, alarm/watchdog false.
• BLE advertisement reception works: 925 reports from 20 distinct devices in one scan, names resolving (TUYA_, Sinilink-APP, LE-Ninja), so scan RX and active-scan TX are both fine.

Also relevant to testing this PR

Release builds used to lose USB entirely under BLE load — UDC_RPI_PICO_STACK_SIZE was raised to 2048 in debug.conf only, so release images kept Zephyr's 512-byte default (tyeth/circuitpython#18, fixed in #19). And BLE bring-up after a warm SWD reset with patchram re-download has been seen to fail and halt on BT_ASSERT; that is being worked separately and would make a bench run of this PR unrunnable until fixed.

A full bench-test task brief will be attached separately.

🤖 Generated with Claude Code

[tyeth]

TASKbenchtestcollectorpr11.md