Skip to content

Build node/ and collector/ with mpy-cross + circup; record results - #10

Open
tyeth wants to merge 6 commits into
mainfrom
ci-build-node-and-collector
Open

Build node/ and collector/ with mpy-cross + circup; record results#10
tyeth wants to merge 6 commits into
mainfrom
ci-build-node-and-collector

Conversation

@tyeth

@tyeth tyeth commented Sep 8, 2026

Copy link
Copy Markdown
Owner

Summary

Attempted a full "build" of both example trees — node/ (remote sensor node) and collector/ (host controller) — meaning: byte-compile every device source with mpy-cross, stage the libraries from each requirements-circup.txt with circup --path, and compile any library that only ships as .py. No hardware was initially involved.

Adds tools/build_bundle.sh (the reproducible version of what was run) and build_results.md (the record of this run). build/ and venv/ are gitignored.

What actually built

mpy-cross: CircuitPython 10.3.0-alpha.4-73-gf1ae373ad0 on 2026-09-08; mpy-cross emitting mpy v6.3 (x86-64 Linux), downloaded from tyeth/circuitpython Actions run 34253440312. Its output headers match the Adafruit bundle's 10.x-mpy files byte-for-byte (43 06 00 1f).

circup 3.0.4, bundles: Adafruit 20260905 (10.x-mpy), Community 20260826, good-enough-technology 202311252150.

Tree Sources compiled Rejected Requested libs resolved Transitive deps Notes
node/ 8/8 0 10/10 (+ SEN5x from custom bundle) 3 lib/ = 74 files, 484 KB
collector/ 15/15 0 9/9 4 lib/ = 66 files, 332 KB

Library versions are listed in build_results.md. Informationally, examples/ (5) and tools/ (9) also pass mpy-cross.

What failed / needs attention

Nothing failed outright. Two things that are not clean:

  • The SEN5x custom bundle has no 10.x-mpy platform. circup logs There was a problem downloading the '10mpy' platform for the 'good-enough-technology/circuitpython_goodenough_bundle' bundle. Falling back to source (.py) files. and installs sensirion_i2c_sen5x + sensirion_i2c_driver as .py trees. All 22 of those files compile with this mpy-cross, so they will load, but they cost flash/RAM that .mpy would not. circup also warns circuitpython_sensirion_i2c_driver is not a known CircuitPython library — a dependency-name mismatch in that bundle's metadata; the real dependency was still installed.
  • The committed boot_out.txt files pin circup to 10.3.0-alpha.4; circup reports 10.3.0 final is out. Libraries were resolved against the latest bundles, not whatever is on the bench boards.

Not covered: nothing was flashed or run. The runtime issues in bugs_issues_and_todos.md (C6 start_ap hard fault, BLE/AP coexistence, ESP-NOW 0x3067 NO_MEM) are unchanged and unverified here. mpy-cross only proves syntax/compiler acceptance — a CircuitPython build missing espnow, _bleio, or alarm would still fail at import.

Prerequisite CircuitPython fork branch (BLE on Raspberry Pi Pico 2 W)

The mpy-cross used here comes from CI of branch zephyr-pico2w-ble on tyeth/circuitpython (tyeth/circuitpython#4). That branch adds a CYW43439 shared-gSPI-bus HCI driver so the Pico 2 W gets BLE under ports/zephyr-cp. It is stacked on tyeth/circuitpython#5 and depends on companion module PRs:

What that CI run actually provides, accurately: run 34253440312 produces only mpy-cross artifacts — mpy-cross, mpy-cross.static, mpy-cross.static-aarch64, mpy-cross.static-raspbian, mpy-cross.static.exe, mpy-cross-macos-arm64. There is no Pico 2 W firmware artifact; the .uf2 must be built locally from ports/zephyr-cp with make BOARD=raspberrypi_rpi_pico2_w_zephyr. The run's overall conclusion is failure — its tests / zephyr job fails — so these mpy-cross binaries come from an otherwise-red run.

Relevance to this repo: both examples target ESP32 Feathers (C6/S3/S2) and are built around ESP-NOW, which the Pico 2 W does not have. The Pico 2 W BLE work is relevant to the BLE UART path only (collector/net_ble.py, node/net_ble.py, adafruit_ble) — not to the ESP-NOW mesh.

Reproduce

MPY_CROSS=/path/to/mpy-cross tools/build_bundle.sh   # or let it gh-download the artifact

🤖 Generated with Claude Code

@tyeth

tyeth commented Sep 8, 2026

Copy link
Copy Markdown
Owner Author

Pushed two commits (30f02f9, 2ae09c7) finishing the CI work that was staged on this branch.

New workflow: .github/workflows/build-bundle.yml
Runs tools/build_bundle.sh on pushes to main, on pull requests, and via workflow_dispatch: fetches mpy-cross from a tyeth/circuitpython Actions run (default 34253440312, overridable with the mpy_cross_run input), byte-compiles node/ and collector/, stages libraries with circup, fails if either lib/ ends up empty or sensirion_i2c_sen5x is missing, and uploads everything as bundle-<sha>. An optional firmware_run dispatch input re-hosts the Pico 2 W firmware artifact alongside the bundle (informational only). permissions: is contents: read only.

Action required: add a repository secret before this workflow can pass
gh run download against another repository cannot use the job-scoped GITHUB_TOKEN, so the workflow reads a secret named CP_CI_TOKEN. Create a fine-grained PAT with repository access limited to tyeth/circuitpython and the single permission Actions: Read-only (Metadata: Read-only is added automatically), then add it under Settings -> Secrets and variables -> Actions as CP_CI_TOKEN. Until it exists the first step fails with an explicit ::error:: naming the secret. Fork PRs do not receive secrets, so the job can only pass for branches in this repo. Full steps are in the new "Continuous integration (this repo)" section of build_results.md.

Revised CI-asset section in build_results.md
Separates the two tyeth/circuitpython runs: 34253440312 (Build CI on zephyr-pico2w-ble; overall failure but usable mpy-cross artifacts, no board firmware) and 34258666665 (Build board (custom) for raspberrypi_rpi_pico2_w_zephyr on the CI-only branch ci/pico2w-ble-assets). It also explains why ci/pico2w-ble-assets must not be merged: its west.yml is repointed at fork branches, and that override is the only reason the firmware is BLE-capable.

Pico 2 W firmware run is dispatched but not complete
As of 2026-09-08T17:53Z, https://github.com/tyeth/circuitpython/actions/runs/34258666665 was still queued with an empty conclusion (created 17:41Z, no runner had picked it up). There is no artifact and no pass/fail result yet; the doc says so explicitly and points readers at the run link for current state. The expected artifact, if it completes, is raspberrypi_rpi_pico2_w_zephyr-en_US-latest (firmware.uf2, firmware.elf).

🤖 Generated with Claude Code

@tyeth

tyeth commented Sep 8, 2026

Copy link
Copy Markdown
Owner Author

Correction to the comment above: the Pico 2 W firmware run has now finished, and it failed. Pushed 108d7e7 updating build_results.md accordingly.

Run 34258666665 — failure, no artifact. https://github.com/tyeth/circuitpython/actions/runs/34258666665 was picked up at ~18:11Z and died after 5m49s in the Set up port step (west update), before Build board ever ran. The upload step was skipped, so the expected artifact raspberrypi_rpi_pico2_w_zephyr-en_US-latest (firmware.uf2 + firmware.elf) does not exist and the .uf2 still has to be built locally with make BOARD=raspberrypi_rpi_pico2_w_zephyr in ports/zephyr-cp.

--- hal_rpi_pico: fetching, need revision integration-pico2w-ble
fatal: couldn't find remote ref integration-pico2w-ble
...
ERROR: update failed for project hal_rpi_pico

Root cause: the CI-only west.yml override on ci/pico2w-ble-assets points hal_rpi_pico at tyeth/hal_rpi_pico @ integration-pico2w-ble, and that branch was never pushed — the fork only has the two PR head branches, cyw43-shared-bus-ble (#1) and flash-ram-helpers-force-inline (#2). This is the known "both hal_rpi_pico commits must be cherry-picked onto one branch" prerequisite showing up in CI. The tyeth/zephyr and tyeth/hal_infineon cyw43-shared-bus-ble refs fetched fine. The two workflow fixes on that branch (tools/board_build_extensions.py + the cyw43-driver submodule checkout) were never reached, so they remain unexercised.

To actually get the asset (not done here — it means pushing to tyeth/hal_rpi_pico and re-dispatching CI on tyeth/circuitpython, which is outside this PR): push a branch to tyeth/hal_rpi_pico carrying both PR commits under the name the manifest expects (or edit the manifest to a real branch name), then re-dispatch Build board (custom) on ci/pico2w-ble-assets. Once a run succeeds, workflow_dispatch of this repo's Build bundle with firmware_run=<run id> will re-host the artifact here.

Everything else in the previous comment stands: the workflow, the CP_CI_TOKEN secret requirement (fine-grained PAT, repository access tyeth/circuitpython only, permission Actions: Read-only), the must-not-merge status of the west.yml override, and the unchanged findings (SEN5x bundle has no 10.x-mpy platform; boot_out.txt pins circup to 10.3.0-alpha.4 while 10.3.0 final is out).

🤖 Generated with Claude Code

@tyeth

tyeth commented Sep 8, 2026

Copy link
Copy Markdown
Owner Author

CI is green on this PR. Build bundle run https://github.com/tyeth/deepsleep_espnow_wifi_and_ble_env_collector/actions/runs/34262403267 passed on 108d7e7 in 25s — so CP_CI_TOKEN has evidently been configured; the two earlier runs on 30f02f9/2ae09c7 failed at the secret guard exactly as designed, before it existed. The CI run reproduced the local build byte-for-byte in outcome: mpy-cross 10.3.0-alpha.4-73-gf1ae373ad0 (mpy v6.3), circup 3.0.4, node/ 8/8, collector/ 15/15, source-only libs 22 + 4 compiled, 0 rejected; node/lib 74 files, collector/lib 66 files; artifact bundle-8a30c328f931db466aeb115b7ff5d80aee962afd uploaded (295 KB).

🤖 Generated with Claude Code

@tyeth

tyeth commented Sep 8, 2026

Copy link
Copy Markdown
Owner Author

The Pico 2 W firmware CI asset now exists.

Run 34262269542 succeeded and produced raspberrypi_rpi_pico2_w_zephyr-en_US-latest (8,114,840 bytes — firmware.uf2 + firmware.elf), so build_results.md's statement that no Pico 2 W firmware artifact exists is now out of date.

The earlier run failed because the CI-only west.yml override referenced integration-pico2w-ble on tyeth/hal_rpi_pico, a branch that had never been pushed. It is pushed now and all three overrides resolve.

Two caveats that still apply to that artifact, tracked as tyeth/circuitpython#12: it is BLE-capable only because of the CI-only manifest override, which must not be merged; and it depends on integration-pico2w-ble, which has no PR of its own.

The whole Bluetooth effort — every PR and open issue across the four forks — is now tracked in tyeth/circuitpython#15. Both Pico W and Pico 2 W BLE are verified working on hardware.

🤖 Generated with Claude Code

@tyeth

tyeth commented Sep 9, 2026

Copy link
Copy Markdown
Owner Author

Correction to the "Prerequisite CircuitPython fork branch" section (2026-09-09). Two statements in the body are now out of date; leaving the history as written and recording the current state here.

  1. "There is no Pico 2 W firmware artifact" — no longer true. The CI-asset branch ci/pico2w-ble-assets @ 94bfd47b5f (the BLE stack plus a CI-only west.yml override, see CI-only west.yml override on ci/pico2w-ble-assets must not be merged, and depends on an unreviewed branch circuitpython#12 / #16) builds both boards green with UF2 artifacts:

    Board Run Conclusion Artifact
    raspberrypi_rpi_pico2_w_zephyr 34287512362 success raspberrypi_rpi_pico2_w_zephyr-en_US-latest (8,852,015 B, expires 2026-12-07)
    raspberrypi_rpi_pico_w_zephyr 34287514167 success raspberrypi_rpi_pico_w_zephyr-en_US-latest

    The UF2 + unstripped ELF for both are parked on the prerelease zephyr-cp-ble-ci-20260909 so they outlive the artifact expiry. Flashing either reformats CIRCUITPY.

  2. "these mpy-cross binaries come from an otherwise-red run" — the red tests / zephyr job on run 34253440312 (zephyr-pico2w-ble @ f4d3e5987f) is not reproduced on a branch off current main: run 34282151673 on zephyr-cp-debug-udc-stack @ 5f0ea1254a (zephyr-cp: raise the rpi_pico UDC thread stack in debug builds circuitpython#5) is fully green, including zephyr-tests / zephyr: success and the native_native_sim board build. That is the "first thing to check" from CI: "tests / zephyr" job is failing on the BLE branches — cause not established circuitpython#11, and it points at the failure being specific to the older branch state (it was 229 commits behind main at the time) rather than to the BLE changes themselves — Run on the Raspberry Pi Pico 2 W (hub) and Pico W (node): capability gates, RTC-less clock, BLE advertisement transport #11 stays open until the original branch is re-run rebased.

Everything else in the body still holds: mpy-cross only proves compiler acceptance, and both trees as they stand on main target ESP-NOW + deep sleep, neither of which exists on the Pico boards. The follow-up that makes the trees run on the Pico 2 W / Pico W against exactly this firmware is #11 — umbrella tyeth/circuitpython#15.

tyeth and others added 4 commits September 10, 2026 23:44
tools/build_bundle.sh byte-compiles node/ and collector/ with mpy-cross,
stages each tree's libraries with circup --path, and compiles the
libraries that only ship as .py (the SEN5x custom bundle has no 10.x-mpy
build). build_results.md records one run against the mpy-cross from
tyeth/circuitpython run 34253440312 (10.3.0-alpha.4-73-gf1ae373ad0):
all 23 device sources compile, all 19 requested libraries resolve.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Add .github/workflows/build-bundle.yml, which runs tools/build_bundle.sh
on pushes to main, pull requests and on demand: it fetches mpy-cross from
a tyeth/circuitpython Actions run, byte-compiles node/ and collector/,
stages their libraries with circup and uploads the result as an artifact.
Cross-repo artifact download needs a token with Actions: read on
tyeth/circuitpython, so the workflow consumes a CP_CI_TOKEN repository
secret rather than the job-scoped GITHUB_TOKEN; permissions are
contents: read only.

Rework the CI-asset section of build_results.md: separate the mpy-cross
run (34253440312, red overall but with usable mpy-cross artifacts) from
the Pico 2 W firmware run (34258666665, "Build board (custom)" on the
CI-only branch ci/pico2w-ble-assets), record that the latter was still
queued with no artifact or result at the time of writing, explain why
ci/pico2w-ble-assets must not be merged, and document how to create
CP_CI_TOKEN.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The run was created at 17:41Z and checked at 17:53Z, so it had been
queued for about twelve minutes, not roughly an hour.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…artifact

The previous revision captured the run while it was still queued. It has
since run and failed after 5m49s in "Set up port" (west update):

    --- hal_rpi_pico: fetching, need revision integration-pico2w-ble
    fatal: couldn't find remote ref integration-pico2w-ble
    ERROR: update failed for project hal_rpi_pico

The CI-only west.yml override on ci/pico2w-ble-assets points hal_rpi_pico
at tyeth/hal_rpi_pico@integration-pico2w-ble, which was never pushed; the
fork only has the two PR head branches (cyw43-shared-bus-ble,
flash-ram-helpers-force-inline). The zephyr and hal_infineon fork refs
resolved. The build step never ran, so no
raspberrypi_rpi_pico2_w_zephyr-en_US-latest artifact exists and the .uf2
still has to be built locally.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tyeth
tyeth force-pushed the ci-build-node-and-collector branch from 108d7e7 to 02a23c7 Compare September 10, 2026 22:45
tyeth and others added 2 commits September 11, 2026 00:18
The firmware run this file recorded as failed has been superseded: the
integration-pico2w-ble branch it was missing now exists on
tyeth/hal_rpi_pico, and Build board (custom) produces both Pico 2 W
(34467425010) and Pico W (34494018766) firmware. Take mpy-cross from the
latter -- it is green, expires two days later, and is the same build as
the firmware. Both compilers emit mpy v6.3, so no compiled output changes.
The 'otherwise-red run' caveat is also gone: tests / zephyr is green again
(tyeth/circuitpython#11).

The custom bundle's missing 10.x-mpy turned out to be a build bug, not a
dormant repo: its build.sh passes --package_folder_prefix wrapped in
literal quotes, which startswith() then never matches, so both libraries
fell back to autodetection and the bundle shipped only conftest. Fixed in
good-enough-technology/CircuitPython_GoodEnough_Bundle#1; the 10.x-mpy
reaches circup once that is merged and released.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measured after writing it down. adafruit/CircuitPython_Community_Bundle
builds a byte-identical bundle with and without the fix (1788 files),
because package_folder_prefix is only consulted by legacy autodetection
and only 11 of its 47 prefix entries still reach that path -- the rest
declare tool.setuptools.packages in pyproject.toml. The two damaged
entries are currently not among the 11.

It is total in our custom bundle for two compounding reasons, now both
stated: two libraries means both entries are damaged, and the Sensirion
drivers carry no [tool.setuptools] metadata so they depend on the prefix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant