zephyr-cp: freeze .mpy modules named in circuitpython.toml - #23
Open
tyeth wants to merge 1 commit into
Open
Conversation
The port had no frozen-module support: its Python-driven build never runs make, and the freeze pipeline lives in py/circuitpy_mpconfig.mk. Reimplement the pipeline in cptools/build_circuitpython.py behind a per-board opt-in, FROZEN_MPY_DIRS = ["frozen/<lib>", ...] in circuitpython.toml (paths relative to the repository root, as $(TOP)/... is in mpconfigboard.mk): - tools/preprocess_frozen_modules.py stages the trees into <build>/frozen_mpy (repo directory dropped, __version__ filled in, examples and tests left out), before the qstr pass; - MICROPY_QSTR_EXTRA_POOL / MICROPY_MODULE_FROZEN_MPY are added to the flags for the qstr pass as well as the compile -- Q(.frozen) and the sys.path entry that uses it are behind MICROPY_MODULE_FROZEN; - after genhdr/qstrdefs.generated.h and root_pointers.h exist, mpy-cross compiles each module (-s with the module path, so mpy-tool derives the frozen name from it) and tools/mpy-tool.py -f -q emits frozen_content.c, fed the same collected qstr list that produced the generated header so the frozen pool numbers from MP_QSTRnumber_of correctly. tools/makemanifest.py is bypassed: it insists on genhdr/qstrdefs.preprocessed.h, which this builder never produces. Only MPY freezing: MICROPY_MODULE_FROZEN_STR would reference the mp_frozen_str_* tables makemanifest emits. - frozen_content.c is compiled with the no-qstr sources (it defines its own MP_QSTR_* enum values and must never go through extraction). pre_zephyr_build_prep.py builds mpy-cross first when a board freezes modules (honouring MICROPY_MPYCROSS), and tools/ci_fetch_deps.py learns the toml key so CI initialises the right frozen/ submodules (it had a TODO for this). The Pico W opts in with adafruit_ble: it has ~42 KB of heap and a BLE node cannot otherwise load the library. Measured on a Pico 2 W (same core, same flags): importing adafruit_ble plus its advertising.standard and services.nordic modules costs 10,384 B of heap frozen vs 21,792 B from .mpy files on CIRCUITPY (gc.mem_alloc() delta after gc.collect(); gc.mem_free() is not usable here, the split heap grows on demand), 0.055 s vs 0.132 s. Freezing the 20 modules adds 28,336 B of flash and no static RAM on the Pico W (1,208,636 -> 1,236,972 B with the rest of this series). The Pico 2 W is not opted in: it has the heap, and a frozen copy would pin the library version for everyone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Sep 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The port had no frozen-module plumbing (the Python builder never runs make;
FROZEN_MPY_DIRis a hard error). This reimplements the pipeline incptools/build_circuitpython.pybehind a per-board opt-in:Stages the trees with
tools/preprocess_frozen_modules.py, adds the frozen-Dflags to the qstr pass as well as the compile (.frozenonsys.pathis behindMICROPY_MODULE_FROZEN), then after the qstr headers exist runsmpy-cross -sper module andtools/mpy-tool.py -f -qagainst the same collected qstr list that producedqstrdefs.generated.h, so the frozen pool numbers correctly fromMP_QSTRnumber_of.makemanifest.pyis bypassed (it hard-codesgenhdr/qstrdefs.preprocessed.h, which this builder never produces).pre_zephyr_build_prep.pybuildsmpy-cross;tools/ci_fetch_deps.pylearns the toml key (it had a TODO).Measured, not estimated (Pico 2 W,
gc.mem_alloc()delta aftergc.collect()—gc.mem_free()is unusable here, the split heap grows on demand and shows a negative delta):import adafruit_ble+advertising.standard+services.nordic.frozen/adafruit_ble/__init__.py).mpyfiles on CIRCUITPY (/mpylib/...)Flash cost: Pico W 1,208,636 → 1,236,972 B (+28,336 B for the 20 modules), static RAM unchanged (238,996 B). The Pico W opts in (~42 KB heap; a BLE node cannot otherwise hold
adafruit_ble). The Pico 2 W does not: it has the heap, and freezing would pin the library version for everyone — that is the trade-off, stated rather than forced.Pico W final image with this whole series: 1,242,092 B flash (79.40 %), 238,996 B RAM (88.41 %) — TCP alone is most of the RAM growth from 84.4 %.
🤖 Generated with Claude Code