Proposal: Extend Coverage Reporting for MicroPython Project to Include Python Tools (mpremote)

[Josverl]

Enhance the MicroPython project's test coverage reporting by including Python-based tools, specifically mpremote, to improve code quality and reliability.

Current Situation:

• Existing coverage reporting primarily targets MicroPython C & C++ core code.
• Coverage reporting via CodeCov.io is already in place but does not yet include mpremote.
• Python tools, notably mpremote, currently lack integrated coverage metrics
• the PoC measures mpremote test coverage at 52% , significantly lower than the ~98% of the rest of MicroPython

Proposed Solution:

• Extend the bash test scripts for mpremote with basic coverage measurement and reporting.
• PoC ./run-mpremote-tests.sh --coverage
• Add mpremote coverage reporting to GitHub actions, and push these statistics to CodeCov.io.

Benefits:

• Clear visibility into test coverage for mpremote.
• Easier identification of untested code paths.

Next Steps:

• Review and refine the existing PoC implementation.
• Integrate the PoC into the main test suite.
• Incrementally expand and refine existing tests to improve coverage and maintainability.

Log term vision:

• Using a standard Python test framework (e.g., pytest ur unit test to be able to add unit tests).

[jonnor]

Sounds good to me :)

[Josverl]

I have been adding additional E2E tests for mpremote code areas that were lacking coverage, and have been able to increate the coverage from 52% --> 66%

The big chunks that are still missing as are:

• repl
• console
• Platform specific code
• Transport Error handling
• romfs exception handling

these are hard to test in E2R , and probably need mocking to perform efficiently.

I'll open a draft PR