Skip to content

Create and enable post-processing job for code coverage reporting #1211

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 24, 2025

Conversation

a-wai
Copy link
Contributor

@a-wai a-wai commented Jun 19, 2025

Add a new template for generating code coverage report, meant to be used as a post-processing job for kernels built with GCOV support.

This job will collect coverage data from every test job of a given kbuild node, report lines/functions coverage for each job, aggregate coverage data for all jobs and generate the following artifacts pertaining to the kbuild node itself:

  • single-page HTML report including lines/functions coverage data for each source file
  • LCOV-compatible tracefile for further processing (filtering, generating detailed HTML report...) by individual developers
  • child nodes with global lines/functions coverage for the aggregated data

Depends on kernelci/kernelci-core#2908 kernelci/kernelci-core#2909 & kernelci/kernelci-core#2916

@a-wai a-wai requested review from nuclearcat and laura-nao June 19, 2025 15:35
@a-wai a-wai force-pushed the coverage-report-initial-template branch 7 times, most recently from 1eeec0d to 905de3a Compare June 25, 2025 14:38
@a-wai
Copy link
Contributor Author

a-wai commented Jun 26, 2025

Successfully tested on staging, as can be seen from the list of child nodes for the corresponding kbuild:

  • each ltp-* node has coverage.functions and coverage.lines child nodes containing the coverage percentage in data[misc][measurement] (except for one which timed out in LAVA, and therefore didn't upload the coverage data tarball -- this will be fixed with a follow-up PR)
  • the coverage-report node contains the HTML report as well as a (gzipped) LCOV-compatible tracefile and job log as artifacts
  • coverage-report also has coverage.* child nodes reporting the coverage percentages for all aggregated tests

@a-wai a-wai force-pushed the coverage-report-initial-template branch from 905de3a to f277c92 Compare June 26, 2025 10:40
@a-wai a-wai force-pushed the coverage-report-initial-template branch 3 times, most recently from 18f0547 to e79ee71 Compare July 10, 2025 10:08
Copy link
Contributor

@pawiecz pawiecz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (from Maestro point of view, please let me know if you'd also like to have coverage reports logic double checked)

Thanks for sending the link to Nodes on Staging that verify this PR works properly

a-wai added 2 commits July 23, 2025 15:42
This new template is meant to be executed as a post-processing job, once
every test job for a given `kbuild` node are complete. It gathers the
raw coverage data from each of those test jobs and processes it as
follows:
* create a single JSON tracefile using `gcovr`
* extract lines/functions coverage percentages for each job
* create `test` child nodes for each job, reporting those percentages
* merge all tracefiles in a single results file and generate both an
  HTML report and `lcov`-compatible tracefile; the HTML report gives a
  quick overview of the code coverage, while the tracefile can then be
  downloaded by developers for more targeted processing
* create `test` child nodes for the `kbuild` job, reporting global
  lines/functions coverage percentages for this run

Signed-off-by: Arnaud Ferraris <[email protected]>
Create a generic job definition meant for post-processing coverage data
for kernel builds where GCOV support is enabled. This job is enabled for
all kernels built with the `coverage` config fragment.

Signed-off-by: Arnaud Ferraris <[email protected]>
@a-wai a-wai force-pushed the coverage-report-initial-template branch from e79ee71 to 729d2b8 Compare July 23, 2025 13:42
@a-wai
Copy link
Contributor Author

a-wai commented Jul 23, 2025

LGTM (from Maestro point of view, please let me know if you'd also like to have coverage reports logic double checked)

I'm reasonably confident with the logic, so I'd say "not mandatory", but if you have enough free cycles feel free to double check

@nuclearcat nuclearcat added this pull request to the merge queue Jul 24, 2025
Merged via the queue into kernelci:main with commit 668165d Jul 24, 2025
4 checks passed
@a-wai a-wai deleted the coverage-report-initial-template branch July 24, 2025 09:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants