Skip to content

Conversation

@grooverdan
Copy link
Member

First draft. Seeing if github actions can handle building the entire infer from scratch.

Their last release 1.2.0 was in June 2024 and its actively developed.

@grooverdan grooverdan force-pushed the MDBF-143-infer branch 6 times, most recently from 822e6bb to b795254 Compare September 10, 2025 06:20
@grooverdan grooverdan marked this pull request as draft September 10, 2025 06:21
@grooverdan grooverdan force-pushed the MDBF-143-infer branch 2 times, most recently from ba8e232 to e484be1 Compare September 11, 2025 08:17
@grooverdan grooverdan changed the title MDBF-143: Add infer (container build) MDBF-143: Add infer Sep 11, 2025
@RazvanLiviuVarzaru
Copy link
Collaborator

These 4 commits, does not seem necessary as you are doing all the work in the shell script.
They are welcome but not in the scope of this patch?

image

@RazvanLiviuVarzaru
Copy link
Collaborator

RazvanLiviuVarzaru commented Oct 27, 2025

Maybe the specifications have already been set, but after analyzing the script, I was wondering if the following flow might be simpler? Still, I believe you’ve already achieved the maximum efficiency in terms of time and resource consumption with the current script, even though it’s a bit difficult for me to understand it completely at the moment. (which is not a blocker of course I just need your guidance)

If the push is on a main branch e.g. 10.6, 10.11,...,main :: perform a full analysis i.e.
    -> build the code
    -> capture
    -> analysis
    -> publish report on CI
If the push is on a feature branch (e.g. bb-*, refs/pull/head, so on) then analyze the diff between the feature branch and #master_branch# as follow:
    [1]:
    -> build the code on the feature branch
    -> infer capture ...
    -> git diff --name-only origin/feature..origin/#master_branch#  > index.txt
    -> infer analyze --changed-files-index index.txt
    -> move the report 
    [2]:
    -> build the code on the #master_branch# (e.g. for bb-10.6-razvan, master_branch=10.6)
    -> infer capture --reactive --mark-if-unchanged
    -> infer analyze --incremental-analysis --changed-files-index index.txt
    -> infer reportdiff --report-current report_for_the feature --report-previous report_of_the_master_branch
    -> publish the diff report on CI

With the above workflow you don't need the bind mounts anymore at the expense of:

  • a full analysis on main branches
  • building twice for a feature branch (although building those targets is very fast)
    So I am not sure it is the best idea, it's just how I imagine it in the simplest form.

@grooverdan grooverdan force-pushed the MDBF-143-infer branch 2 times, most recently from 1512189 to 9cd42b6 Compare October 28, 2025 07:00
@grooverdan
Copy link
Member Author

Maybe the specifications have already been set

not totally, its hopefully a bit simpler now

achieved the maximum efficiency in terms of time and resource consumption with the current script

I did try. The incremental builds aren't saving much in capture, and not sure how much in analyse.

At some point if no decent incremental results are there that could be removed leaving just the need for X main branch saves.

if need_to_full build;
     -> build the code
     -> capture
     -> analysis
else
     -> build the code
     -> capture --incremental
     -> analysis --incremental
fi

save result to $infer

diff to $master_branch branch to generate incremental report using json "db" manipulation

# quasy save diff also just for ref bug probably could be removed.

publish report on CI

@grooverdan grooverdan marked this pull request as ready for review October 28, 2025 07:17
Copy link
Collaborator

@fauust fauust left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with @RazvanLiviuVarzaru, it's a bit complex to understand what the big picture is, probably because I don't know the tool. I'll probably need to read the infer doc next and come with (maybe) some design arguments.

Small changes requested for consistency mostly.


if [ ! -d bld ]; then
mkdir bld
build
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?

@RazvanLiviuVarzaru RazvanLiviuVarzaru force-pushed the MDBF-143-infer branch 3 times, most recently from d4c0ecc to 474b2e4 Compare October 28, 2025 12:31
@RazvanLiviuVarzaru
Copy link
Collaborator

RazvanLiviuVarzaru commented Oct 28, 2025

Added a worker: 4f32380

@RazvanLiviuVarzaru
Copy link
Collaborator

RazvanLiviuVarzaru commented Nov 5, 2025

@grooverdan
Did a couple of more minor changes and rebased & squashed everything.
Started a test: https://buildbot.dev.mariadb.org/#/builders/800/builds/3 but it failed on timeout.

One thing I've noticed is that infer in the absence of a TTY is not showing any progress, except warnings/errors. The risk is, buildbot will kill the container if no output is received for a given amount of time.

I tried --progress-bar-style plain to see if it changes anything but it is the same as in CI,
will stay "stuck" at Starting translating X files.

If there aren't any methods to force it show some output, I can only increase the step timeout to something like 1-2 hours. 93d7342 With increased timeout: https://buildbot.dev.mariadb.org/#/builders/800/builds/4

Side note:
This patch was manually deployed on buildbot.dev.mariadb.org and on push only tarball-docker -> infer are triggered so you can make as many pushes as you like to my fork, if you need to.

@RazvanLiviuVarzaru
Copy link
Collaborator

RazvanLiviuVarzaru commented Nov 6, 2025

@grooverdan please check 702d7ce, let me know if I missed anything

Here are two runs after this commit:

Although I don't see the issue I've introduced, in the second run.

I think it's best to not show the diff in the output because it's producing too much logs
that are hard to follow anyway from the buildbot interface (UI can be slow). Instead, I suggest only letting
the user know there is a problem and where to find the logs.

This preforms static analysis on the MariaDB codebase
by maintaining a git source repository as a shared volume.

Because static analysis takes time, a lot of time, there
is a shared cache volume to store build results from main
branches of the codebase so that as much incremental usage
can occur.

Infer runs in to phases, a capture and an analyze.

Infer output are in a result-dir this contains:
* report.json - what infer tools use
* report.txt - the human readable version of this
* capture.db - the sqlite3 version presentation of captured files and the
               relation to functions definitions.
* results.db - the analyze phase outputs

Of these, the report.json is desirable as the long term record of vulnerabilities.
and the main_diff containing the difference from the last
main X.Y branch commit.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants