Must rebuild almost everything on new git commit for irods/irods (main branch)

[alanking]

Whenever I make a change, I expect a rebuild to only rebuild what was changed. I've noticed when working on the main branch (not 4-3-stable) of irods/irods that things work as expected for any uncommitted changes. As soon as I create a new commit, it's as if the build sees all the diffs for the branch and rebuilds pretty much everything (but not all of it).

An example would illustrate what I mean (this is just an example - don't worry about the actual contents of the diff). Here, I have a large-ish diff from irods/irods#8410. Let's say I have packages built using the dev environment, and then I make some small change to one unit test:

diff --git a/unit_tests/src/test_authenticate_client.cpp b/unit_tests/src/test_authenticate_client.cpp
index 0aae17f84..4892df91b 100644
--- a/unit_tests/src/test_authenticate_client.cpp
+++ b/unit_tests/src/test_authenticate_client.cpp
@@ -90,6 +90,8 @@ TEST_CASE("rc_authenticate_client: minimum viable inputs")
     const auto ctx = nlohmann::json{{ia::scheme_name, env.rodsAuthScheme}};
     REQUIRE(0 == rc_authenticate_client(comm, ctx.dump().c_str()));
 
+    CHECK_THROWS(irods::query{comm, query_str});
+
     // Show that user can now run a query on a protected collection after authenticating.
     CHECK(!irods::query{comm, query_str}.empty());
 }

When I rebuild, it rebuilds 3 objects:

<snip>
+ ninja -j 7 package                
[2/3] Run CPack packaging tool...                                                                                                       
CPack: Create package using DEB                                                                                                         
<snip>

As soon as I stage the changes and create a commit (note: I did not change any of the previous commits) and build again (note: I did not make any changes AT ALL since the last build - only made a git commit), it rebuilds 1000 objects:

<snip>
+ ninja -j 7 package
[108/1000] Building CXX object lib/genquery1/CMakeFiles/irods_genquery1_parser.dir/dsl/parser.cpp.o
/irods_build/lib/genquery1/dsl/parser.cpp:432:9: warning: variable 'yynerrs_' set but not used [-Wunused-but-set-variable]
    int yynerrs_ = 0;
        ^
1 warning generated.
[789/1000] Building CXX object plugins/microservices/CMakeFiles/irods_microservice_plugin-test_issue_6829.dir/src/test_issue_6829.cpp.o
<still going>

As a result of this issue, I either have to wait to commit things (risking losing work) or wait to build things each time. My machine has 8 cores, so a simple no-diff commit as described above takes 15-20 minutes, which is quite slow.

I haven't tried building manually, using non-Docker volume build caches, or anything like that. That's why I'm filing this issue in the dev environment.

Any tips or commiseration would be appreciated. :)

Other possibly pertinent details...

$ docker --version
Docker version 24.0.6, build ed223bc
$ git --version
git version 2.34.1
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.3 LTS
Release:        22.04
Codename:       jammy

[alanking]

I should also mention that if I even switch branches to work on something else and then switch back without changing anything, it will do the same.

[korydraughn]

I am experiencing this as well.

[trel]

so we need to ignore some git-related files... which I believe only live in .git directories?