fix: make twoliter work in parallel invocation scenarios #478
Conversation
|
So yeah, perhaps "obviously" we hit the race of the I guess a question for you guys is if you'd prefer me unwrapping |
|
Hey @mikn sorry for the silence here. This has been competing with other priorities of mine, and I know the parallelization concept will require a lot of attention, so I want to make sure I can give this the focus it deserves. We will work to make some space to review this soon! Thanks for submitting this. |
|
It's completely fine! I have myself been drowning in other work. I finally am updating all of the BR tooling on our side again, trying to upstream as much as possible to slim down our footprint and I once again, after having seen the very generous changes made by @jmt-lab here e616ed0 Ironically, I actually hit parallelization troubles here already: because this is the path (when you run "build variant" on the twoliter side): https://github.com/bottlerocket-os/twoliter/blob/develop/twoliter/embedded/Makefile.toml#L1015 Here is the outtake: Now, I don't know why the check licenses flow would mess with the cargo folder... but it does. :D |
Issue number:
Closes #477
Description of changes: Fixed race conditions in Twoliter's verification tag system and tools directory installation to prevent corruption when multiple concurrent processes attempt to manipulate these resources:
Verification Tag System Improvements:
Tools Directory Installation Improvements:
The changes (hopefully?) maintain backward compatibility while making Twoliter more robust in environments where multiple processes run in parallel (e.g., Bazel builds).
Testing done:
variantbuildsAfter this patch, it now looks like this:
I do understand the thinking behind removing the lock files early on, but there seems to be no actions that you have that checks for their existence before or without actually verifying them, so I do not think it is something worth guarding against. This is quite a lot of cowboy coding though, so feel free to correct me if I have misunderstood something!
Terms of contribution:
By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.