First time running a moon command after setup-toolchain fails #18
Comments
|
@milesj thanks for helping me with moonrepo/moon#1783. Hopefully you also have a bit of time to look into this one? |
|
@Blissful89 this is odd. If you set |
|
@milesj. Sorry for the delay, I live in the eu-timezone. It is hard to replicate, because it only does it the first run through. |
|
It looks like those 2 commands are running in parallel, instead of the 1st running and then piping to the 2nd. That's probably what is causing this, some kind of race condition. I'm not sure why exactly that pipe would run in parallel... |
|
I am indeed running different setup-toolchain dependent workflows in parallel. Is that problematic? I did set the cache: false on this run so I thought it would not interfere with anything. |
|
Is that running on the same box, or on different boxes? I can never tell with GH. But yeah, it's a bit problematic, since moon does a bunch of stuff for setting up the workspace/toolchain, which can conflict. You can maybe have both of these steps depend on a parent step that does purely the setup stuff? Probably |
|
I am just running on default runners. The lint task has a different set of requirements of active tooling, therefor I do # lint / unit test / anything that needs tooling
- name: Inject Moon
uses: moonrepo/setup-toolchain@v0
with:
auto-install: trueWhilst I don't need all those tooling in the moon query projects task, so I do # moon query task
- name: Inject Moon
uses: moonrepo/setup-toolchain@v0
with:
cache: falseThey both run in parallel because lint does not need to know the affected projects, so it is just running a "simple" jobs:
affected:
runs-on: ubuntu-latest
name: Getting affected projects 🔍...
unit:
runs-on: ubuntu-latest
name: Running ${{ matrix.workspace.name }} unit tests 🌡️...
if: needs.affected.outputs.matrix != '[]'
needs: [affected]
strategy:
matrix:
workspace: ${{ fromJSON(needs.affected.outputs.matrix) }}
lint:
runs-on: ubuntu-latest
name: Running affected lint 🔬...In schematics thats: Our linting process is extra tedious, because we are running multiple code inspections, so I would prefer to keep that task in parallel. It is taking long enough as it is. |
|
Ok that should be on 2 separate machines, so they wouldn't conflict. The logs are very confusing though, especially these lines: 2 instances of moon is running, 1 millisecond apart, while the This may be a bug in moon in regards to piping and not waiting for stdin correctly? I'll dig further. |
|
After sleeping on this and reading a bit of docs, I think I know what's going on. So when commands are piped, they run in parallel (I thought there was some delay), but the 2nd command waits for the stdout of the 1st through stdin. This is all working correctly, but, the proto install happens before the stdin check occurs, so we're hitting this race condition where they both are modifying the file system at the same time. I'm going to add some file system locks to help alleviate this. I'm surprised this hasn't cropped up sooner? Did you upgrade moon recently? |
|
Actually yes. I recently upgraded from 1.29 to 1.31. I also reported a bug in moon itself regarding task inherritence. (Which you solved practically instantly by the way 😅) edit maybe also important information is when this started occuring I also recently flushed the github caches. So it wasn't doing proto installs for a while because it was using the caches. |
|
This should help: moonrepo/moon#1788 |
Do I need to update my |
|
Yeah once I release it. |
|
@milesj . I see a release of 1.31.2. I will go and update our pipelines and see if it has effect. |
|
update - Just pinned 1.31.2, ran pipeline 5-6 times. Seems issue is resolved: Thanks a lot for the fix @milesj . Rock solid response from you guys! |
|
Awesome, glad it worked! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
I am using setup-toolchain to run moon query in the pipeline. Ever since the new releases the last couple of days, the first run of my ci fails. I have to give a second "kick" and retry to make it work.
Steps to reproduce
This is a simplified setup of our workflow. Note that we disable cache to prevent caches clashing and creating a race condition with a
setup-toolchainwithauto-install: trueExpected behavior
Usually we place the affected output into the $GITHUB_OUTPUT.
Screenshots
We now get an error stating:
The text was updated successfully, but these errors were encountered: