Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallel make race for lz4 build #18

Open
nbuwe opened this issue Mar 19, 2024 · 1 comment
Open

Parallel make race for lz4 build #18

nbuwe opened this issue Mar 19, 2024 · 1 comment

Comments

@nbuwe
Copy link

nbuwe commented Mar 19, 2024

The lz4 submodule build is defined as:

criu/Makefile

Lines 248 to 253 in daeef8f

LZ4_OBJS = lz4/lib/liblz4.a criu/liblz4io.a
$(LZ4_OBJS) :
git submodule init
git submodule update
$(Q) env -i PATH=$$PATH make CC=$(CC) CFLAGS="$(CFLAGS)" -C lz4 lib lz4
$(Q) $(AR) rcs criu/liblz4io.a lz4/programs/lz4io.o

which is racy if parallel make is used. Since these two targets are independent (as defined), the make considers them to be suitable to run in parallel, so two identical make jobs will simultaneously run in the lz4 directory, stepping on each other toes. The likely result is a cryptic message from the linker in one of the jobs that tries to use an object file (that that job has previously built) while the other jobs starts writing to that same object file and truncates it. (The message is cryptic b/c linker tries to interpret an empty file as a linker script, but by the time you take a look, the other job has done writing out the object file, so it's seemingly back to normal).

@nbuwe
Copy link
Author

nbuwe commented Mar 19, 2024

The dependency can be introduced with something like (untested):

.PHONY: git-submodules
git-submodules:
	git submodule init
	git submodule update

lz4/lib/liblz4.a: git-submodules
	$(Q) env -i PATH="$$PATH" make CC=$(CC) CFLAGS="$(CFLAGS)" V="$(V)" -C lz4 lib

criu/liblz4io.a: lz4/lib/liblz4.a
	$(Q) env -i PATH="$$PATH" make CC=$(CC) CFLAGS="$(CFLAGS)" V="$(V)" -C lz4 lz4
	$(Q) $(AR) rcs criu/liblz4io.a lz4/programs/lz4io.o

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant