POC: Add a relocatable build flag #861
Conversation
zanieb
added
python:3.15
platform:darwin
zanieb
force-pushed
the
zb/relocatable-flag
branch
2 times, most recently
from
63daa7f to
c2e856d
Compare
zanieb
force-pushed
the
zb/relocatable-flag
branch
from
c2e856d to
d715fa3
Compare
geofft
left a comment
geofft
left a comment
There was a problem hiding this comment.
Unfortunately I think there's a good amount of stuff missing compared to our current implementation, and this is a hairy enough spot that I don't think we should have any regressions in 3.15 compared to what we do on older versions.
On glibc/Linux, we add a DT_NEEDED for bin/python to $ORIGIN/../lib/libpython3.x.so.1.0 (and for lib/libpython3.so to $ORIGIN/libpython3.x.so.1.0) , to avoid issues with LD_LIBRARY_PATH taking precedence. I'm relatively sure this is annoying to do without a separate patchelf step, but maybe it can be done.
We also set an DT_RPATH for bin/python on Linux (required on musl, compatibility on glibc). This is not a DT_RUNPATH, so this needs -Wl,--disable-new-dtags.
I don't see either of those in the Makefile diff, in addition to the other things noted in the comments.
If you really wanted, you could start by upstreaming this gated to macOS only, which might be a little less code to write.
Also speaking of platform support, the dynamic linkers on the BSDs and Solaris works much like glibc, so most of our Linux support should apply there too in the version that gets upstreamed.
In either case, I would error out if --enable-relocatable is passed on another platform.
| Author: Zanie Blue <[email protected]> | ||
| Date: Fri Nov 21 15:44:42 2025 -0600 | ||
|
|
||
| Set linker arguments when `--enabled-relocatable` is used |
There was a problem hiding this comment.
| Set linker arguments when `--enabled-relocatable` is used | |
| Set linker arguments when `--enable-relocatable` is used |
| -change /install/lib/${LIBPYTHON_SHARED_LIBRARY_BASENAME} @executable_path/${LIBPYTHON_SHARED_LIBRARY_BASENAME} \ | ||
| ${ROOT}/out/python/install/lib/${LIBPYTHON_SHARED_LIBRARY_BASENAME} | ||
|
|
||
| # We also normalize /tools/deps/lib/libz.1.dylib to the system location. |
There was a problem hiding this comment.
How come you didn't need to preserve this?
| # At the moment, python3 and libpython don't have shared-library | ||
| # dependencies, but at some point we will want to run this for | ||
| # them too. | ||
| for module in ${ROOT}/out/python/install/lib/python*/lib-dynload/*.so; do | ||
| install_name_tool -add_rpath @loader_path/../.. "$module" | ||
| done |
There was a problem hiding this comment.
This almost certainly needs to be preserved, this fixes up _tkinter, right? That is, you need this to go into the CPython Makefile rule for building shared-library extension modules (... which we might not be using, to make things more exciting).
| @@ -1025,10 +1025,19 @@ libpython$(LDVERSION).so: $(LIBRARY_OBJS) $(DTRACE_OBJS) | ||
| fi | ||
|
|
||
| libpython3.so: libpython$(LDVERSION).so |
There was a problem hiding this comment.
You need to patch the libpython$(LDVERSION).so target too. (LINKFORSHARED is only used when building the interpreter.)
The goal here is to construct two upstreamable patches
--enable-relocatableconfigure flag to gate new behaviorAs a PoC, we apply these patches here for Python 3.15 and skip our post-build relocatability patchelf invocations.