Embedding Python in C++ without linking libpython: Recommended but not explained?

[emmenlau]

I've read the documentation at https://pybind11.readthedocs.io/en/stable/compiling.html a few times and googled for a few hours, but I'm still missing a solution to this (seemingly) simple task. The documentation recommends exactly what I would like to do:

On Linux and macOS, it’s better to (intentionally) not link against libpython. The symbols will be resolved when the extension library is loaded into a Python binary. This is preferable because you might have several different installations of a given Python version (e.g. the system-provided Python, and one that ships with a piece of commercial software). In this way, the plugin will work with both versions, instead of possibly importing a second Python library into a process that already contains one (which will lead to a segfault).

Now the problem is that it does not describe how this should be done with cmake (or is it possible I overlooked it?). When I just remove pybind11::embed from target_link_libraries then I get lots of unresolved symbol errors in my executables.

Through lots of searching I found flags like PYBIND11_NOPYTHON but I'm not sure they really help me. I've also tried to manually exclude the missing symbols from ld.lld with flags like -Wl,--undefined=PyExc_ImportError in target_link_options, but that seems not to work (maybe I'm using the wrong syntax?).

Now I'm getting quite frustrated: Why is the documentation recommending something and then not giving any hints as to how that goal can be achieved? Am I overlooking something, or am I on the wrong track?

On a side note, I do not think its a good idea to ignore all undefined symbols. This was recommended on some obscure web pages. But that seems really bad practice and its therefore not a path I have followed.

[skoslowski]

This section of the doc applies to extension modules only. They are plugins and it's the responsibility of the host application, python, to provide those symbols (by linking libpython). When you embed Python you are creating your own host application that can load (import) extension modules. So it has to link libpython, that is after all where the runtime is implemented.

[emmenlau]

Dear @skoslowski , thanks for this clarification, this helps a lot. We wanted to keep the libpython dependency optional because in many cases, our application will not actually use python. Therefore I was happy to read that this is supported. From the cited paragraph I was under the impression that pybind11 has the required logic to load libpython at runtime when needed (i.e. something like dlopen would do on Unix).

But your clarification helps a lot. So pybind11 has no mechanism to dynamically load python when required, no?