How should I introduce pybind11 into my cmake project without using pybind11_add_module?

[Kexiii]

I'm building a static library which will be used by other pybind11 C++ project, and I want to use pybind11::gil_scoped_release in the code but I don't want to build a python module, so I can't call pybind11_add_module. How should I write my CMakeLists.txt then?
I tried add add_subdirectory(third_party/pybind11) to my CMakeLists.txt, but it seems not enough:

my.cc

#include "pybind11/pybind11.h" // fatal error: pybind11/pybind11.h: No such file or directory