Expose a private virtual function in Python

[francesco-romano]

Hello.

I have been struggling a bit with this, and I could not find a solution (or not even a workaround that won't imply changing my pure C++ APIs).

I have a C++ class

class A {
public:
  bool doSomething() {
     return doSomethingInternal();
  }

private:
  virtual bool doSomethingInternal() {
     return true;  // default impl.
  }
};

Private virtual methods are perfectly fine in C++. But when I try to create the trampoline class, the resulting pybind code does not compile (which makes sense as while you can override that method, you can't call the base implementation).

I also tried the following (which also helps as, not shown here but I need to catch any Python exception and return false for that):

class _PyA: public A {
  public:
   virtual bool doSomethingPublic() { return true; }

 private:
  bool doSomethingInternal() override {
      return doSomethingPublic();
  }
};

// Trampoline
class PyA: public _PyA {
  public:
     bool doSomethingPublic() override {
       PYBIND11_OVERRIDE(bool, _PyA, doSomethingPublic, );
}
};

...
// Expose the base C++ class.
py::class_<A>(m, "_A", py::is_final())  // Python users cannot inherit from this.
      .def("do_something", & A:: doSomething)


py::class_<_PyA, PyA>(m, "PyA")
      .def(py::init<>())
      .def("_do_something", & _PyA:: doSomethingPublic)
      .def("do_something", & PyA:: doSomething)  // Need to re-expose the base class methods.

This compiles but, this will not allow me to expose the following

class Container {
   public:
     void addObject(A*);
  private:
   std::vector<A*> objects_;
};

as PyA objects are not see as inheriting from A.

Any suggestion?

[francesco-romano]

I found a possible solution, but I am unsure how correct it is.

The setup is greatly simplified.
Having the same base class as before:

class A {
public:
  bool doSomething() {
     return doSomethingInternal();
  }

private:
  virtual bool doSomethingInternal() {
     return true;  // default impl.
  }
};

Now my bindings look like:

namespace py = ::pybind11;

class PyA: public A {
  static constexpr char kDoSomethingInternalPythonMethodName[] =
      "_do_something";

 public:
  PyA() = default;

  bool doSomethingInternal() override {
    // Do not use the default macro as they do not allow changing the return
    // type. We do not want the Python user to return a bool
    // but to throw native Python exceptions in case of errors.
    py::gil_scoped_acquire gil;

    // Find the overridden method in Python. Note that the method does not have
    // the same signature as the C++ one. It returns void but can throw
    // exceptions.
    py::function do_something_python =
        py::get_override(this, kDoSomethingInternalPythonMethodName);
    if (do_something_python) {
      try {
        do_something_python();
      } catch (py::error_already_set& e) {
        return false;
      }
    }
    return true;
  }
};


// And the bindings code just list the public methods.
PYBIND11_MODULE(pybind_abstract_class, m) {
// This still use the order of classes as with trampoline class, i.e. <Base, Derived>
  py::class_<A, PyA>(m, "A")
      .def(py::init<>())
      .def("do_something", &A::doSomething);
}

And now my Python code:

class PyClass(pybind_abstract_class.A):

  def _do_something(self) -> None:
    print('This is Python printing')


class PyExceptionClass(pybind_abstract_class.A):

  def _do_something(self) -> None:
    print('Will throw')
    raise ValueError('exception')


# ... Somewhere in the code ...
obj = PyClass()
# This will print "This is Python printing".
obj.do_something()

# ...
obj2 = PyExceptionClass()
obj.do_something()  # This returns false.

Do you think this is correct? (It works, but I would like to avoid suboptimal solutions or non documented behaviours)