Automatic downcast to "intermediate" class?

[Holt59]

I have a class hierarchy like so:

class A : public std::enable_shared_from_this<A>
{
public:
    virtual int g() {
        return 0;
    }
    virtual ~A() {}
};

class B : public virtual A
{
public:
    virtual int g()
    {
        return 1;
    }
    ~B() {}
};

class C : public B {};

Both A and B have bindings:

PYBIND11_MODULE(module, m)
{
    py::class_<A, std::shared_ptr<A>> a(m, "A");
    py::class_<B, std::shared_ptr<B>, A>(m, "B");
}

If I return a std::shared_ptr<B> from a function returning a std::shared_ptr<A>, I get proper downcasting, but not if I return a std::shared_ptr<C>. Is there a way to get proper downcasting, i.e., get a B on the Python side when returning a std::shared_ptr<C>?

I tried using polymorphic_type_hook:

namespace pybind11
{
    template <>
    struct polymorphic_type_hook<A>
    {
        static const void *get(const A *src, const std::type_info *&type)
        {
            // note that src may be nullptr
            if (auto p = dynamic_cast<const B*>(src))
            {
                type = &typeid(B);
                return p;
            }
            return src;
        }
    };
} // namespace pybind11

But that completely breaks stuff, e.g., if something returns a std::shared_ptr<B> (or C), the output of g() in Python is something like -1175756672 instead of 1. In my actual application, this just crashes everything.

[jiwaszki]

There are two ways: easy and advanced. Easy hot-fix is to simply bind C class, thus provide casters for it:

py::class_<C, std::shared_ptr<C>, B> cls_c(m, "C");

This also allows to inherit all function bindings of B.

However you want upcasting (upcasting from child to parent). Here comes a solution with the custom caster, notice the lack of explicit C class bindings:

namespace pybind11
{
      namespace detail
      {
            template <>
            struct type_caster<std::shared_ptr<C>>
            {
            public:
                  // Python -> C++
                  bool load(handle src, bool convert)
                  {
                        if (!isinstance<std::shared_ptr<C>>(src))
                        {
                              return false;
                        }
                        // Add code here if necessary...
                        return true;
                  }

                  // C++ -> Python
                  static handle cast(std::shared_ptr<C> &&src, return_value_policy policy, handle parent)
                  {
                        // If there is a need to cast shared_ptr<C> back to Python, use caster from parent class
                        // which is already registered when py::class_ is called during compilation bindings.
                        // Leave rest of the work to the internals of C++.
                        return type_caster<std::shared_ptr<B>>::cast(src, policy, parent);
                  }

                  // Name of type is just for demonstration:)
                  PYBIND11_TYPE_CASTER(std::shared_ptr<C>, const_name("C_downgraded"));
            };
      }
}

PYBIND11_MODULE(mymodule, m)
{
      py::class_<A, std::shared_ptr<A>> cls_a(m, "A");
      py::class_<B, std::shared_ptr<B>, A> cls_b(m, "B");

      cls_b.def("g", &B::g);

      m.def("fun", []()
            { return std::make_shared<A>(); });

      m.def("bar", []()
            { return std::make_shared<B>(); });

      m.def("boo", []()
            { return std::make_shared<C>(); });
}

Now let's test it out:

>>> import mymodule
>>> tmp = mymodule.fun()
>>> tmp
<mymodule.A object at 0x1012980b0>
>>> tmp = mymodule.bar()
>>> tmp
<mymodule.B object at 0x101298130>
>>> tmp.g()
1
>>> tmp = mymodule.boo()
>>> tmp
<mymodule.B object at 0x1012980f0>
>>> tmp.g()
1

Nice! Even if you look into help(mymodule) you will notice how custom caster affected the module:

FUNCTIONS
    bar(...) method of builtins.PyCapsule instance
        bar() -> mymodule.B
    
    boo(...) method of builtins.PyCapsule instance
        boo() -> C_downgraded
    
    fun(...) method of builtins.PyCapsule instance
        fun() -> mymodule.A

I hope that provides a base for your fully working solution, cheers!

[Holt59]

@jiwaszki Thanks for your solution, unfortunately, I have no clue about the actual implementation under B in the bindings code, hence I cannot add anything that would actually use a C.

I managed to fix it by using the polymorphic_type_hook above, my issue was that I needed to add py::multiple_inheritance() to the bindings of B. I guess py::multiple_inheritance is also need for virtual inheritance, or something like that... I admit that this is not clear to me. The problem was not actually with C , because even with shared_ptr<B> returned I had issues (objects shifted in memory).

[jiwaszki]

Glad to know that problem was resolved! Yes, probably the issue was connected to multiple classes being inherited by B.