classes.rst revision 11986:c12e4625ab56
1Classes 2####### 3 4This section presents advanced binding code for classes and it is assumed 5that you are already familiar with the basics from :doc:`/classes`. 6 7.. _overriding_virtuals: 8 9Overriding virtual functions in Python 10====================================== 11 12Suppose that a C++ class or interface has a virtual function that we'd like to 13to override from within Python (we'll focus on the class ``Animal``; ``Dog`` is 14given as a specific example of how one would do this with traditional C++ 15code). 16 17.. code-block:: cpp 18 19 class Animal { 20 public: 21 virtual ~Animal() { } 22 virtual std::string go(int n_times) = 0; 23 }; 24 25 class Dog : public Animal { 26 public: 27 std::string go(int n_times) override { 28 std::string result; 29 for (int i=0; i<n_times; ++i) 30 result += "woof! "; 31 return result; 32 } 33 }; 34 35Let's also suppose that we are given a plain function which calls the 36function ``go()`` on an arbitrary ``Animal`` instance. 37 38.. code-block:: cpp 39 40 std::string call_go(Animal *animal) { 41 return animal->go(3); 42 } 43 44Normally, the binding code for these classes would look as follows: 45 46.. code-block:: cpp 47 48 PYBIND11_PLUGIN(example) { 49 py::module m("example", "pybind11 example plugin"); 50 51 py::class_<Animal> animal(m, "Animal"); 52 animal 53 .def("go", &Animal::go); 54 55 py::class_<Dog>(m, "Dog", animal) 56 .def(py::init<>()); 57 58 m.def("call_go", &call_go); 59 60 return m.ptr(); 61 } 62 63However, these bindings are impossible to extend: ``Animal`` is not 64constructible, and we clearly require some kind of "trampoline" that 65redirects virtual calls back to Python. 66 67Defining a new type of ``Animal`` from within Python is possible but requires a 68helper class that is defined as follows: 69 70.. code-block:: cpp 71 72 class PyAnimal : public Animal { 73 public: 74 /* Inherit the constructors */ 75 using Animal::Animal; 76 77 /* Trampoline (need one for each virtual function) */ 78 std::string go(int n_times) override { 79 PYBIND11_OVERLOAD_PURE( 80 std::string, /* Return type */ 81 Animal, /* Parent class */ 82 go, /* Name of function */ 83 n_times /* Argument(s) */ 84 ); 85 } 86 }; 87 88The macro :func:`PYBIND11_OVERLOAD_PURE` should be used for pure virtual 89functions, and :func:`PYBIND11_OVERLOAD` should be used for functions which have 90a default implementation. There are also two alternate macros 91:func:`PYBIND11_OVERLOAD_PURE_NAME` and :func:`PYBIND11_OVERLOAD_NAME` which 92take a string-valued name argument between the *Parent class* and *Name of the 93function* slots. This is useful when the C++ and Python versions of the 94function have different names, e.g. ``operator()`` vs ``__call__``. 95 96The binding code also needs a few minor adaptations (highlighted): 97 98.. code-block:: cpp 99 :emphasize-lines: 4,6,7 100 101 PYBIND11_PLUGIN(example) { 102 py::module m("example", "pybind11 example plugin"); 103 104 py::class_<Animal, PyAnimal /* <--- trampoline*/> animal(m, "Animal"); 105 animal 106 .def(py::init<>()) 107 .def("go", &Animal::go); 108 109 py::class_<Dog>(m, "Dog", animal) 110 .def(py::init<>()); 111 112 m.def("call_go", &call_go); 113 114 return m.ptr(); 115 } 116 117Importantly, pybind11 is made aware of the trampoline helper class by 118specifying it as an extra template argument to :class:`class_`. (This can also 119be combined with other template arguments such as a custom holder type; the 120order of template types does not matter). Following this, we are able to 121define a constructor as usual. 122 123Note, however, that the above is sufficient for allowing python classes to 124extend ``Animal``, but not ``Dog``: see ref:`virtual_and_inheritance` for the 125necessary steps required to providing proper overload support for inherited 126classes. 127 128The Python session below shows how to override ``Animal::go`` and invoke it via 129a virtual method call. 130 131.. code-block:: pycon 132 133 >>> from example import * 134 >>> d = Dog() 135 >>> call_go(d) 136 u'woof! woof! woof! ' 137 >>> class Cat(Animal): 138 ... def go(self, n_times): 139 ... return "meow! " * n_times 140 ... 141 >>> c = Cat() 142 >>> call_go(c) 143 u'meow! meow! meow! ' 144 145Please take a look at the :ref:`macro_notes` before using this feature. 146 147.. note:: 148 149 When the overridden type returns a reference or pointer to a type that 150 pybind11 converts from Python (for example, numeric values, std::string, 151 and other built-in value-converting types), there are some limitations to 152 be aware of: 153 154 - because in these cases there is no C++ variable to reference (the value 155 is stored in the referenced Python variable), pybind11 provides one in 156 the PYBIND11_OVERLOAD macros (when needed) with static storage duration. 157 Note that this means that invoking the overloaded method on *any* 158 instance will change the referenced value stored in *all* instances of 159 that type. 160 161 - Attempts to modify a non-const reference will not have the desired 162 effect: it will change only the static cache variable, but this change 163 will not propagate to underlying Python instance, and the change will be 164 replaced the next time the overload is invoked. 165 166.. seealso:: 167 168 The file :file:`tests/test_virtual_functions.cpp` contains a complete 169 example that demonstrates how to override virtual functions using pybind11 170 in more detail. 171 172.. _virtual_and_inheritance: 173 174Combining virtual functions and inheritance 175=========================================== 176 177When combining virtual methods with inheritance, you need to be sure to provide 178an override for each method for which you want to allow overrides from derived 179python classes. For example, suppose we extend the above ``Animal``/``Dog`` 180example as follows: 181 182.. code-block:: cpp 183 184 class Animal { 185 public: 186 virtual std::string go(int n_times) = 0; 187 virtual std::string name() { return "unknown"; } 188 }; 189 class Dog : public class Animal { 190 public: 191 std::string go(int n_times) override { 192 std::string result; 193 for (int i=0; i<n_times; ++i) 194 result += bark() + " "; 195 return result; 196 } 197 virtual std::string bark() { return "woof!"; } 198 }; 199 200then the trampoline class for ``Animal`` must, as described in the previous 201section, override ``go()`` and ``name()``, but in order to allow python code to 202inherit properly from ``Dog``, we also need a trampoline class for ``Dog`` that 203overrides both the added ``bark()`` method *and* the ``go()`` and ``name()`` 204methods inherited from ``Animal`` (even though ``Dog`` doesn't directly 205override the ``name()`` method): 206 207.. code-block:: cpp 208 209 class PyAnimal : public Animal { 210 public: 211 using Animal::Animal; // Inherit constructors 212 std::string go(int n_times) override { PYBIND11_OVERLOAD_PURE(std::string, Animal, go, n_times); } 213 std::string name() override { PYBIND11_OVERLOAD(std::string, Animal, name, ); } 214 }; 215 class PyDog : public Dog { 216 public: 217 using Dog::Dog; // Inherit constructors 218 std::string go(int n_times) override { PYBIND11_OVERLOAD_PURE(std::string, Dog, go, n_times); } 219 std::string name() override { PYBIND11_OVERLOAD(std::string, Dog, name, ); } 220 std::string bark() override { PYBIND11_OVERLOAD(std::string, Dog, bark, ); } 221 }; 222 223A registered class derived from a pybind11-registered class with virtual 224methods requires a similar trampoline class, *even if* it doesn't explicitly 225declare or override any virtual methods itself: 226 227.. code-block:: cpp 228 229 class Husky : public Dog {}; 230 class PyHusky : public Husky { 231 using Dog::Dog; // Inherit constructors 232 std::string go(int n_times) override { PYBIND11_OVERLOAD_PURE(std::string, Husky, go, n_times); } 233 std::string name() override { PYBIND11_OVERLOAD(std::string, Husky, name, ); } 234 std::string bark() override { PYBIND11_OVERLOAD(std::string, Husky, bark, ); } 235 }; 236 237There is, however, a technique that can be used to avoid this duplication 238(which can be especially helpful for a base class with several virtual 239methods). The technique involves using template trampoline classes, as 240follows: 241 242.. code-block:: cpp 243 244 template <class AnimalBase = Animal> class PyAnimal : public AnimalBase { 245 using AnimalBase::AnimalBase; // Inherit constructors 246 std::string go(int n_times) override { PYBIND11_OVERLOAD_PURE(std::string, AnimalBase, go, n_times); } 247 std::string name() override { PYBIND11_OVERLOAD(std::string, AnimalBase, name, ); } 248 }; 249 template <class DogBase = Dog> class PyDog : public PyAnimal<DogBase> { 250 using PyAnimal<DogBase>::PyAnimal; // Inherit constructors 251 // Override PyAnimal's pure virtual go() with a non-pure one: 252 std::string go(int n_times) override { PYBIND11_OVERLOAD(std::string, DogBase, go, n_times); } 253 std::string bark() override { PYBIND11_OVERLOAD(std::string, DogBase, bark, ); } 254 }; 255 256This technique has the advantage of requiring just one trampoline method to be 257declared per virtual method and pure virtual method override. It does, 258however, require the compiler to generate at least as many methods (and 259possibly more, if both pure virtual and overridden pure virtual methods are 260exposed, as above). 261 262The classes are then registered with pybind11 using: 263 264.. code-block:: cpp 265 266 py::class_<Animal, PyAnimal<>> animal(m, "Animal"); 267 py::class_<Dog, PyDog<>> dog(m, "Dog"); 268 py::class_<Husky, PyDog<Husky>> husky(m, "Husky"); 269 // ... add animal, dog, husky definitions 270 271Note that ``Husky`` did not require a dedicated trampoline template class at 272all, since it neither declares any new virtual methods nor provides any pure 273virtual method implementations. 274 275With either the repeated-virtuals or templated trampoline methods in place, you 276can now create a python class that inherits from ``Dog``: 277 278.. code-block:: python 279 280 class ShihTzu(Dog): 281 def bark(self): 282 return "yip!" 283 284.. seealso:: 285 286 See the file :file:`tests/test_virtual_functions.cpp` for complete examples 287 using both the duplication and templated trampoline approaches. 288 289Extended trampoline class functionality 290======================================= 291 292The trampoline classes described in the previous sections are, by default, only 293initialized when needed. More specifically, they are initialized when a python 294class actually inherits from a registered type (instead of merely creating an 295instance of the registered type), or when a registered constructor is only 296valid for the trampoline class but not the registered class. This is primarily 297for performance reasons: when the trampoline class is not needed for anything 298except virtual method dispatching, not initializing the trampoline class 299improves performance by avoiding needing to do a run-time check to see if the 300inheriting python instance has an overloaded method. 301 302Sometimes, however, it is useful to always initialize a trampoline class as an 303intermediate class that does more than just handle virtual method dispatching. 304For example, such a class might perform extra class initialization, extra 305destruction operations, and might define new members and methods to enable a 306more python-like interface to a class. 307 308In order to tell pybind11 that it should *always* initialize the trampoline 309class when creating new instances of a type, the class constructors should be 310declared using ``py::init_alias<Args, ...>()`` instead of the usual 311``py::init<Args, ...>()``. This forces construction via the trampoline class, 312ensuring member initialization and (eventual) destruction. 313 314.. seealso:: 315 316 See the file :file:`tests/test_alias_initialization.cpp` for complete examples 317 showing both normal and forced trampoline instantiation. 318 319.. _custom_constructors: 320 321Custom constructors 322=================== 323 324The syntax for binding constructors was previously introduced, but it only 325works when a constructor with the given parameters actually exists on the C++ 326side. To extend this to more general cases, let's take a look at what actually 327happens under the hood: the following statement 328 329.. code-block:: cpp 330 331 py::class_<Example>(m, "Example") 332 .def(py::init<int>()); 333 334is short hand notation for 335 336.. code-block:: cpp 337 338 py::class_<Example>(m, "Example") 339 .def("__init__", 340 [](Example &instance, int arg) { 341 new (&instance) Example(arg); 342 } 343 ); 344 345In other words, :func:`init` creates an anonymous function that invokes an 346in-place constructor. Memory allocation etc. is already take care of beforehand 347within pybind11. 348 349.. _classes_with_non_public_destructors: 350 351Non-public destructors 352====================== 353 354If a class has a private or protected destructor (as might e.g. be the case in 355a singleton pattern), a compile error will occur when creating bindings via 356pybind11. The underlying issue is that the ``std::unique_ptr`` holder type that 357is responsible for managing the lifetime of instances will reference the 358destructor even if no deallocations ever take place. In order to expose classes 359with private or protected destructors, it is possible to override the holder 360type via a holder type argument to ``class_``. Pybind11 provides a helper class 361``py::nodelete`` that disables any destructor invocations. In this case, it is 362crucial that instances are deallocated on the C++ side to avoid memory leaks. 363 364.. code-block:: cpp 365 366 /* ... definition ... */ 367 368 class MyClass { 369 private: 370 ~MyClass() { } 371 }; 372 373 /* ... binding code ... */ 374 375 py::class_<MyClass, std::unique_ptr<MyClass, py::nodelete>>(m, "MyClass") 376 .def(py::init<>) 377 378Implicit conversions 379==================== 380 381Suppose that instances of two types ``A`` and ``B`` are used in a project, and 382that an ``A`` can easily be converted into an instance of type ``B`` (examples of this 383could be a fixed and an arbitrary precision number type). 384 385.. code-block:: cpp 386 387 py::class_<A>(m, "A") 388 /// ... members ... 389 390 py::class_<B>(m, "B") 391 .def(py::init<A>()) 392 /// ... members ... 393 394 m.def("func", 395 [](const B &) { /* .... */ } 396 ); 397 398To invoke the function ``func`` using a variable ``a`` containing an ``A`` 399instance, we'd have to write ``func(B(a))`` in Python. On the other hand, C++ 400will automatically apply an implicit type conversion, which makes it possible 401to directly write ``func(a)``. 402 403In this situation (i.e. where ``B`` has a constructor that converts from 404``A``), the following statement enables similar implicit conversions on the 405Python side: 406 407.. code-block:: cpp 408 409 py::implicitly_convertible<A, B>(); 410 411.. note:: 412 413 Implicit conversions from ``A`` to ``B`` only work when ``B`` is a custom 414 data type that is exposed to Python via pybind11. 415 416.. _static_properties: 417 418Static properties 419================= 420 421The section on :ref:`properties` discussed the creation of instance properties 422that are implemented in terms of C++ getters and setters. 423 424Static properties can also be created in a similar way to expose getters and 425setters of static class attributes. It is important to note that the implicit 426``self`` argument also exists in this case and is used to pass the Python 427``type`` subclass instance. This parameter will often not be needed by the C++ 428side, and the following example illustrates how to instantiate a lambda getter 429function that ignores it: 430 431.. code-block:: cpp 432 433 py::class_<Foo>(m, "Foo") 434 .def_property_readonly_static("foo", [](py::object /* self */) { return Foo(); }); 435 436Operator overloading 437==================== 438 439Suppose that we're given the following ``Vector2`` class with a vector addition 440and scalar multiplication operation, all implemented using overloaded operators 441in C++. 442 443.. code-block:: cpp 444 445 class Vector2 { 446 public: 447 Vector2(float x, float y) : x(x), y(y) { } 448 449 Vector2 operator+(const Vector2 &v) const { return Vector2(x + v.x, y + v.y); } 450 Vector2 operator*(float value) const { return Vector2(x * value, y * value); } 451 Vector2& operator+=(const Vector2 &v) { x += v.x; y += v.y; return *this; } 452 Vector2& operator*=(float v) { x *= v; y *= v; return *this; } 453 454 friend Vector2 operator*(float f, const Vector2 &v) { 455 return Vector2(f * v.x, f * v.y); 456 } 457 458 std::string toString() const { 459 return "[" + std::to_string(x) + ", " + std::to_string(y) + "]"; 460 } 461 private: 462 float x, y; 463 }; 464 465The following snippet shows how the above operators can be conveniently exposed 466to Python. 467 468.. code-block:: cpp 469 470 #include <pybind11/operators.h> 471 472 PYBIND11_PLUGIN(example) { 473 py::module m("example", "pybind11 example plugin"); 474 475 py::class_<Vector2>(m, "Vector2") 476 .def(py::init<float, float>()) 477 .def(py::self + py::self) 478 .def(py::self += py::self) 479 .def(py::self *= float()) 480 .def(float() * py::self) 481 .def("__repr__", &Vector2::toString); 482 483 return m.ptr(); 484 } 485 486Note that a line like 487 488.. code-block:: cpp 489 490 .def(py::self * float()) 491 492is really just short hand notation for 493 494.. code-block:: cpp 495 496 .def("__mul__", [](const Vector2 &a, float b) { 497 return a * b; 498 }, py::is_operator()) 499 500This can be useful for exposing additional operators that don't exist on the 501C++ side, or to perform other types of customization. The ``py::is_operator`` 502flag marker is needed to inform pybind11 that this is an operator, which 503returns ``NotImplemented`` when invoked with incompatible arguments rather than 504throwing a type error. 505 506.. note:: 507 508 To use the more convenient ``py::self`` notation, the additional 509 header file :file:`pybind11/operators.h` must be included. 510 511.. seealso:: 512 513 The file :file:`tests/test_operator_overloading.cpp` contains a 514 complete example that demonstrates how to work with overloaded operators in 515 more detail. 516 517Pickling support 518================ 519 520Python's ``pickle`` module provides a powerful facility to serialize and 521de-serialize a Python object graph into a binary data stream. To pickle and 522unpickle C++ classes using pybind11, two additional functions must be provided. 523Suppose the class in question has the following signature: 524 525.. code-block:: cpp 526 527 class Pickleable { 528 public: 529 Pickleable(const std::string &value) : m_value(value) { } 530 const std::string &value() const { return m_value; } 531 532 void setExtra(int extra) { m_extra = extra; } 533 int extra() const { return m_extra; } 534 private: 535 std::string m_value; 536 int m_extra = 0; 537 }; 538 539The binding code including the requisite ``__setstate__`` and ``__getstate__`` methods [#f3]_ 540looks as follows: 541 542.. code-block:: cpp 543 544 py::class_<Pickleable>(m, "Pickleable") 545 .def(py::init<std::string>()) 546 .def("value", &Pickleable::value) 547 .def("extra", &Pickleable::extra) 548 .def("setExtra", &Pickleable::setExtra) 549 .def("__getstate__", [](const Pickleable &p) { 550 /* Return a tuple that fully encodes the state of the object */ 551 return py::make_tuple(p.value(), p.extra()); 552 }) 553 .def("__setstate__", [](Pickleable &p, py::tuple t) { 554 if (t.size() != 2) 555 throw std::runtime_error("Invalid state!"); 556 557 /* Invoke the in-place constructor. Note that this is needed even 558 when the object just has a trivial default constructor */ 559 new (&p) Pickleable(t[0].cast<std::string>()); 560 561 /* Assign any additional state */ 562 p.setExtra(t[1].cast<int>()); 563 }); 564 565An instance can now be pickled as follows: 566 567.. code-block:: python 568 569 try: 570 import cPickle as pickle # Use cPickle on Python 2.7 571 except ImportError: 572 import pickle 573 574 p = Pickleable("test_value") 575 p.setExtra(15) 576 data = pickle.dumps(p, 2) 577 578Note that only the cPickle module is supported on Python 2.7. The second 579argument to ``dumps`` is also crucial: it selects the pickle protocol version 5802, since the older version 1 is not supported. Newer versions are also fine—for 581instance, specify ``-1`` to always use the latest available version. Beware: 582failure to follow these instructions will cause important pybind11 memory 583allocation routines to be skipped during unpickling, which will likely lead to 584memory corruption and/or segmentation faults. 585 586.. seealso:: 587 588 The file :file:`tests/test_pickling.cpp` contains a complete example 589 that demonstrates how to pickle and unpickle types using pybind11 in more 590 detail. 591 592.. [#f3] http://docs.python.org/3/library/pickle.html#pickling-class-instances 593 594Multiple Inheritance 595==================== 596 597pybind11 can create bindings for types that derive from multiple base types 598(aka. *multiple inheritance*). To do so, specify all bases in the template 599arguments of the ``class_`` declaration: 600 601.. code-block:: cpp 602 603 py::class_<MyType, BaseType1, BaseType2, BaseType3>(m, "MyType") 604 ... 605 606The base types can be specified in arbitrary order, and they can even be 607interspersed with alias types and holder types (discussed earlier in this 608document)---pybind11 will automatically find out which is which. The only 609requirement is that the first template argument is the type to be declared. 610 611There are two caveats regarding the implementation of this feature: 612 6131. When only one base type is specified for a C++ type that actually has 614 multiple bases, pybind11 will assume that it does not participate in 615 multiple inheritance, which can lead to undefined behavior. In such cases, 616 add the tag ``multiple_inheritance``: 617 618 .. code-block:: cpp 619 620 py::class_<MyType, BaseType2>(m, "MyType", py::multiple_inheritance()); 621 622 The tag is redundant and does not need to be specified when multiple base 623 types are listed. 624 6252. As was previously discussed in the section on :ref:`overriding_virtuals`, it 626 is easy to create Python types that derive from C++ classes. It is even 627 possible to make use of multiple inheritance to declare a Python class which 628 has e.g. a C++ and a Python class as bases. However, any attempt to create a 629 type that has *two or more* C++ classes in its hierarchy of base types will 630 fail with a fatal error message: ``TypeError: multiple bases have instance 631 lay-out conflict``. Core Python types that are implemented in C (e.g. 632 ``dict``, ``list``, ``Exception``, etc.) also fall under this combination 633 and cannot be combined with C++ types bound using pybind11 via multiple 634 inheritance. 635