classes.rst revision 12037:d28054ac6ec9
1Classes 2####### 3 4This section presents advanced binding code for classes and it is assumed 5that you are already familiar with the basics from :doc:`/classes`. 6 7.. _overriding_virtuals: 8 9Overriding virtual functions in Python 10====================================== 11 12Suppose that a C++ class or interface has a virtual function that we'd like to 13to override from within Python (we'll focus on the class ``Animal``; ``Dog`` is 14given as a specific example of how one would do this with traditional C++ 15code). 16 17.. code-block:: cpp 18 19 class Animal { 20 public: 21 virtual ~Animal() { } 22 virtual std::string go(int n_times) = 0; 23 }; 24 25 class Dog : public Animal { 26 public: 27 std::string go(int n_times) override { 28 std::string result; 29 for (int i=0; i<n_times; ++i) 30 result += "woof! "; 31 return result; 32 } 33 }; 34 35Let's also suppose that we are given a plain function which calls the 36function ``go()`` on an arbitrary ``Animal`` instance. 37 38.. code-block:: cpp 39 40 std::string call_go(Animal *animal) { 41 return animal->go(3); 42 } 43 44Normally, the binding code for these classes would look as follows: 45 46.. code-block:: cpp 47 48 PYBIND11_PLUGIN(example) { 49 py::module m("example", "pybind11 example plugin"); 50 51 py::class_<Animal> animal(m, "Animal"); 52 animal 53 .def("go", &Animal::go); 54 55 py::class_<Dog>(m, "Dog", animal) 56 .def(py::init<>()); 57 58 m.def("call_go", &call_go); 59 60 return m.ptr(); 61 } 62 63However, these bindings are impossible to extend: ``Animal`` is not 64constructible, and we clearly require some kind of "trampoline" that 65redirects virtual calls back to Python. 66 67Defining a new type of ``Animal`` from within Python is possible but requires a 68helper class that is defined as follows: 69 70.. code-block:: cpp 71 72 class PyAnimal : public Animal { 73 public: 74 /* Inherit the constructors */ 75 using Animal::Animal; 76 77 /* Trampoline (need one for each virtual function) */ 78 std::string go(int n_times) override { 79 PYBIND11_OVERLOAD_PURE( 80 std::string, /* Return type */ 81 Animal, /* Parent class */ 82 go, /* Name of function in C++ (must match Python name) */ 83 n_times /* Argument(s) */ 84 ); 85 } 86 }; 87 88The macro :func:`PYBIND11_OVERLOAD_PURE` should be used for pure virtual 89functions, and :func:`PYBIND11_OVERLOAD` should be used for functions which have 90a default implementation. There are also two alternate macros 91:func:`PYBIND11_OVERLOAD_PURE_NAME` and :func:`PYBIND11_OVERLOAD_NAME` which 92take a string-valued name argument between the *Parent class* and *Name of the 93function* slots, which defines the name of function in Python. This is required 94when the C++ and Python versions of the 95function have different names, e.g. ``operator()`` vs ``__call__``. 96 97The binding code also needs a few minor adaptations (highlighted): 98 99.. code-block:: cpp 100 :emphasize-lines: 4,6,7 101 102 PYBIND11_PLUGIN(example) { 103 py::module m("example", "pybind11 example plugin"); 104 105 py::class_<Animal, PyAnimal /* <--- trampoline*/> animal(m, "Animal"); 106 animal 107 .def(py::init<>()) 108 .def("go", &Animal::go); 109 110 py::class_<Dog>(m, "Dog", animal) 111 .def(py::init<>()); 112 113 m.def("call_go", &call_go); 114 115 return m.ptr(); 116 } 117 118Importantly, pybind11 is made aware of the trampoline helper class by 119specifying it as an extra template argument to :class:`class_`. (This can also 120be combined with other template arguments such as a custom holder type; the 121order of template types does not matter). Following this, we are able to 122define a constructor as usual. 123 124Bindings should be made against the actual class, not the trampoline helper class. 125 126.. code-block:: cpp 127 128 py::class_<Animal, PyAnimal /* <--- trampoline*/> animal(m, "Animal"); 129 animal 130 .def(py::init<>()) 131 .def("go", &PyAnimal::go); /* <--- THIS IS WRONG, use &Animal::go */ 132 133Note, however, that the above is sufficient for allowing python classes to 134extend ``Animal``, but not ``Dog``: see ref:`virtual_and_inheritance` for the 135necessary steps required to providing proper overload support for inherited 136classes. 137 138The Python session below shows how to override ``Animal::go`` and invoke it via 139a virtual method call. 140 141.. code-block:: pycon 142 143 >>> from example import * 144 >>> d = Dog() 145 >>> call_go(d) 146 u'woof! woof! woof! ' 147 >>> class Cat(Animal): 148 ... def go(self, n_times): 149 ... return "meow! " * n_times 150 ... 151 >>> c = Cat() 152 >>> call_go(c) 153 u'meow! meow! meow! ' 154 155Please take a look at the :ref:`macro_notes` before using this feature. 156 157.. note:: 158 159 When the overridden type returns a reference or pointer to a type that 160 pybind11 converts from Python (for example, numeric values, std::string, 161 and other built-in value-converting types), there are some limitations to 162 be aware of: 163 164 - because in these cases there is no C++ variable to reference (the value 165 is stored in the referenced Python variable), pybind11 provides one in 166 the PYBIND11_OVERLOAD macros (when needed) with static storage duration. 167 Note that this means that invoking the overloaded method on *any* 168 instance will change the referenced value stored in *all* instances of 169 that type. 170 171 - Attempts to modify a non-const reference will not have the desired 172 effect: it will change only the static cache variable, but this change 173 will not propagate to underlying Python instance, and the change will be 174 replaced the next time the overload is invoked. 175 176.. seealso:: 177 178 The file :file:`tests/test_virtual_functions.cpp` contains a complete 179 example that demonstrates how to override virtual functions using pybind11 180 in more detail. 181 182.. _virtual_and_inheritance: 183 184Combining virtual functions and inheritance 185=========================================== 186 187When combining virtual methods with inheritance, you need to be sure to provide 188an override for each method for which you want to allow overrides from derived 189python classes. For example, suppose we extend the above ``Animal``/``Dog`` 190example as follows: 191 192.. code-block:: cpp 193 194 class Animal { 195 public: 196 virtual std::string go(int n_times) = 0; 197 virtual std::string name() { return "unknown"; } 198 }; 199 class Dog : public Animal { 200 public: 201 std::string go(int n_times) override { 202 std::string result; 203 for (int i=0; i<n_times; ++i) 204 result += bark() + " "; 205 return result; 206 } 207 virtual std::string bark() { return "woof!"; } 208 }; 209 210then the trampoline class for ``Animal`` must, as described in the previous 211section, override ``go()`` and ``name()``, but in order to allow python code to 212inherit properly from ``Dog``, we also need a trampoline class for ``Dog`` that 213overrides both the added ``bark()`` method *and* the ``go()`` and ``name()`` 214methods inherited from ``Animal`` (even though ``Dog`` doesn't directly 215override the ``name()`` method): 216 217.. code-block:: cpp 218 219 class PyAnimal : public Animal { 220 public: 221 using Animal::Animal; // Inherit constructors 222 std::string go(int n_times) override { PYBIND11_OVERLOAD_PURE(std::string, Animal, go, n_times); } 223 std::string name() override { PYBIND11_OVERLOAD(std::string, Animal, name, ); } 224 }; 225 class PyDog : public Dog { 226 public: 227 using Dog::Dog; // Inherit constructors 228 std::string go(int n_times) override { PYBIND11_OVERLOAD_PURE(std::string, Dog, go, n_times); } 229 std::string name() override { PYBIND11_OVERLOAD(std::string, Dog, name, ); } 230 std::string bark() override { PYBIND11_OVERLOAD(std::string, Dog, bark, ); } 231 }; 232 233.. note:: 234 235 Note the trailing commas in the ``PYBIND11_OVERLOAD`` calls to ``name()`` 236 and ``bark()``. These are needed to portably implement a trampoline for a 237 function that does not take any arguments. For functions that take 238 a nonzero number of arguments, the trailing comma must be omitted. 239 240A registered class derived from a pybind11-registered class with virtual 241methods requires a similar trampoline class, *even if* it doesn't explicitly 242declare or override any virtual methods itself: 243 244.. code-block:: cpp 245 246 class Husky : public Dog {}; 247 class PyHusky : public Husky { 248 public: 249 using Husky::Husky; // Inherit constructors 250 std::string go(int n_times) override { PYBIND11_OVERLOAD_PURE(std::string, Husky, go, n_times); } 251 std::string name() override { PYBIND11_OVERLOAD(std::string, Husky, name, ); } 252 std::string bark() override { PYBIND11_OVERLOAD(std::string, Husky, bark, ); } 253 }; 254 255There is, however, a technique that can be used to avoid this duplication 256(which can be especially helpful for a base class with several virtual 257methods). The technique involves using template trampoline classes, as 258follows: 259 260.. code-block:: cpp 261 262 template <class AnimalBase = Animal> class PyAnimal : public AnimalBase { 263 public: 264 using AnimalBase::AnimalBase; // Inherit constructors 265 std::string go(int n_times) override { PYBIND11_OVERLOAD_PURE(std::string, AnimalBase, go, n_times); } 266 std::string name() override { PYBIND11_OVERLOAD(std::string, AnimalBase, name, ); } 267 }; 268 template <class DogBase = Dog> class PyDog : public PyAnimal<DogBase> { 269 public: 270 using PyAnimal<DogBase>::PyAnimal; // Inherit constructors 271 // Override PyAnimal's pure virtual go() with a non-pure one: 272 std::string go(int n_times) override { PYBIND11_OVERLOAD(std::string, DogBase, go, n_times); } 273 std::string bark() override { PYBIND11_OVERLOAD(std::string, DogBase, bark, ); } 274 }; 275 276This technique has the advantage of requiring just one trampoline method to be 277declared per virtual method and pure virtual method override. It does, 278however, require the compiler to generate at least as many methods (and 279possibly more, if both pure virtual and overridden pure virtual methods are 280exposed, as above). 281 282The classes are then registered with pybind11 using: 283 284.. code-block:: cpp 285 286 py::class_<Animal, PyAnimal<>> animal(m, "Animal"); 287 py::class_<Dog, PyDog<>> dog(m, "Dog"); 288 py::class_<Husky, PyDog<Husky>> husky(m, "Husky"); 289 // ... add animal, dog, husky definitions 290 291Note that ``Husky`` did not require a dedicated trampoline template class at 292all, since it neither declares any new virtual methods nor provides any pure 293virtual method implementations. 294 295With either the repeated-virtuals or templated trampoline methods in place, you 296can now create a python class that inherits from ``Dog``: 297 298.. code-block:: python 299 300 class ShihTzu(Dog): 301 def bark(self): 302 return "yip!" 303 304.. seealso:: 305 306 See the file :file:`tests/test_virtual_functions.cpp` for complete examples 307 using both the duplication and templated trampoline approaches. 308 309Extended trampoline class functionality 310======================================= 311 312The trampoline classes described in the previous sections are, by default, only 313initialized when needed. More specifically, they are initialized when a python 314class actually inherits from a registered type (instead of merely creating an 315instance of the registered type), or when a registered constructor is only 316valid for the trampoline class but not the registered class. This is primarily 317for performance reasons: when the trampoline class is not needed for anything 318except virtual method dispatching, not initializing the trampoline class 319improves performance by avoiding needing to do a run-time check to see if the 320inheriting python instance has an overloaded method. 321 322Sometimes, however, it is useful to always initialize a trampoline class as an 323intermediate class that does more than just handle virtual method dispatching. 324For example, such a class might perform extra class initialization, extra 325destruction operations, and might define new members and methods to enable a 326more python-like interface to a class. 327 328In order to tell pybind11 that it should *always* initialize the trampoline 329class when creating new instances of a type, the class constructors should be 330declared using ``py::init_alias<Args, ...>()`` instead of the usual 331``py::init<Args, ...>()``. This forces construction via the trampoline class, 332ensuring member initialization and (eventual) destruction. 333 334.. seealso:: 335 336 See the file :file:`tests/test_alias_initialization.cpp` for complete examples 337 showing both normal and forced trampoline instantiation. 338 339.. _custom_constructors: 340 341Custom constructors 342=================== 343 344The syntax for binding constructors was previously introduced, but it only 345works when a constructor with the given parameters actually exists on the C++ 346side. To extend this to more general cases, let's take a look at what actually 347happens under the hood: the following statement 348 349.. code-block:: cpp 350 351 py::class_<Example>(m, "Example") 352 .def(py::init<int>()); 353 354is short hand notation for 355 356.. code-block:: cpp 357 358 py::class_<Example>(m, "Example") 359 .def("__init__", 360 [](Example &instance, int arg) { 361 new (&instance) Example(arg); 362 } 363 ); 364 365In other words, :func:`init` creates an anonymous function that invokes an 366in-place constructor. Memory allocation etc. is already take care of beforehand 367within pybind11. 368 369.. _classes_with_non_public_destructors: 370 371Non-public destructors 372====================== 373 374If a class has a private or protected destructor (as might e.g. be the case in 375a singleton pattern), a compile error will occur when creating bindings via 376pybind11. The underlying issue is that the ``std::unique_ptr`` holder type that 377is responsible for managing the lifetime of instances will reference the 378destructor even if no deallocations ever take place. In order to expose classes 379with private or protected destructors, it is possible to override the holder 380type via a holder type argument to ``class_``. Pybind11 provides a helper class 381``py::nodelete`` that disables any destructor invocations. In this case, it is 382crucial that instances are deallocated on the C++ side to avoid memory leaks. 383 384.. code-block:: cpp 385 386 /* ... definition ... */ 387 388 class MyClass { 389 private: 390 ~MyClass() { } 391 }; 392 393 /* ... binding code ... */ 394 395 py::class_<MyClass, std::unique_ptr<MyClass, py::nodelete>>(m, "MyClass") 396 .def(py::init<>()) 397 398.. _implicit_conversions: 399 400Implicit conversions 401==================== 402 403Suppose that instances of two types ``A`` and ``B`` are used in a project, and 404that an ``A`` can easily be converted into an instance of type ``B`` (examples of this 405could be a fixed and an arbitrary precision number type). 406 407.. code-block:: cpp 408 409 py::class_<A>(m, "A") 410 /// ... members ... 411 412 py::class_<B>(m, "B") 413 .def(py::init<A>()) 414 /// ... members ... 415 416 m.def("func", 417 [](const B &) { /* .... */ } 418 ); 419 420To invoke the function ``func`` using a variable ``a`` containing an ``A`` 421instance, we'd have to write ``func(B(a))`` in Python. On the other hand, C++ 422will automatically apply an implicit type conversion, which makes it possible 423to directly write ``func(a)``. 424 425In this situation (i.e. where ``B`` has a constructor that converts from 426``A``), the following statement enables similar implicit conversions on the 427Python side: 428 429.. code-block:: cpp 430 431 py::implicitly_convertible<A, B>(); 432 433.. note:: 434 435 Implicit conversions from ``A`` to ``B`` only work when ``B`` is a custom 436 data type that is exposed to Python via pybind11. 437 438.. _static_properties: 439 440Static properties 441================= 442 443The section on :ref:`properties` discussed the creation of instance properties 444that are implemented in terms of C++ getters and setters. 445 446Static properties can also be created in a similar way to expose getters and 447setters of static class attributes. Note that the implicit ``self`` argument 448also exists in this case and is used to pass the Python ``type`` subclass 449instance. This parameter will often not be needed by the C++ side, and the 450following example illustrates how to instantiate a lambda getter function 451that ignores it: 452 453.. code-block:: cpp 454 455 py::class_<Foo>(m, "Foo") 456 .def_property_readonly_static("foo", [](py::object /* self */) { return Foo(); }); 457 458Operator overloading 459==================== 460 461Suppose that we're given the following ``Vector2`` class with a vector addition 462and scalar multiplication operation, all implemented using overloaded operators 463in C++. 464 465.. code-block:: cpp 466 467 class Vector2 { 468 public: 469 Vector2(float x, float y) : x(x), y(y) { } 470 471 Vector2 operator+(const Vector2 &v) const { return Vector2(x + v.x, y + v.y); } 472 Vector2 operator*(float value) const { return Vector2(x * value, y * value); } 473 Vector2& operator+=(const Vector2 &v) { x += v.x; y += v.y; return *this; } 474 Vector2& operator*=(float v) { x *= v; y *= v; return *this; } 475 476 friend Vector2 operator*(float f, const Vector2 &v) { 477 return Vector2(f * v.x, f * v.y); 478 } 479 480 std::string toString() const { 481 return "[" + std::to_string(x) + ", " + std::to_string(y) + "]"; 482 } 483 private: 484 float x, y; 485 }; 486 487The following snippet shows how the above operators can be conveniently exposed 488to Python. 489 490.. code-block:: cpp 491 492 #include <pybind11/operators.h> 493 494 PYBIND11_PLUGIN(example) { 495 py::module m("example", "pybind11 example plugin"); 496 497 py::class_<Vector2>(m, "Vector2") 498 .def(py::init<float, float>()) 499 .def(py::self + py::self) 500 .def(py::self += py::self) 501 .def(py::self *= float()) 502 .def(float() * py::self) 503 .def(py::self * float()) 504 .def("__repr__", &Vector2::toString); 505 506 return m.ptr(); 507 } 508 509Note that a line like 510 511.. code-block:: cpp 512 513 .def(py::self * float()) 514 515is really just short hand notation for 516 517.. code-block:: cpp 518 519 .def("__mul__", [](const Vector2 &a, float b) { 520 return a * b; 521 }, py::is_operator()) 522 523This can be useful for exposing additional operators that don't exist on the 524C++ side, or to perform other types of customization. The ``py::is_operator`` 525flag marker is needed to inform pybind11 that this is an operator, which 526returns ``NotImplemented`` when invoked with incompatible arguments rather than 527throwing a type error. 528 529.. note:: 530 531 To use the more convenient ``py::self`` notation, the additional 532 header file :file:`pybind11/operators.h` must be included. 533 534.. seealso:: 535 536 The file :file:`tests/test_operator_overloading.cpp` contains a 537 complete example that demonstrates how to work with overloaded operators in 538 more detail. 539 540Pickling support 541================ 542 543Python's ``pickle`` module provides a powerful facility to serialize and 544de-serialize a Python object graph into a binary data stream. To pickle and 545unpickle C++ classes using pybind11, two additional functions must be provided. 546Suppose the class in question has the following signature: 547 548.. code-block:: cpp 549 550 class Pickleable { 551 public: 552 Pickleable(const std::string &value) : m_value(value) { } 553 const std::string &value() const { return m_value; } 554 555 void setExtra(int extra) { m_extra = extra; } 556 int extra() const { return m_extra; } 557 private: 558 std::string m_value; 559 int m_extra = 0; 560 }; 561 562The binding code including the requisite ``__setstate__`` and ``__getstate__`` methods [#f3]_ 563looks as follows: 564 565.. code-block:: cpp 566 567 py::class_<Pickleable>(m, "Pickleable") 568 .def(py::init<std::string>()) 569 .def("value", &Pickleable::value) 570 .def("extra", &Pickleable::extra) 571 .def("setExtra", &Pickleable::setExtra) 572 .def("__getstate__", [](const Pickleable &p) { 573 /* Return a tuple that fully encodes the state of the object */ 574 return py::make_tuple(p.value(), p.extra()); 575 }) 576 .def("__setstate__", [](Pickleable &p, py::tuple t) { 577 if (t.size() != 2) 578 throw std::runtime_error("Invalid state!"); 579 580 /* Invoke the in-place constructor. Note that this is needed even 581 when the object just has a trivial default constructor */ 582 new (&p) Pickleable(t[0].cast<std::string>()); 583 584 /* Assign any additional state */ 585 p.setExtra(t[1].cast<int>()); 586 }); 587 588An instance can now be pickled as follows: 589 590.. code-block:: python 591 592 try: 593 import cPickle as pickle # Use cPickle on Python 2.7 594 except ImportError: 595 import pickle 596 597 p = Pickleable("test_value") 598 p.setExtra(15) 599 data = pickle.dumps(p, 2) 600 601Note that only the cPickle module is supported on Python 2.7. The second 602argument to ``dumps`` is also crucial: it selects the pickle protocol version 6032, since the older version 1 is not supported. Newer versions are also fine—for 604instance, specify ``-1`` to always use the latest available version. Beware: 605failure to follow these instructions will cause important pybind11 memory 606allocation routines to be skipped during unpickling, which will likely lead to 607memory corruption and/or segmentation faults. 608 609.. seealso:: 610 611 The file :file:`tests/test_pickling.cpp` contains a complete example 612 that demonstrates how to pickle and unpickle types using pybind11 in more 613 detail. 614 615.. [#f3] http://docs.python.org/3/library/pickle.html#pickling-class-instances 616 617Multiple Inheritance 618==================== 619 620pybind11 can create bindings for types that derive from multiple base types 621(aka. *multiple inheritance*). To do so, specify all bases in the template 622arguments of the ``class_`` declaration: 623 624.. code-block:: cpp 625 626 py::class_<MyType, BaseType1, BaseType2, BaseType3>(m, "MyType") 627 ... 628 629The base types can be specified in arbitrary order, and they can even be 630interspersed with alias types and holder types (discussed earlier in this 631document)---pybind11 will automatically find out which is which. The only 632requirement is that the first template argument is the type to be declared. 633 634There are two caveats regarding the implementation of this feature: 635 6361. When only one base type is specified for a C++ type that actually has 637 multiple bases, pybind11 will assume that it does not participate in 638 multiple inheritance, which can lead to undefined behavior. In such cases, 639 add the tag ``multiple_inheritance``: 640 641 .. code-block:: cpp 642 643 py::class_<MyType, BaseType2>(m, "MyType", py::multiple_inheritance()); 644 645 The tag is redundant and does not need to be specified when multiple base 646 types are listed. 647 6482. As was previously discussed in the section on :ref:`overriding_virtuals`, it 649 is easy to create Python types that derive from C++ classes. It is even 650 possible to make use of multiple inheritance to declare a Python class which 651 has e.g. a C++ and a Python class as bases. However, any attempt to create a 652 type that has *two or more* C++ classes in its hierarchy of base types will 653 fail with a fatal error message: ``TypeError: multiple bases have instance 654 lay-out conflict``. Core Python types that are implemented in C (e.g. 655 ``dict``, ``list``, ``Exception``, etc.) also fall under this combination 656 and cannot be combined with C++ types bound using pybind11 via multiple 657 inheritance. 658