functions.rst (11986:c12e4625ab56) | functions.rst (12037:d28054ac6ec9) |
---|---|
1Functions 2######### 3 4Before proceeding with this section, make sure that you are already familiar 5with the basics of binding functions and classes, as explained in :doc:`/basics` 6and :doc:`/classes`. The following guide is applicable to both free and member 7functions, i.e. *methods* in Python. 8 | 1Functions 2######### 3 4Before proceeding with this section, make sure that you are already familiar 5with the basics of binding functions and classes, as explained in :doc:`/basics` 6and :doc:`/classes`. The following guide is applicable to both free and member 7functions, i.e. *methods* in Python. 8 |
9.. _return_value_policies: 10 |
|
9Return value policies 10===================== 11 12Python and C++ use fundamentally different ways of managing the memory and 13lifetime of objects managed by them. This can lead to issues when creating 14bindings for functions that return a non-trivial type. Just by looking at the 15type information, it is not clear whether Python should take charge of the 16returned value and eventually free its resources, or if this is handled on the | 11Return value policies 12===================== 13 14Python and C++ use fundamentally different ways of managing the memory and 15lifetime of objects managed by them. This can lead to issues when creating 16bindings for functions that return a non-trivial type. Just by looking at the 17type information, it is not clear whether Python should take charge of the 18returned value and eventually free its resources, or if this is handled on the |
17C++ side. For this reason, pybind11 provides a several `return value policy` | 19C++ side. For this reason, pybind11 provides a several *return value policy* |
18annotations that can be passed to the :func:`module::def` and 19:func:`class_::def` functions. The default policy is 20:enum:`return_value_policy::automatic`. 21 22Return value policies are tricky, and it's very important to get them right. 23Just to illustrate what can go wrong, consider the following simple example: 24 25.. code-block:: cpp 26 | 20annotations that can be passed to the :func:`module::def` and 21:func:`class_::def` functions. The default policy is 22:enum:`return_value_policy::automatic`. 23 24Return value policies are tricky, and it's very important to get them right. 25Just to illustrate what can go wrong, consider the following simple example: 26 27.. code-block:: cpp 28 |
27 /* Function declaration */ | 29 /* Function declaration */ |
28 Data *get_data() { return _data; /* (pointer to a static data structure) */ } 29 ... 30 | 30 Data *get_data() { return _data; /* (pointer to a static data structure) */ } 31 ... 32 |
31 /* Binding code */ | 33 /* Binding code */ |
32 m.def("get_data", &get_data); // <-- KABOOM, will cause crash when called from Python 33 34What's going on here? When ``get_data()`` is called from Python, the return 35value (a native C++ type) must be wrapped to turn it into a usable Python type. 36In this case, the default return value policy (:enum:`return_value_policy::automatic`) 37causes pybind11 to assume ownership of the static ``_data`` instance. 38 39When Python's garbage collector eventually deletes the Python 40wrapper, pybind11 will also attempt to delete the C++ instance (via ``operator 41delete()``) due to the implied ownership. At this point, the entire application 42will come crashing down, though errors could also be more subtle and involve 43silent data corruption. 44 45In the above example, the policy :enum:`return_value_policy::reference` should have 46been specified so that the global data instance is only *referenced* without any | 34 m.def("get_data", &get_data); // <-- KABOOM, will cause crash when called from Python 35 36What's going on here? When ``get_data()`` is called from Python, the return 37value (a native C++ type) must be wrapped to turn it into a usable Python type. 38In this case, the default return value policy (:enum:`return_value_policy::automatic`) 39causes pybind11 to assume ownership of the static ``_data`` instance. 40 41When Python's garbage collector eventually deletes the Python 42wrapper, pybind11 will also attempt to delete the C++ instance (via ``operator 43delete()``) due to the implied ownership. At this point, the entire application 44will come crashing down, though errors could also be more subtle and involve 45silent data corruption. 46 47In the above example, the policy :enum:`return_value_policy::reference` should have 48been specified so that the global data instance is only *referenced* without any |
47implied transfer of ownership, i.e.: | 49implied transfer of ownership, i.e.: |
48 49.. code-block:: cpp 50 51 m.def("get_data", &get_data, return_value_policy::reference); 52 53On the other hand, this is not the right policy for many other situations, 54where ignoring ownership could lead to resource leaks. 55As a developer using pybind11, it's important to be familiar with the different --- 27 unchanged lines hidden (view full) --- 83| | of a parent object, namely the implicit ``this``, or ``self`` argument of | 84| | the called method or property. Internally, this policy works just like | 85| | :enum:`return_value_policy::reference` but additionally applies a | 86| | ``keep_alive<0, 1>`` *call policy* (described in the next section) that | 87| | prevents the parent object from being garbage collected as long as the | 88| | return value is referenced by Python. This is the default policy for | 89| | property getters created via ``def_property``, ``def_readwrite``, etc. | 90+--------------------------------------------------+----------------------------------------------------------------------------+ | 50 51.. code-block:: cpp 52 53 m.def("get_data", &get_data, return_value_policy::reference); 54 55On the other hand, this is not the right policy for many other situations, 56where ignoring ownership could lead to resource leaks. 57As a developer using pybind11, it's important to be familiar with the different --- 27 unchanged lines hidden (view full) --- 85| | of a parent object, namely the implicit ``this``, or ``self`` argument of | 86| | the called method or property. Internally, this policy works just like | 87| | :enum:`return_value_policy::reference` but additionally applies a | 88| | ``keep_alive<0, 1>`` *call policy* (described in the next section) that | 89| | prevents the parent object from being garbage collected as long as the | 90| | return value is referenced by Python. This is the default policy for | 91| | property getters created via ``def_property``, ``def_readwrite``, etc. | 92+--------------------------------------------------+----------------------------------------------------------------------------+ |
91| :enum:`return_value_policy::automatic` | This is the default return value policy, which falls back to the policy | | 93| :enum:`return_value_policy::automatic` | **Default policy.** This policy falls back to the policy | |
92| | :enum:`return_value_policy::take_ownership` when the return value is a | | 94| | :enum:`return_value_policy::take_ownership` when the return value is a | |
93| | pointer. Otherwise, it uses :enum:`return_value::move` or | 94| | :enum:`return_value::copy` for rvalue and lvalue references, respectively. | 95| | See above for a description of what all of these different policies do. | | 95| | pointer. Otherwise, it uses :enum:`return_value_policy::move` or | 96| | :enum:`return_value_policy::copy` for rvalue and lvalue references, | 97| | respectively. See above for a description of what all of these different | 98| | policies do. | |
96+--------------------------------------------------+----------------------------------------------------------------------------+ 97| :enum:`return_value_policy::automatic_reference` | As above, but use policy :enum:`return_value_policy::reference` when the | 98| | return value is a pointer. This is the default conversion policy for | 99| | function arguments when calling Python functions manually from C++ code | 100| | (i.e. via handle::operator()). You probably won't need to use this. | 101+--------------------------------------------------+----------------------------------------------------------------------------+ 102 103Return value policies can also be applied to properties: --- 49 unchanged lines hidden (view full) --- 153 can lead to crashes or undefined behavior. For functions returning smart 154 pointers, it is not necessary to specify a return value policy. 155 156.. _call_policies: 157 158Additional call policies 159======================== 160 | 99+--------------------------------------------------+----------------------------------------------------------------------------+ 100| :enum:`return_value_policy::automatic_reference` | As above, but use policy :enum:`return_value_policy::reference` when the | 101| | return value is a pointer. This is the default conversion policy for | 102| | function arguments when calling Python functions manually from C++ code | 103| | (i.e. via handle::operator()). You probably won't need to use this. | 104+--------------------------------------------------+----------------------------------------------------------------------------+ 105 106Return value policies can also be applied to properties: --- 49 unchanged lines hidden (view full) --- 156 can lead to crashes or undefined behavior. For functions returning smart 157 pointers, it is not necessary to specify a return value policy. 158 159.. _call_policies: 160 161Additional call policies 162======================== 163 |
161In addition to the above return value policies, further `call policies` can be 162specified to indicate dependencies between parameters. There is currently just | 164In addition to the above return value policies, further *call policies* can be 165specified to indicate dependencies between parameters. In general, call policies 166are required when the C++ object is any kind of container and another object is being 167added to the container. 168 169There is currently just |
163one policy named ``keep_alive<Nurse, Patient>``, which indicates that the 164argument with index ``Patient`` should be kept alive at least until the 165argument with index ``Nurse`` is freed by the garbage collector. Argument 166indices start at one, while zero refers to the return value. For methods, index 167``1`` refers to the implicit ``this`` pointer, while regular arguments begin at 168index ``2``. Arbitrarily many call policies can be specified. When a ``Nurse`` 169with value ``None`` is detected at runtime, the call policy does nothing. 170 --- 31 unchanged lines hidden (view full) --- 202makes it possible to directly work with native Python types on the C++ side. 203For instance, the following statement iterates over a Python ``dict``: 204 205.. code-block:: cpp 206 207 void print_dict(py::dict dict) { 208 /* Easily interact with Python types */ 209 for (auto item : dict) | 170one policy named ``keep_alive<Nurse, Patient>``, which indicates that the 171argument with index ``Patient`` should be kept alive at least until the 172argument with index ``Nurse`` is freed by the garbage collector. Argument 173indices start at one, while zero refers to the return value. For methods, index 174``1`` refers to the implicit ``this`` pointer, while regular arguments begin at 175index ``2``. Arbitrarily many call policies can be specified. When a ``Nurse`` 176with value ``None`` is detected at runtime, the call policy does nothing. 177 --- 31 unchanged lines hidden (view full) --- 209makes it possible to directly work with native Python types on the C++ side. 210For instance, the following statement iterates over a Python ``dict``: 211 212.. code-block:: cpp 213 214 void print_dict(py::dict dict) { 215 /* Easily interact with Python types */ 216 for (auto item : dict) |
210 std::cout << "key=" << item.first << ", " 211 << "value=" << item.second << std::endl; | 217 std::cout << "key=" << std::string(py::str(item.first)) << ", " 218 << "value=" << std::string(py::str(item.second)) << std::endl; |
212 } 213 214It can be exported: 215 216.. code-block:: cpp 217 218 m.def("print_dict", &print_dict); 219 --- 27 unchanged lines hidden (view full) --- 247 if (kwargs) 248 /// .. do something with kwargs 249 } 250 251 /// Binding code 252 m.def("generic", &generic); 253 254The class ``py::args`` derives from ``py::tuple`` and ``py::kwargs`` derives | 219 } 220 221It can be exported: 222 223.. code-block:: cpp 224 225 m.def("print_dict", &print_dict); 226 --- 27 unchanged lines hidden (view full) --- 254 if (kwargs) 255 /// .. do something with kwargs 256 } 257 258 /// Binding code 259 m.def("generic", &generic); 260 261The class ``py::args`` derives from ``py::tuple`` and ``py::kwargs`` derives |
255from ``py::dict``. Note that the ``kwargs`` argument is invalid if no keyword 256arguments were actually provided. Please refer to the other examples for 257details on how to iterate over these, and on how to cast their entries into 258C++ objects. A demonstration is also available in 259``tests/test_kwargs_and_defaults.cpp``. | 262from ``py::dict``. |
260 | 263 |
261.. warning:: | 264You may also use just one or the other, and may combine these with other 265arguments as long as the ``py::args`` and ``py::kwargs`` arguments are the last 266arguments accepted by the function. |
262 | 267 |
263 Unlike Python, pybind11 does not allow combining normal parameters with the 264 ``args`` / ``kwargs`` special parameters. | 268Please refer to the other examples for details on how to iterate over these, 269and on how to cast their entries into C++ objects. A demonstration is also 270available in ``tests/test_kwargs_and_defaults.cpp``. |
265 | 271 |
272.. note:: 273 274 When combining \*args or \*\*kwargs with :ref:`keyword_args` you should 275 *not* include ``py::arg`` tags for the ``py::args`` and ``py::kwargs`` 276 arguments. 277 |
|
266Default arguments revisited 267=========================== 268 269The section on :ref:`default_args` previously discussed basic usage of default 270arguments using pybind11. One noteworthy aspect of their implementation is that 271default arguments are converted to Python objects right at declaration time. 272Consider the following example: 273 --- 30 unchanged lines hidden (view full) --- 304Sometimes it may be necessary to pass a null pointer value as a default 305argument. In this case, remember to cast it to the underlying type in question, 306like so: 307 308.. code-block:: cpp 309 310 py::class_<MyClass>("MyClass") 311 .def("myFunction", py::arg("arg") = (SomeType *) nullptr); | 278Default arguments revisited 279=========================== 280 281The section on :ref:`default_args` previously discussed basic usage of default 282arguments using pybind11. One noteworthy aspect of their implementation is that 283default arguments are converted to Python objects right at declaration time. 284Consider the following example: 285 --- 30 unchanged lines hidden (view full) --- 316Sometimes it may be necessary to pass a null pointer value as a default 317argument. In this case, remember to cast it to the underlying type in question, 318like so: 319 320.. code-block:: cpp 321 322 py::class_<MyClass>("MyClass") 323 .def("myFunction", py::arg("arg") = (SomeType *) nullptr); |
324 325.. _nonconverting_arguments: 326 327Non-converting arguments 328======================== 329 330Certain argument types may support conversion from one type to another. Some 331examples of conversions are: 332 333* :ref:`implicit_conversions` declared using ``py::implicitly_convertible<A,B>()`` 334* Calling a method accepting a double with an integer argument 335* Calling a ``std::complex<float>`` argument with a non-complex python type 336 (for example, with a float). (Requires the optional ``pybind11/complex.h`` 337 header). 338* Calling a function taking an Eigen matrix reference with a numpy array of the 339 wrong type or of an incompatible data layout. (Requires the optional 340 ``pybind11/eigen.h`` header). 341 342This behaviour is sometimes undesirable: the binding code may prefer to raise 343an error rather than convert the argument. This behaviour can be obtained 344through ``py::arg`` by calling the ``.noconvert()`` method of the ``py::arg`` 345object, such as: 346 347.. code-block:: cpp 348 349 m.def("floats_only", [](double f) { return 0.5 * f; }, py::arg("f").noconvert()); 350 m.def("floats_preferred", [](double f) { return 0.5 * f; }, py::arg("f")); 351 352Attempting the call the second function (the one without ``.noconvert()``) with 353an integer will succeed, but attempting to call the ``.noconvert()`` version 354will fail with a ``TypeError``: 355 356.. code-block:: pycon 357 358 >>> floats_preferred(4) 359 2.0 360 >>> floats_only(4) 361 Traceback (most recent call last): 362 File "<stdin>", line 1, in <module> 363 TypeError: floats_only(): incompatible function arguments. The following argument types are supported: 364 1. (f: float) -> float 365 366 Invoked with: 4 367 368You may, of course, combine this with the :var:`_a` shorthand notation (see 369:ref:`keyword_args`) and/or :ref:`default_args`. It is also permitted to omit 370the argument name by using the ``py::arg()`` constructor without an argument 371name, i.e. by specifying ``py::arg().noconvert()``. 372 373.. note:: 374 375 When specifying ``py::arg`` options it is necessary to provide the same 376 number of options as the bound function has arguments. Thus if you want to 377 enable no-convert behaviour for just one of several arguments, you will 378 need to specify a ``py::arg()`` annotation for each argument with the 379 no-convert argument modified to ``py::arg().noconvert()``. 380 381Overload resolution order 382========================= 383 384When a function or method with multiple overloads is called from Python, 385pybind11 determines which overload to call in two passes. The first pass 386attempts to call each overload without allowing argument conversion (as if 387every argument had been specified as ``py::arg().noconvert()`` as decribed 388above). 389 390If no overload succeeds in the no-conversion first pass, a second pass is 391attempted in which argument conversion is allowed (except where prohibited via 392an explicit ``py::arg().noconvert()`` attribute in the function definition). 393 394If the second pass also fails a ``TypeError`` is raised. 395 396Within each pass, overloads are tried in the order they were registered with 397pybind11. 398 399What this means in practice is that pybind11 will prefer any overload that does 400not require conversion of arguments to an overload that does, but otherwise prefers 401earlier-defined overloads to later-defined ones. 402 403.. note:: 404 405 pybind11 does *not* further prioritize based on the number/pattern of 406 overloaded arguments. That is, pybind11 does not prioritize a function 407 requiring one conversion over one requiring three, but only prioritizes 408 overloads requiring no conversion at all to overloads that require 409 conversion of at least one argument. |
|