functions.rst (11986:c12e4625ab56) functions.rst (12037:d28054ac6ec9)
1Functions
2#########
3
4Before proceeding with this section, make sure that you are already familiar
5with the basics of binding functions and classes, as explained in :doc:`/basics`
6and :doc:`/classes`. The following guide is applicable to both free and member
7functions, i.e. *methods* in Python.
8
1Functions
2#########
3
4Before proceeding with this section, make sure that you are already familiar
5with the basics of binding functions and classes, as explained in :doc:`/basics`
6and :doc:`/classes`. The following guide is applicable to both free and member
7functions, i.e. *methods* in Python.
8
9.. _return_value_policies:
10
9Return value policies
10=====================
11
12Python and C++ use fundamentally different ways of managing the memory and
13lifetime of objects managed by them. This can lead to issues when creating
14bindings for functions that return a non-trivial type. Just by looking at the
15type information, it is not clear whether Python should take charge of the
16returned value and eventually free its resources, or if this is handled on the
11Return value policies
12=====================
13
14Python and C++ use fundamentally different ways of managing the memory and
15lifetime of objects managed by them. This can lead to issues when creating
16bindings for functions that return a non-trivial type. Just by looking at the
17type information, it is not clear whether Python should take charge of the
18returned value and eventually free its resources, or if this is handled on the
17C++ side. For this reason, pybind11 provides a several `return value policy`
19C++ side. For this reason, pybind11 provides a several *return value policy*
18annotations that can be passed to the :func:`module::def` and
19:func:`class_::def` functions. The default policy is
20:enum:`return_value_policy::automatic`.
21
22Return value policies are tricky, and it's very important to get them right.
23Just to illustrate what can go wrong, consider the following simple example:
24
25.. code-block:: cpp
26
20annotations that can be passed to the :func:`module::def` and
21:func:`class_::def` functions. The default policy is
22:enum:`return_value_policy::automatic`.
23
24Return value policies are tricky, and it's very important to get them right.
25Just to illustrate what can go wrong, consider the following simple example:
26
27.. code-block:: cpp
28
27 /* Function declaration */
29 /* Function declaration */
28 Data *get_data() { return _data; /* (pointer to a static data structure) */ }
29 ...
30
30 Data *get_data() { return _data; /* (pointer to a static data structure) */ }
31 ...
32
31 /* Binding code */
33 /* Binding code */
32 m.def("get_data", &get_data); // <-- KABOOM, will cause crash when called from Python
33
34What's going on here? When ``get_data()`` is called from Python, the return
35value (a native C++ type) must be wrapped to turn it into a usable Python type.
36In this case, the default return value policy (:enum:`return_value_policy::automatic`)
37causes pybind11 to assume ownership of the static ``_data`` instance.
38
39When Python's garbage collector eventually deletes the Python
40wrapper, pybind11 will also attempt to delete the C++ instance (via ``operator
41delete()``) due to the implied ownership. At this point, the entire application
42will come crashing down, though errors could also be more subtle and involve
43silent data corruption.
44
45In the above example, the policy :enum:`return_value_policy::reference` should have
46been specified so that the global data instance is only *referenced* without any
34 m.def("get_data", &get_data); // <-- KABOOM, will cause crash when called from Python
35
36What's going on here? When ``get_data()`` is called from Python, the return
37value (a native C++ type) must be wrapped to turn it into a usable Python type.
38In this case, the default return value policy (:enum:`return_value_policy::automatic`)
39causes pybind11 to assume ownership of the static ``_data`` instance.
40
41When Python's garbage collector eventually deletes the Python
42wrapper, pybind11 will also attempt to delete the C++ instance (via ``operator
43delete()``) due to the implied ownership. At this point, the entire application
44will come crashing down, though errors could also be more subtle and involve
45silent data corruption.
46
47In the above example, the policy :enum:`return_value_policy::reference` should have
48been specified so that the global data instance is only *referenced* without any
47implied transfer of ownership, i.e.:
49implied transfer of ownership, i.e.:
48
49.. code-block:: cpp
50
51 m.def("get_data", &get_data, return_value_policy::reference);
52
53On the other hand, this is not the right policy for many other situations,
54where ignoring ownership could lead to resource leaks.
55As a developer using pybind11, it's important to be familiar with the different

--- 27 unchanged lines hidden (view full) ---

83| | of a parent object, namely the implicit ``this``, or ``self`` argument of |
84| | the called method or property. Internally, this policy works just like |
85| | :enum:`return_value_policy::reference` but additionally applies a |
86| | ``keep_alive<0, 1>`` *call policy* (described in the next section) that |
87| | prevents the parent object from being garbage collected as long as the |
88| | return value is referenced by Python. This is the default policy for |
89| | property getters created via ``def_property``, ``def_readwrite``, etc. |
90+--------------------------------------------------+----------------------------------------------------------------------------+
50
51.. code-block:: cpp
52
53 m.def("get_data", &get_data, return_value_policy::reference);
54
55On the other hand, this is not the right policy for many other situations,
56where ignoring ownership could lead to resource leaks.
57As a developer using pybind11, it's important to be familiar with the different

--- 27 unchanged lines hidden (view full) ---

85| | of a parent object, namely the implicit ``this``, or ``self`` argument of |
86| | the called method or property. Internally, this policy works just like |
87| | :enum:`return_value_policy::reference` but additionally applies a |
88| | ``keep_alive<0, 1>`` *call policy* (described in the next section) that |
89| | prevents the parent object from being garbage collected as long as the |
90| | return value is referenced by Python. This is the default policy for |
91| | property getters created via ``def_property``, ``def_readwrite``, etc. |
92+--------------------------------------------------+----------------------------------------------------------------------------+
91| :enum:`return_value_policy::automatic` | This is the default return value policy, which falls back to the policy |
93| :enum:`return_value_policy::automatic` | **Default policy.** This policy falls back to the policy |
92| | :enum:`return_value_policy::take_ownership` when the return value is a |
94| | :enum:`return_value_policy::take_ownership` when the return value is a |
93| | pointer. Otherwise, it uses :enum:`return_value::move` or |
94| | :enum:`return_value::copy` for rvalue and lvalue references, respectively. |
95| | See above for a description of what all of these different policies do. |
95| | pointer. Otherwise, it uses :enum:`return_value_policy::move` or |
96| | :enum:`return_value_policy::copy` for rvalue and lvalue references, |
97| | respectively. See above for a description of what all of these different |
98| | policies do. |
96+--------------------------------------------------+----------------------------------------------------------------------------+
97| :enum:`return_value_policy::automatic_reference` | As above, but use policy :enum:`return_value_policy::reference` when the |
98| | return value is a pointer. This is the default conversion policy for |
99| | function arguments when calling Python functions manually from C++ code |
100| | (i.e. via handle::operator()). You probably won't need to use this. |
101+--------------------------------------------------+----------------------------------------------------------------------------+
102
103Return value policies can also be applied to properties:

--- 49 unchanged lines hidden (view full) ---

153 can lead to crashes or undefined behavior. For functions returning smart
154 pointers, it is not necessary to specify a return value policy.
155
156.. _call_policies:
157
158Additional call policies
159========================
160
99+--------------------------------------------------+----------------------------------------------------------------------------+
100| :enum:`return_value_policy::automatic_reference` | As above, but use policy :enum:`return_value_policy::reference` when the |
101| | return value is a pointer. This is the default conversion policy for |
102| | function arguments when calling Python functions manually from C++ code |
103| | (i.e. via handle::operator()). You probably won't need to use this. |
104+--------------------------------------------------+----------------------------------------------------------------------------+
105
106Return value policies can also be applied to properties:

--- 49 unchanged lines hidden (view full) ---

156 can lead to crashes or undefined behavior. For functions returning smart
157 pointers, it is not necessary to specify a return value policy.
158
159.. _call_policies:
160
161Additional call policies
162========================
163
161In addition to the above return value policies, further `call policies` can be
162specified to indicate dependencies between parameters. There is currently just
164In addition to the above return value policies, further *call policies* can be
165specified to indicate dependencies between parameters. In general, call policies
166are required when the C++ object is any kind of container and another object is being
167added to the container.
168
169There is currently just
163one policy named ``keep_alive<Nurse, Patient>``, which indicates that the
164argument with index ``Patient`` should be kept alive at least until the
165argument with index ``Nurse`` is freed by the garbage collector. Argument
166indices start at one, while zero refers to the return value. For methods, index
167``1`` refers to the implicit ``this`` pointer, while regular arguments begin at
168index ``2``. Arbitrarily many call policies can be specified. When a ``Nurse``
169with value ``None`` is detected at runtime, the call policy does nothing.
170

--- 31 unchanged lines hidden (view full) ---

202makes it possible to directly work with native Python types on the C++ side.
203For instance, the following statement iterates over a Python ``dict``:
204
205.. code-block:: cpp
206
207 void print_dict(py::dict dict) {
208 /* Easily interact with Python types */
209 for (auto item : dict)
170one policy named ``keep_alive<Nurse, Patient>``, which indicates that the
171argument with index ``Patient`` should be kept alive at least until the
172argument with index ``Nurse`` is freed by the garbage collector. Argument
173indices start at one, while zero refers to the return value. For methods, index
174``1`` refers to the implicit ``this`` pointer, while regular arguments begin at
175index ``2``. Arbitrarily many call policies can be specified. When a ``Nurse``
176with value ``None`` is detected at runtime, the call policy does nothing.
177

--- 31 unchanged lines hidden (view full) ---

209makes it possible to directly work with native Python types on the C++ side.
210For instance, the following statement iterates over a Python ``dict``:
211
212.. code-block:: cpp
213
214 void print_dict(py::dict dict) {
215 /* Easily interact with Python types */
216 for (auto item : dict)
210 std::cout << "key=" << item.first << ", "
211 << "value=" << item.second << std::endl;
217 std::cout << "key=" << std::string(py::str(item.first)) << ", "
218 << "value=" << std::string(py::str(item.second)) << std::endl;
212 }
213
214It can be exported:
215
216.. code-block:: cpp
217
218 m.def("print_dict", &print_dict);
219

--- 27 unchanged lines hidden (view full) ---

247 if (kwargs)
248 /// .. do something with kwargs
249 }
250
251 /// Binding code
252 m.def("generic", &generic);
253
254The class ``py::args`` derives from ``py::tuple`` and ``py::kwargs`` derives
219 }
220
221It can be exported:
222
223.. code-block:: cpp
224
225 m.def("print_dict", &print_dict);
226

--- 27 unchanged lines hidden (view full) ---

254 if (kwargs)
255 /// .. do something with kwargs
256 }
257
258 /// Binding code
259 m.def("generic", &generic);
260
261The class ``py::args`` derives from ``py::tuple`` and ``py::kwargs`` derives
255from ``py::dict``. Note that the ``kwargs`` argument is invalid if no keyword
256arguments were actually provided. Please refer to the other examples for
257details on how to iterate over these, and on how to cast their entries into
258C++ objects. A demonstration is also available in
259``tests/test_kwargs_and_defaults.cpp``.
262from ``py::dict``.
260
263
261.. warning::
264You may also use just one or the other, and may combine these with other
265arguments as long as the ``py::args`` and ``py::kwargs`` arguments are the last
266arguments accepted by the function.
262
267
263 Unlike Python, pybind11 does not allow combining normal parameters with the
264 ``args`` / ``kwargs`` special parameters.
268Please refer to the other examples for details on how to iterate over these,
269and on how to cast their entries into C++ objects. A demonstration is also
270available in ``tests/test_kwargs_and_defaults.cpp``.
265
271
272.. note::
273
274 When combining \*args or \*\*kwargs with :ref:`keyword_args` you should
275 *not* include ``py::arg`` tags for the ``py::args`` and ``py::kwargs``
276 arguments.
277
266Default arguments revisited
267===========================
268
269The section on :ref:`default_args` previously discussed basic usage of default
270arguments using pybind11. One noteworthy aspect of their implementation is that
271default arguments are converted to Python objects right at declaration time.
272Consider the following example:
273

--- 30 unchanged lines hidden (view full) ---

304Sometimes it may be necessary to pass a null pointer value as a default
305argument. In this case, remember to cast it to the underlying type in question,
306like so:
307
308.. code-block:: cpp
309
310 py::class_<MyClass>("MyClass")
311 .def("myFunction", py::arg("arg") = (SomeType *) nullptr);
278Default arguments revisited
279===========================
280
281The section on :ref:`default_args` previously discussed basic usage of default
282arguments using pybind11. One noteworthy aspect of their implementation is that
283default arguments are converted to Python objects right at declaration time.
284Consider the following example:
285

--- 30 unchanged lines hidden (view full) ---

316Sometimes it may be necessary to pass a null pointer value as a default
317argument. In this case, remember to cast it to the underlying type in question,
318like so:
319
320.. code-block:: cpp
321
322 py::class_<MyClass>("MyClass")
323 .def("myFunction", py::arg("arg") = (SomeType *) nullptr);
324
325.. _nonconverting_arguments:
326
327Non-converting arguments
328========================
329
330Certain argument types may support conversion from one type to another. Some
331examples of conversions are:
332
333* :ref:`implicit_conversions` declared using ``py::implicitly_convertible<A,B>()``
334* Calling a method accepting a double with an integer argument
335* Calling a ``std::complex<float>`` argument with a non-complex python type
336 (for example, with a float). (Requires the optional ``pybind11/complex.h``
337 header).
338* Calling a function taking an Eigen matrix reference with a numpy array of the
339 wrong type or of an incompatible data layout. (Requires the optional
340 ``pybind11/eigen.h`` header).
341
342This behaviour is sometimes undesirable: the binding code may prefer to raise
343an error rather than convert the argument. This behaviour can be obtained
344through ``py::arg`` by calling the ``.noconvert()`` method of the ``py::arg``
345object, such as:
346
347.. code-block:: cpp
348
349 m.def("floats_only", [](double f) { return 0.5 * f; }, py::arg("f").noconvert());
350 m.def("floats_preferred", [](double f) { return 0.5 * f; }, py::arg("f"));
351
352Attempting the call the second function (the one without ``.noconvert()``) with
353an integer will succeed, but attempting to call the ``.noconvert()`` version
354will fail with a ``TypeError``:
355
356.. code-block:: pycon
357
358 >>> floats_preferred(4)
359 2.0
360 >>> floats_only(4)
361 Traceback (most recent call last):
362 File "<stdin>", line 1, in <module>
363 TypeError: floats_only(): incompatible function arguments. The following argument types are supported:
364 1. (f: float) -> float
365
366 Invoked with: 4
367
368You may, of course, combine this with the :var:`_a` shorthand notation (see
369:ref:`keyword_args`) and/or :ref:`default_args`. It is also permitted to omit
370the argument name by using the ``py::arg()`` constructor without an argument
371name, i.e. by specifying ``py::arg().noconvert()``.
372
373.. note::
374
375 When specifying ``py::arg`` options it is necessary to provide the same
376 number of options as the bound function has arguments. Thus if you want to
377 enable no-convert behaviour for just one of several arguments, you will
378 need to specify a ``py::arg()`` annotation for each argument with the
379 no-convert argument modified to ``py::arg().noconvert()``.
380
381Overload resolution order
382=========================
383
384When a function or method with multiple overloads is called from Python,
385pybind11 determines which overload to call in two passes. The first pass
386attempts to call each overload without allowing argument conversion (as if
387every argument had been specified as ``py::arg().noconvert()`` as decribed
388above).
389
390If no overload succeeds in the no-conversion first pass, a second pass is
391attempted in which argument conversion is allowed (except where prohibited via
392an explicit ``py::arg().noconvert()`` attribute in the function definition).
393
394If the second pass also fails a ``TypeError`` is raised.
395
396Within each pass, overloads are tried in the order they were registered with
397pybind11.
398
399What this means in practice is that pybind11 will prefer any overload that does
400not require conversion of arguments to an overload that does, but otherwise prefers
401earlier-defined overloads to later-defined ones.
402
403.. note::
404
405 pybind11 does *not* further prioritize based on the number/pattern of
406 overloaded arguments. That is, pybind11 does not prioritize a function
407 requiring one conversion over one requiring three, but only prioritizes
408 overloads requiring no conversion at all to overloads that require
409 conversion of at least one argument.