111986Sandreas.sandberg@arm.comFunctions 211986Sandreas.sandberg@arm.com######### 311986Sandreas.sandberg@arm.com 411986Sandreas.sandberg@arm.comBefore proceeding with this section, make sure that you are already familiar 511986Sandreas.sandberg@arm.comwith the basics of binding functions and classes, as explained in :doc:`/basics` 611986Sandreas.sandberg@arm.comand :doc:`/classes`. The following guide is applicable to both free and member 711986Sandreas.sandberg@arm.comfunctions, i.e. *methods* in Python. 811986Sandreas.sandberg@arm.com 912037Sandreas.sandberg@arm.com.. _return_value_policies: 1012037Sandreas.sandberg@arm.com 1111986Sandreas.sandberg@arm.comReturn value policies 1211986Sandreas.sandberg@arm.com===================== 1311986Sandreas.sandberg@arm.com 1411986Sandreas.sandberg@arm.comPython and C++ use fundamentally different ways of managing the memory and 1511986Sandreas.sandberg@arm.comlifetime of objects managed by them. This can lead to issues when creating 1611986Sandreas.sandberg@arm.combindings for functions that return a non-trivial type. Just by looking at the 1711986Sandreas.sandberg@arm.comtype information, it is not clear whether Python should take charge of the 1811986Sandreas.sandberg@arm.comreturned value and eventually free its resources, or if this is handled on the 1912037Sandreas.sandberg@arm.comC++ side. For this reason, pybind11 provides a several *return value policy* 2011986Sandreas.sandberg@arm.comannotations that can be passed to the :func:`module::def` and 2111986Sandreas.sandberg@arm.com:func:`class_::def` functions. The default policy is 2211986Sandreas.sandberg@arm.com:enum:`return_value_policy::automatic`. 2311986Sandreas.sandberg@arm.com 2411986Sandreas.sandberg@arm.comReturn value policies are tricky, and it's very important to get them right. 2511986Sandreas.sandberg@arm.comJust to illustrate what can go wrong, consider the following simple example: 2611986Sandreas.sandberg@arm.com 2711986Sandreas.sandberg@arm.com.. code-block:: cpp 2811986Sandreas.sandberg@arm.com 2912037Sandreas.sandberg@arm.com /* Function declaration */ 3011986Sandreas.sandberg@arm.com Data *get_data() { return _data; /* (pointer to a static data structure) */ } 3111986Sandreas.sandberg@arm.com ... 3211986Sandreas.sandberg@arm.com 3312037Sandreas.sandberg@arm.com /* Binding code */ 3411986Sandreas.sandberg@arm.com m.def("get_data", &get_data); // <-- KABOOM, will cause crash when called from Python 3511986Sandreas.sandberg@arm.com 3611986Sandreas.sandberg@arm.comWhat's going on here? When ``get_data()`` is called from Python, the return 3711986Sandreas.sandberg@arm.comvalue (a native C++ type) must be wrapped to turn it into a usable Python type. 3811986Sandreas.sandberg@arm.comIn this case, the default return value policy (:enum:`return_value_policy::automatic`) 3911986Sandreas.sandberg@arm.comcauses pybind11 to assume ownership of the static ``_data`` instance. 4011986Sandreas.sandberg@arm.com 4111986Sandreas.sandberg@arm.comWhen Python's garbage collector eventually deletes the Python 4211986Sandreas.sandberg@arm.comwrapper, pybind11 will also attempt to delete the C++ instance (via ``operator 4311986Sandreas.sandberg@arm.comdelete()``) due to the implied ownership. At this point, the entire application 4411986Sandreas.sandberg@arm.comwill come crashing down, though errors could also be more subtle and involve 4511986Sandreas.sandberg@arm.comsilent data corruption. 4611986Sandreas.sandberg@arm.com 4711986Sandreas.sandberg@arm.comIn the above example, the policy :enum:`return_value_policy::reference` should have 4811986Sandreas.sandberg@arm.combeen specified so that the global data instance is only *referenced* without any 4912037Sandreas.sandberg@arm.comimplied transfer of ownership, i.e.: 5011986Sandreas.sandberg@arm.com 5111986Sandreas.sandberg@arm.com.. code-block:: cpp 5211986Sandreas.sandberg@arm.com 5311986Sandreas.sandberg@arm.com m.def("get_data", &get_data, return_value_policy::reference); 5411986Sandreas.sandberg@arm.com 5511986Sandreas.sandberg@arm.comOn the other hand, this is not the right policy for many other situations, 5611986Sandreas.sandberg@arm.comwhere ignoring ownership could lead to resource leaks. 5711986Sandreas.sandberg@arm.comAs a developer using pybind11, it's important to be familiar with the different 5811986Sandreas.sandberg@arm.comreturn value policies, including which situation calls for which one of them. 5911986Sandreas.sandberg@arm.comThe following table provides an overview of available policies: 6011986Sandreas.sandberg@arm.com 6111986Sandreas.sandberg@arm.com.. tabularcolumns:: |p{0.5\textwidth}|p{0.45\textwidth}| 6211986Sandreas.sandberg@arm.com 6311986Sandreas.sandberg@arm.com+--------------------------------------------------+----------------------------------------------------------------------------+ 6411986Sandreas.sandberg@arm.com| Return value policy | Description | 6511986Sandreas.sandberg@arm.com+==================================================+============================================================================+ 6611986Sandreas.sandberg@arm.com| :enum:`return_value_policy::take_ownership` | Reference an existing object (i.e. do not create a new copy) and take | 6711986Sandreas.sandberg@arm.com| | ownership. Python will call the destructor and delete operator when the | 6811986Sandreas.sandberg@arm.com| | object's reference count reaches zero. Undefined behavior ensues when the | 6911986Sandreas.sandberg@arm.com| | C++ side does the same, or when the data was not dynamically allocated. | 7011986Sandreas.sandberg@arm.com+--------------------------------------------------+----------------------------------------------------------------------------+ 7111986Sandreas.sandberg@arm.com| :enum:`return_value_policy::copy` | Create a new copy of the returned object, which will be owned by Python. | 7211986Sandreas.sandberg@arm.com| | This policy is comparably safe because the lifetimes of the two instances | 7311986Sandreas.sandberg@arm.com| | are decoupled. | 7411986Sandreas.sandberg@arm.com+--------------------------------------------------+----------------------------------------------------------------------------+ 7511986Sandreas.sandberg@arm.com| :enum:`return_value_policy::move` | Use ``std::move`` to move the return value contents into a new instance | 7611986Sandreas.sandberg@arm.com| | that will be owned by Python. This policy is comparably safe because the | 7711986Sandreas.sandberg@arm.com| | lifetimes of the two instances (move source and destination) are decoupled.| 7811986Sandreas.sandberg@arm.com+--------------------------------------------------+----------------------------------------------------------------------------+ 7911986Sandreas.sandberg@arm.com| :enum:`return_value_policy::reference` | Reference an existing object, but do not take ownership. The C++ side is | 8011986Sandreas.sandberg@arm.com| | responsible for managing the object's lifetime and deallocating it when | 8111986Sandreas.sandberg@arm.com| | it is no longer used. Warning: undefined behavior will ensue when the C++ | 8211986Sandreas.sandberg@arm.com| | side deletes an object that is still referenced and used by Python. | 8311986Sandreas.sandberg@arm.com+--------------------------------------------------+----------------------------------------------------------------------------+ 8411986Sandreas.sandberg@arm.com| :enum:`return_value_policy::reference_internal` | Indicates that the lifetime of the return value is tied to the lifetime | 8511986Sandreas.sandberg@arm.com| | of a parent object, namely the implicit ``this``, or ``self`` argument of | 8611986Sandreas.sandberg@arm.com| | the called method or property. Internally, this policy works just like | 8711986Sandreas.sandberg@arm.com| | :enum:`return_value_policy::reference` but additionally applies a | 8811986Sandreas.sandberg@arm.com| | ``keep_alive<0, 1>`` *call policy* (described in the next section) that | 8911986Sandreas.sandberg@arm.com| | prevents the parent object from being garbage collected as long as the | 9011986Sandreas.sandberg@arm.com| | return value is referenced by Python. This is the default policy for | 9111986Sandreas.sandberg@arm.com| | property getters created via ``def_property``, ``def_readwrite``, etc. | 9211986Sandreas.sandberg@arm.com+--------------------------------------------------+----------------------------------------------------------------------------+ 9312037Sandreas.sandberg@arm.com| :enum:`return_value_policy::automatic` | **Default policy.** This policy falls back to the policy | 9411986Sandreas.sandberg@arm.com| | :enum:`return_value_policy::take_ownership` when the return value is a | 9512037Sandreas.sandberg@arm.com| | pointer. Otherwise, it uses :enum:`return_value_policy::move` or | 9612037Sandreas.sandberg@arm.com| | :enum:`return_value_policy::copy` for rvalue and lvalue references, | 9712037Sandreas.sandberg@arm.com| | respectively. See above for a description of what all of these different | 9812037Sandreas.sandberg@arm.com| | policies do. | 9911986Sandreas.sandberg@arm.com+--------------------------------------------------+----------------------------------------------------------------------------+ 10011986Sandreas.sandberg@arm.com| :enum:`return_value_policy::automatic_reference` | As above, but use policy :enum:`return_value_policy::reference` when the | 10111986Sandreas.sandberg@arm.com| | return value is a pointer. This is the default conversion policy for | 10211986Sandreas.sandberg@arm.com| | function arguments when calling Python functions manually from C++ code | 10311986Sandreas.sandberg@arm.com| | (i.e. via handle::operator()). You probably won't need to use this. | 10411986Sandreas.sandberg@arm.com+--------------------------------------------------+----------------------------------------------------------------------------+ 10511986Sandreas.sandberg@arm.com 10611986Sandreas.sandberg@arm.comReturn value policies can also be applied to properties: 10711986Sandreas.sandberg@arm.com 10811986Sandreas.sandberg@arm.com.. code-block:: cpp 10911986Sandreas.sandberg@arm.com 11011986Sandreas.sandberg@arm.com class_<MyClass>(m, "MyClass") 11111986Sandreas.sandberg@arm.com .def_property("data", &MyClass::getData, &MyClass::setData, 11211986Sandreas.sandberg@arm.com py::return_value_policy::copy); 11311986Sandreas.sandberg@arm.com 11411986Sandreas.sandberg@arm.comTechnically, the code above applies the policy to both the getter and the 11511986Sandreas.sandberg@arm.comsetter function, however, the setter doesn't really care about *return* 11611986Sandreas.sandberg@arm.comvalue policies which makes this a convenient terse syntax. Alternatively, 11711986Sandreas.sandberg@arm.comtargeted arguments can be passed through the :class:`cpp_function` constructor: 11811986Sandreas.sandberg@arm.com 11911986Sandreas.sandberg@arm.com.. code-block:: cpp 12011986Sandreas.sandberg@arm.com 12111986Sandreas.sandberg@arm.com class_<MyClass>(m, "MyClass") 12211986Sandreas.sandberg@arm.com .def_property("data" 12311986Sandreas.sandberg@arm.com py::cpp_function(&MyClass::getData, py::return_value_policy::copy), 12411986Sandreas.sandberg@arm.com py::cpp_function(&MyClass::setData) 12511986Sandreas.sandberg@arm.com ); 12611986Sandreas.sandberg@arm.com 12711986Sandreas.sandberg@arm.com.. warning:: 12811986Sandreas.sandberg@arm.com 12914299Sbbruce@ucdavis.edu Code with invalid return value policies might access uninitialized memory or 13011986Sandreas.sandberg@arm.com free data structures multiple times, which can lead to hard-to-debug 13111986Sandreas.sandberg@arm.com non-determinism and segmentation faults, hence it is worth spending the 13211986Sandreas.sandberg@arm.com time to understand all the different options in the table above. 13311986Sandreas.sandberg@arm.com 13411986Sandreas.sandberg@arm.com.. note:: 13511986Sandreas.sandberg@arm.com 13611986Sandreas.sandberg@arm.com One important aspect of the above policies is that they only apply to 13711986Sandreas.sandberg@arm.com instances which pybind11 has *not* seen before, in which case the policy 13811986Sandreas.sandberg@arm.com clarifies essential questions about the return value's lifetime and 13911986Sandreas.sandberg@arm.com ownership. When pybind11 knows the instance already (as identified by its 14011986Sandreas.sandberg@arm.com type and address in memory), it will return the existing Python object 14111986Sandreas.sandberg@arm.com wrapper rather than creating a new copy. 14211986Sandreas.sandberg@arm.com 14311986Sandreas.sandberg@arm.com.. note:: 14411986Sandreas.sandberg@arm.com 14511986Sandreas.sandberg@arm.com The next section on :ref:`call_policies` discusses *call policies* that can be 14611986Sandreas.sandberg@arm.com specified *in addition* to a return value policy from the list above. Call 14711986Sandreas.sandberg@arm.com policies indicate reference relationships that can involve both return values 14811986Sandreas.sandberg@arm.com and parameters of functions. 14911986Sandreas.sandberg@arm.com 15011986Sandreas.sandberg@arm.com.. note:: 15111986Sandreas.sandberg@arm.com 15211986Sandreas.sandberg@arm.com As an alternative to elaborate call policies and lifetime management logic, 15311986Sandreas.sandberg@arm.com consider using smart pointers (see the section on :ref:`smart_pointers` for 15411986Sandreas.sandberg@arm.com details). Smart pointers can tell whether an object is still referenced from 15511986Sandreas.sandberg@arm.com C++ or Python, which generally eliminates the kinds of inconsistencies that 15611986Sandreas.sandberg@arm.com can lead to crashes or undefined behavior. For functions returning smart 15711986Sandreas.sandberg@arm.com pointers, it is not necessary to specify a return value policy. 15811986Sandreas.sandberg@arm.com 15911986Sandreas.sandberg@arm.com.. _call_policies: 16011986Sandreas.sandberg@arm.com 16111986Sandreas.sandberg@arm.comAdditional call policies 16211986Sandreas.sandberg@arm.com======================== 16311986Sandreas.sandberg@arm.com 16412037Sandreas.sandberg@arm.comIn addition to the above return value policies, further *call policies* can be 16512391Sjason@lowepower.comspecified to indicate dependencies between parameters or ensure a certain state 16612391Sjason@lowepower.comfor the function call. 16712037Sandreas.sandberg@arm.com 16812391Sjason@lowepower.comKeep alive 16912391Sjason@lowepower.com---------- 17012391Sjason@lowepower.com 17112391Sjason@lowepower.comIn general, this policy is required when the C++ object is any kind of container 17212391Sjason@lowepower.comand another object is being added to the container. ``keep_alive<Nurse, Patient>`` 17312391Sjason@lowepower.comindicates that the argument with index ``Patient`` should be kept alive at least 17412391Sjason@lowepower.comuntil the argument with index ``Nurse`` is freed by the garbage collector. Argument 17511986Sandreas.sandberg@arm.comindices start at one, while zero refers to the return value. For methods, index 17611986Sandreas.sandberg@arm.com``1`` refers to the implicit ``this`` pointer, while regular arguments begin at 17711986Sandreas.sandberg@arm.comindex ``2``. Arbitrarily many call policies can be specified. When a ``Nurse`` 17811986Sandreas.sandberg@arm.comwith value ``None`` is detected at runtime, the call policy does nothing. 17911986Sandreas.sandberg@arm.com 18012391Sjason@lowepower.comWhen the nurse is not a pybind11-registered type, the implementation internally 18112391Sjason@lowepower.comrelies on the ability to create a *weak reference* to the nurse object. When 18212391Sjason@lowepower.comthe nurse object is not a pybind11-registered type and does not support weak 18312391Sjason@lowepower.comreferences, an exception will be thrown. 18411986Sandreas.sandberg@arm.com 18511986Sandreas.sandberg@arm.comConsider the following example: here, the binding code for a list append 18611986Sandreas.sandberg@arm.comoperation ties the lifetime of the newly added element to the underlying 18711986Sandreas.sandberg@arm.comcontainer: 18811986Sandreas.sandberg@arm.com 18911986Sandreas.sandberg@arm.com.. code-block:: cpp 19011986Sandreas.sandberg@arm.com 19111986Sandreas.sandberg@arm.com py::class_<List>(m, "List") 19211986Sandreas.sandberg@arm.com .def("append", &List::append, py::keep_alive<1, 2>()); 19311986Sandreas.sandberg@arm.com 19412391Sjason@lowepower.comFor consistency, the argument indexing is identical for constructors. Index 19512391Sjason@lowepower.com``1`` still refers to the implicit ``this`` pointer, i.e. the object which is 19612391Sjason@lowepower.combeing constructed. Index ``0`` refers to the return type which is presumed to 19712391Sjason@lowepower.combe ``void`` when a constructor is viewed like a function. The following example 19812391Sjason@lowepower.comties the lifetime of the constructor element to the constructed object: 19912391Sjason@lowepower.com 20012391Sjason@lowepower.com.. code-block:: cpp 20112391Sjason@lowepower.com 20212391Sjason@lowepower.com py::class_<Nurse>(m, "Nurse") 20312391Sjason@lowepower.com .def(py::init<Patient &>(), py::keep_alive<1, 2>()); 20412391Sjason@lowepower.com 20511986Sandreas.sandberg@arm.com.. note:: 20611986Sandreas.sandberg@arm.com 20711986Sandreas.sandberg@arm.com ``keep_alive`` is analogous to the ``with_custodian_and_ward`` (if Nurse, 20811986Sandreas.sandberg@arm.com Patient != 0) and ``with_custodian_and_ward_postcall`` (if Nurse/Patient == 20911986Sandreas.sandberg@arm.com 0) policies from Boost.Python. 21011986Sandreas.sandberg@arm.com 21112391Sjason@lowepower.comCall guard 21212391Sjason@lowepower.com---------- 21312391Sjason@lowepower.com 21412391Sjason@lowepower.comThe ``call_guard<T>`` policy allows any scope guard type ``T`` to be placed 21512391Sjason@lowepower.comaround the function call. For example, this definition: 21612391Sjason@lowepower.com 21712391Sjason@lowepower.com.. code-block:: cpp 21812391Sjason@lowepower.com 21912391Sjason@lowepower.com m.def("foo", foo, py::call_guard<T>()); 22012391Sjason@lowepower.com 22112391Sjason@lowepower.comis equivalent to the following pseudocode: 22212391Sjason@lowepower.com 22312391Sjason@lowepower.com.. code-block:: cpp 22412391Sjason@lowepower.com 22512391Sjason@lowepower.com m.def("foo", [](args...) { 22612391Sjason@lowepower.com T scope_guard; 22712391Sjason@lowepower.com return foo(args...); // forwarded arguments 22812391Sjason@lowepower.com }); 22912391Sjason@lowepower.com 23012391Sjason@lowepower.comThe only requirement is that ``T`` is default-constructible, but otherwise any 23112391Sjason@lowepower.comscope guard will work. This is very useful in combination with `gil_scoped_release`. 23212391Sjason@lowepower.comSee :ref:`gil`. 23312391Sjason@lowepower.com 23412391Sjason@lowepower.comMultiple guards can also be specified as ``py::call_guard<T1, T2, T3...>``. The 23512391Sjason@lowepower.comconstructor order is left to right and destruction happens in reverse. 23612391Sjason@lowepower.com 23711986Sandreas.sandberg@arm.com.. seealso:: 23811986Sandreas.sandberg@arm.com 23912391Sjason@lowepower.com The file :file:`tests/test_call_policies.cpp` contains a complete example 24012391Sjason@lowepower.com that demonstrates using `keep_alive` and `call_guard` in more detail. 24111986Sandreas.sandberg@arm.com 24211986Sandreas.sandberg@arm.com.. _python_objects_as_args: 24311986Sandreas.sandberg@arm.com 24411986Sandreas.sandberg@arm.comPython objects as arguments 24511986Sandreas.sandberg@arm.com=========================== 24611986Sandreas.sandberg@arm.com 24711986Sandreas.sandberg@arm.compybind11 exposes all major Python types using thin C++ wrapper classes. These 24811986Sandreas.sandberg@arm.comwrapper classes can also be used as parameters of functions in bindings, which 24911986Sandreas.sandberg@arm.commakes it possible to directly work with native Python types on the C++ side. 25011986Sandreas.sandberg@arm.comFor instance, the following statement iterates over a Python ``dict``: 25111986Sandreas.sandberg@arm.com 25211986Sandreas.sandberg@arm.com.. code-block:: cpp 25311986Sandreas.sandberg@arm.com 25411986Sandreas.sandberg@arm.com void print_dict(py::dict dict) { 25511986Sandreas.sandberg@arm.com /* Easily interact with Python types */ 25611986Sandreas.sandberg@arm.com for (auto item : dict) 25712037Sandreas.sandberg@arm.com std::cout << "key=" << std::string(py::str(item.first)) << ", " 25812037Sandreas.sandberg@arm.com << "value=" << std::string(py::str(item.second)) << std::endl; 25911986Sandreas.sandberg@arm.com } 26011986Sandreas.sandberg@arm.com 26111986Sandreas.sandberg@arm.comIt can be exported: 26211986Sandreas.sandberg@arm.com 26311986Sandreas.sandberg@arm.com.. code-block:: cpp 26411986Sandreas.sandberg@arm.com 26511986Sandreas.sandberg@arm.com m.def("print_dict", &print_dict); 26611986Sandreas.sandberg@arm.com 26711986Sandreas.sandberg@arm.comAnd used in Python as usual: 26811986Sandreas.sandberg@arm.com 26911986Sandreas.sandberg@arm.com.. code-block:: pycon 27011986Sandreas.sandberg@arm.com 27111986Sandreas.sandberg@arm.com >>> print_dict({'foo': 123, 'bar': 'hello'}) 27211986Sandreas.sandberg@arm.com key=foo, value=123 27311986Sandreas.sandberg@arm.com key=bar, value=hello 27411986Sandreas.sandberg@arm.com 27511986Sandreas.sandberg@arm.comFor more information on using Python objects in C++, see :doc:`/advanced/pycpp/index`. 27611986Sandreas.sandberg@arm.com 27711986Sandreas.sandberg@arm.comAccepting \*args and \*\*kwargs 27811986Sandreas.sandberg@arm.com=============================== 27911986Sandreas.sandberg@arm.com 28011986Sandreas.sandberg@arm.comPython provides a useful mechanism to define functions that accept arbitrary 28111986Sandreas.sandberg@arm.comnumbers of arguments and keyword arguments: 28211986Sandreas.sandberg@arm.com 28311986Sandreas.sandberg@arm.com.. code-block:: python 28411986Sandreas.sandberg@arm.com 28511986Sandreas.sandberg@arm.com def generic(*args, **kwargs): 28611986Sandreas.sandberg@arm.com ... # do something with args and kwargs 28711986Sandreas.sandberg@arm.com 28811986Sandreas.sandberg@arm.comSuch functions can also be created using pybind11: 28911986Sandreas.sandberg@arm.com 29011986Sandreas.sandberg@arm.com.. code-block:: cpp 29111986Sandreas.sandberg@arm.com 29211986Sandreas.sandberg@arm.com void generic(py::args args, py::kwargs kwargs) { 29311986Sandreas.sandberg@arm.com /// .. do something with args 29411986Sandreas.sandberg@arm.com if (kwargs) 29511986Sandreas.sandberg@arm.com /// .. do something with kwargs 29611986Sandreas.sandberg@arm.com } 29711986Sandreas.sandberg@arm.com 29811986Sandreas.sandberg@arm.com /// Binding code 29911986Sandreas.sandberg@arm.com m.def("generic", &generic); 30011986Sandreas.sandberg@arm.com 30111986Sandreas.sandberg@arm.comThe class ``py::args`` derives from ``py::tuple`` and ``py::kwargs`` derives 30212037Sandreas.sandberg@arm.comfrom ``py::dict``. 30311986Sandreas.sandberg@arm.com 30412037Sandreas.sandberg@arm.comYou may also use just one or the other, and may combine these with other 30512037Sandreas.sandberg@arm.comarguments as long as the ``py::args`` and ``py::kwargs`` arguments are the last 30612037Sandreas.sandberg@arm.comarguments accepted by the function. 30711986Sandreas.sandberg@arm.com 30812037Sandreas.sandberg@arm.comPlease refer to the other examples for details on how to iterate over these, 30912037Sandreas.sandberg@arm.comand on how to cast their entries into C++ objects. A demonstration is also 31012037Sandreas.sandberg@arm.comavailable in ``tests/test_kwargs_and_defaults.cpp``. 31112037Sandreas.sandberg@arm.com 31212037Sandreas.sandberg@arm.com.. note:: 31312037Sandreas.sandberg@arm.com 31412037Sandreas.sandberg@arm.com When combining \*args or \*\*kwargs with :ref:`keyword_args` you should 31512037Sandreas.sandberg@arm.com *not* include ``py::arg`` tags for the ``py::args`` and ``py::kwargs`` 31612037Sandreas.sandberg@arm.com arguments. 31711986Sandreas.sandberg@arm.com 31811986Sandreas.sandberg@arm.comDefault arguments revisited 31911986Sandreas.sandberg@arm.com=========================== 32011986Sandreas.sandberg@arm.com 32111986Sandreas.sandberg@arm.comThe section on :ref:`default_args` previously discussed basic usage of default 32211986Sandreas.sandberg@arm.comarguments using pybind11. One noteworthy aspect of their implementation is that 32311986Sandreas.sandberg@arm.comdefault arguments are converted to Python objects right at declaration time. 32411986Sandreas.sandberg@arm.comConsider the following example: 32511986Sandreas.sandberg@arm.com 32611986Sandreas.sandberg@arm.com.. code-block:: cpp 32711986Sandreas.sandberg@arm.com 32811986Sandreas.sandberg@arm.com py::class_<MyClass>("MyClass") 32911986Sandreas.sandberg@arm.com .def("myFunction", py::arg("arg") = SomeType(123)); 33011986Sandreas.sandberg@arm.com 33111986Sandreas.sandberg@arm.comIn this case, pybind11 must already be set up to deal with values of the type 33211986Sandreas.sandberg@arm.com``SomeType`` (via a prior instantiation of ``py::class_<SomeType>``), or an 33311986Sandreas.sandberg@arm.comexception will be thrown. 33411986Sandreas.sandberg@arm.com 33511986Sandreas.sandberg@arm.comAnother aspect worth highlighting is that the "preview" of the default argument 33611986Sandreas.sandberg@arm.comin the function signature is generated using the object's ``__repr__`` method. 33711986Sandreas.sandberg@arm.comIf not available, the signature may not be very helpful, e.g.: 33811986Sandreas.sandberg@arm.com 33911986Sandreas.sandberg@arm.com.. code-block:: pycon 34011986Sandreas.sandberg@arm.com 34111986Sandreas.sandberg@arm.com FUNCTIONS 34211986Sandreas.sandberg@arm.com ... 34311986Sandreas.sandberg@arm.com | myFunction(...) 34411986Sandreas.sandberg@arm.com | Signature : (MyClass, arg : SomeType = <SomeType object at 0x101b7b080>) -> NoneType 34511986Sandreas.sandberg@arm.com ... 34611986Sandreas.sandberg@arm.com 34711986Sandreas.sandberg@arm.comThe first way of addressing this is by defining ``SomeType.__repr__``. 34811986Sandreas.sandberg@arm.comAlternatively, it is possible to specify the human-readable preview of the 34911986Sandreas.sandberg@arm.comdefault argument manually using the ``arg_v`` notation: 35011986Sandreas.sandberg@arm.com 35111986Sandreas.sandberg@arm.com.. code-block:: cpp 35211986Sandreas.sandberg@arm.com 35311986Sandreas.sandberg@arm.com py::class_<MyClass>("MyClass") 35411986Sandreas.sandberg@arm.com .def("myFunction", py::arg_v("arg", SomeType(123), "SomeType(123)")); 35511986Sandreas.sandberg@arm.com 35611986Sandreas.sandberg@arm.comSometimes it may be necessary to pass a null pointer value as a default 35711986Sandreas.sandberg@arm.comargument. In this case, remember to cast it to the underlying type in question, 35811986Sandreas.sandberg@arm.comlike so: 35911986Sandreas.sandberg@arm.com 36011986Sandreas.sandberg@arm.com.. code-block:: cpp 36111986Sandreas.sandberg@arm.com 36211986Sandreas.sandberg@arm.com py::class_<MyClass>("MyClass") 36311986Sandreas.sandberg@arm.com .def("myFunction", py::arg("arg") = (SomeType *) nullptr); 36412037Sandreas.sandberg@arm.com 36512037Sandreas.sandberg@arm.com.. _nonconverting_arguments: 36612037Sandreas.sandberg@arm.com 36712037Sandreas.sandberg@arm.comNon-converting arguments 36812037Sandreas.sandberg@arm.com======================== 36912037Sandreas.sandberg@arm.com 37012037Sandreas.sandberg@arm.comCertain argument types may support conversion from one type to another. Some 37112037Sandreas.sandberg@arm.comexamples of conversions are: 37212037Sandreas.sandberg@arm.com 37312037Sandreas.sandberg@arm.com* :ref:`implicit_conversions` declared using ``py::implicitly_convertible<A,B>()`` 37412037Sandreas.sandberg@arm.com* Calling a method accepting a double with an integer argument 37512037Sandreas.sandberg@arm.com* Calling a ``std::complex<float>`` argument with a non-complex python type 37612037Sandreas.sandberg@arm.com (for example, with a float). (Requires the optional ``pybind11/complex.h`` 37712037Sandreas.sandberg@arm.com header). 37812037Sandreas.sandberg@arm.com* Calling a function taking an Eigen matrix reference with a numpy array of the 37912037Sandreas.sandberg@arm.com wrong type or of an incompatible data layout. (Requires the optional 38012037Sandreas.sandberg@arm.com ``pybind11/eigen.h`` header). 38112037Sandreas.sandberg@arm.com 38212037Sandreas.sandberg@arm.comThis behaviour is sometimes undesirable: the binding code may prefer to raise 38312037Sandreas.sandberg@arm.coman error rather than convert the argument. This behaviour can be obtained 38412037Sandreas.sandberg@arm.comthrough ``py::arg`` by calling the ``.noconvert()`` method of the ``py::arg`` 38512037Sandreas.sandberg@arm.comobject, such as: 38612037Sandreas.sandberg@arm.com 38712037Sandreas.sandberg@arm.com.. code-block:: cpp 38812037Sandreas.sandberg@arm.com 38912037Sandreas.sandberg@arm.com m.def("floats_only", [](double f) { return 0.5 * f; }, py::arg("f").noconvert()); 39012037Sandreas.sandberg@arm.com m.def("floats_preferred", [](double f) { return 0.5 * f; }, py::arg("f")); 39112037Sandreas.sandberg@arm.com 39212037Sandreas.sandberg@arm.comAttempting the call the second function (the one without ``.noconvert()``) with 39312037Sandreas.sandberg@arm.coman integer will succeed, but attempting to call the ``.noconvert()`` version 39412037Sandreas.sandberg@arm.comwill fail with a ``TypeError``: 39512037Sandreas.sandberg@arm.com 39612037Sandreas.sandberg@arm.com.. code-block:: pycon 39712037Sandreas.sandberg@arm.com 39812037Sandreas.sandberg@arm.com >>> floats_preferred(4) 39912037Sandreas.sandberg@arm.com 2.0 40012037Sandreas.sandberg@arm.com >>> floats_only(4) 40112037Sandreas.sandberg@arm.com Traceback (most recent call last): 40212037Sandreas.sandberg@arm.com File "<stdin>", line 1, in <module> 40312037Sandreas.sandberg@arm.com TypeError: floats_only(): incompatible function arguments. The following argument types are supported: 40412037Sandreas.sandberg@arm.com 1. (f: float) -> float 40512037Sandreas.sandberg@arm.com 40612037Sandreas.sandberg@arm.com Invoked with: 4 40712037Sandreas.sandberg@arm.com 40812037Sandreas.sandberg@arm.comYou may, of course, combine this with the :var:`_a` shorthand notation (see 40912037Sandreas.sandberg@arm.com:ref:`keyword_args`) and/or :ref:`default_args`. It is also permitted to omit 41012037Sandreas.sandberg@arm.comthe argument name by using the ``py::arg()`` constructor without an argument 41112037Sandreas.sandberg@arm.comname, i.e. by specifying ``py::arg().noconvert()``. 41212037Sandreas.sandberg@arm.com 41312037Sandreas.sandberg@arm.com.. note:: 41412037Sandreas.sandberg@arm.com 41512037Sandreas.sandberg@arm.com When specifying ``py::arg`` options it is necessary to provide the same 41612037Sandreas.sandberg@arm.com number of options as the bound function has arguments. Thus if you want to 41712037Sandreas.sandberg@arm.com enable no-convert behaviour for just one of several arguments, you will 41812037Sandreas.sandberg@arm.com need to specify a ``py::arg()`` annotation for each argument with the 41912037Sandreas.sandberg@arm.com no-convert argument modified to ``py::arg().noconvert()``. 42012037Sandreas.sandberg@arm.com 42112391Sjason@lowepower.com.. _none_arguments: 42212391Sjason@lowepower.com 42312391Sjason@lowepower.comAllow/Prohibiting None arguments 42412391Sjason@lowepower.com================================ 42512391Sjason@lowepower.com 42612391Sjason@lowepower.comWhen a C++ type registered with :class:`py::class_` is passed as an argument to 42712391Sjason@lowepower.coma function taking the instance as pointer or shared holder (e.g. ``shared_ptr`` 42812391Sjason@lowepower.comor a custom, copyable holder as described in :ref:`smart_pointers`), pybind 42912391Sjason@lowepower.comallows ``None`` to be passed from Python which results in calling the C++ 43012391Sjason@lowepower.comfunction with ``nullptr`` (or an empty holder) for the argument. 43112391Sjason@lowepower.com 43212391Sjason@lowepower.comTo explicitly enable or disable this behaviour, using the 43312391Sjason@lowepower.com``.none`` method of the :class:`py::arg` object: 43412391Sjason@lowepower.com 43512391Sjason@lowepower.com.. code-block:: cpp 43612391Sjason@lowepower.com 43712391Sjason@lowepower.com py::class_<Dog>(m, "Dog").def(py::init<>()); 43812391Sjason@lowepower.com py::class_<Cat>(m, "Cat").def(py::init<>()); 43912391Sjason@lowepower.com m.def("bark", [](Dog *dog) -> std::string { 44012391Sjason@lowepower.com if (dog) return "woof!"; /* Called with a Dog instance */ 44114299Sbbruce@ucdavis.edu else return "(no dog)"; /* Called with None, dog == nullptr */ 44212391Sjason@lowepower.com }, py::arg("dog").none(true)); 44312391Sjason@lowepower.com m.def("meow", [](Cat *cat) -> std::string { 44412391Sjason@lowepower.com // Can't be called with None argument 44512391Sjason@lowepower.com return "meow"; 44612391Sjason@lowepower.com }, py::arg("cat").none(false)); 44712391Sjason@lowepower.com 44812391Sjason@lowepower.comWith the above, the Python call ``bark(None)`` will return the string ``"(no 44912391Sjason@lowepower.comdog)"``, while attempting to call ``meow(None)`` will raise a ``TypeError``: 45012391Sjason@lowepower.com 45112391Sjason@lowepower.com.. code-block:: pycon 45212391Sjason@lowepower.com 45312391Sjason@lowepower.com >>> from animals import Dog, Cat, bark, meow 45412391Sjason@lowepower.com >>> bark(Dog()) 45512391Sjason@lowepower.com 'woof!' 45612391Sjason@lowepower.com >>> meow(Cat()) 45712391Sjason@lowepower.com 'meow' 45812391Sjason@lowepower.com >>> bark(None) 45912391Sjason@lowepower.com '(no dog)' 46012391Sjason@lowepower.com >>> meow(None) 46112391Sjason@lowepower.com Traceback (most recent call last): 46212391Sjason@lowepower.com File "<stdin>", line 1, in <module> 46312391Sjason@lowepower.com TypeError: meow(): incompatible function arguments. The following argument types are supported: 46412391Sjason@lowepower.com 1. (cat: animals.Cat) -> str 46512391Sjason@lowepower.com 46612391Sjason@lowepower.com Invoked with: None 46712391Sjason@lowepower.com 46812391Sjason@lowepower.comThe default behaviour when the tag is unspecified is to allow ``None``. 46912391Sjason@lowepower.com 47014299Sbbruce@ucdavis.edu.. note:: 47114299Sbbruce@ucdavis.edu 47214299Sbbruce@ucdavis.edu Even when ``.none(true)`` is specified for an argument, ``None`` will be converted to a 47314299Sbbruce@ucdavis.edu ``nullptr`` *only* for custom and :ref:`opaque <opaque>` types. Pointers to built-in types 47414299Sbbruce@ucdavis.edu (``double *``, ``int *``, ...) and STL types (``std::vector<T> *``, ...; if ``pybind11/stl.h`` 47514299Sbbruce@ucdavis.edu is included) are copied when converted to C++ (see :doc:`/advanced/cast/overview`) and will 47614299Sbbruce@ucdavis.edu not allow ``None`` as argument. To pass optional argument of these copied types consider 47714299Sbbruce@ucdavis.edu using ``std::optional<T>`` 47814299Sbbruce@ucdavis.edu 47912037Sandreas.sandberg@arm.comOverload resolution order 48012037Sandreas.sandberg@arm.com========================= 48112037Sandreas.sandberg@arm.com 48212037Sandreas.sandberg@arm.comWhen a function or method with multiple overloads is called from Python, 48312037Sandreas.sandberg@arm.compybind11 determines which overload to call in two passes. The first pass 48412037Sandreas.sandberg@arm.comattempts to call each overload without allowing argument conversion (as if 48514299Sbbruce@ucdavis.eduevery argument had been specified as ``py::arg().noconvert()`` as described 48612037Sandreas.sandberg@arm.comabove). 48712037Sandreas.sandberg@arm.com 48812037Sandreas.sandberg@arm.comIf no overload succeeds in the no-conversion first pass, a second pass is 48912037Sandreas.sandberg@arm.comattempted in which argument conversion is allowed (except where prohibited via 49012037Sandreas.sandberg@arm.coman explicit ``py::arg().noconvert()`` attribute in the function definition). 49112037Sandreas.sandberg@arm.com 49212037Sandreas.sandberg@arm.comIf the second pass also fails a ``TypeError`` is raised. 49312037Sandreas.sandberg@arm.com 49412037Sandreas.sandberg@arm.comWithin each pass, overloads are tried in the order they were registered with 49512037Sandreas.sandberg@arm.compybind11. 49612037Sandreas.sandberg@arm.com 49712037Sandreas.sandberg@arm.comWhat this means in practice is that pybind11 will prefer any overload that does 49812037Sandreas.sandberg@arm.comnot require conversion of arguments to an overload that does, but otherwise prefers 49912037Sandreas.sandberg@arm.comearlier-defined overloads to later-defined ones. 50012037Sandreas.sandberg@arm.com 50112037Sandreas.sandberg@arm.com.. note:: 50212037Sandreas.sandberg@arm.com 50312037Sandreas.sandberg@arm.com pybind11 does *not* further prioritize based on the number/pattern of 50412037Sandreas.sandberg@arm.com overloaded arguments. That is, pybind11 does not prioritize a function 50512037Sandreas.sandberg@arm.com requiring one conversion over one requiring three, but only prioritizes 50612037Sandreas.sandberg@arm.com overloads requiring no conversion at all to overloads that require 50712037Sandreas.sandberg@arm.com conversion of at least one argument. 508