misc.rst revision 11986
111986Sandreas.sandberg@arm.comMiscellaneous 211986Sandreas.sandberg@arm.com############# 311986Sandreas.sandberg@arm.com 411986Sandreas.sandberg@arm.com.. _macro_notes: 511986Sandreas.sandberg@arm.com 611986Sandreas.sandberg@arm.comGeneral notes regarding convenience macros 711986Sandreas.sandberg@arm.com========================================== 811986Sandreas.sandberg@arm.com 911986Sandreas.sandberg@arm.compybind11 provides a few convenience macros such as 1011986Sandreas.sandberg@arm.com:func:`PYBIND11_MAKE_OPAQUE` and :func:`PYBIND11_DECLARE_HOLDER_TYPE`, and 1111986Sandreas.sandberg@arm.com``PYBIND11_OVERLOAD_*``. Since these are "just" macros that are evaluated 1211986Sandreas.sandberg@arm.comin the preprocessor (which has no concept of types), they *will* get confused 1311986Sandreas.sandberg@arm.comby commas in a template argument such as ``PYBIND11_OVERLOAD(MyReturnValue<T1, 1411986Sandreas.sandberg@arm.comT2>, myFunc)``. In this case, the preprocessor assumes that the comma indicates 1511986Sandreas.sandberg@arm.comthe beginning of the next parameter. Use a ``typedef`` to bind the template to 1611986Sandreas.sandberg@arm.comanother name and use it in the macro to avoid this problem. 1711986Sandreas.sandberg@arm.com 1811986Sandreas.sandberg@arm.com 1911986Sandreas.sandberg@arm.comGlobal Interpreter Lock (GIL) 2011986Sandreas.sandberg@arm.com============================= 2111986Sandreas.sandberg@arm.com 2211986Sandreas.sandberg@arm.comThe classes :class:`gil_scoped_release` and :class:`gil_scoped_acquire` can be 2311986Sandreas.sandberg@arm.comused to acquire and release the global interpreter lock in the body of a C++ 2411986Sandreas.sandberg@arm.comfunction call. In this way, long-running C++ code can be parallelized using 2511986Sandreas.sandberg@arm.commultiple Python threads. Taking :ref:`overriding_virtuals` as an example, this 2611986Sandreas.sandberg@arm.comcould be realized as follows (important changes highlighted): 2711986Sandreas.sandberg@arm.com 2811986Sandreas.sandberg@arm.com.. code-block:: cpp 2911986Sandreas.sandberg@arm.com :emphasize-lines: 8,9,33,34 3011986Sandreas.sandberg@arm.com 3111986Sandreas.sandberg@arm.com class PyAnimal : public Animal { 3211986Sandreas.sandberg@arm.com public: 3311986Sandreas.sandberg@arm.com /* Inherit the constructors */ 3411986Sandreas.sandberg@arm.com using Animal::Animal; 3511986Sandreas.sandberg@arm.com 3611986Sandreas.sandberg@arm.com /* Trampoline (need one for each virtual function) */ 3711986Sandreas.sandberg@arm.com std::string go(int n_times) { 3811986Sandreas.sandberg@arm.com /* Acquire GIL before calling Python code */ 3911986Sandreas.sandberg@arm.com py::gil_scoped_acquire acquire; 4011986Sandreas.sandberg@arm.com 4111986Sandreas.sandberg@arm.com PYBIND11_OVERLOAD_PURE( 4211986Sandreas.sandberg@arm.com std::string, /* Return type */ 4311986Sandreas.sandberg@arm.com Animal, /* Parent class */ 4411986Sandreas.sandberg@arm.com go, /* Name of function */ 4511986Sandreas.sandberg@arm.com n_times /* Argument(s) */ 4611986Sandreas.sandberg@arm.com ); 4711986Sandreas.sandberg@arm.com } 4811986Sandreas.sandberg@arm.com }; 4911986Sandreas.sandberg@arm.com 5011986Sandreas.sandberg@arm.com PYBIND11_PLUGIN(example) { 5111986Sandreas.sandberg@arm.com py::module m("example", "pybind11 example plugin"); 5211986Sandreas.sandberg@arm.com 5311986Sandreas.sandberg@arm.com py::class_<Animal, PyAnimal> animal(m, "Animal"); 5411986Sandreas.sandberg@arm.com animal 5511986Sandreas.sandberg@arm.com .def(py::init<>()) 5611986Sandreas.sandberg@arm.com .def("go", &Animal::go); 5711986Sandreas.sandberg@arm.com 5811986Sandreas.sandberg@arm.com py::class_<Dog>(m, "Dog", animal) 5911986Sandreas.sandberg@arm.com .def(py::init<>()); 6011986Sandreas.sandberg@arm.com 6111986Sandreas.sandberg@arm.com m.def("call_go", [](Animal *animal) -> std::string { 6211986Sandreas.sandberg@arm.com /* Release GIL before calling into (potentially long-running) C++ code */ 6311986Sandreas.sandberg@arm.com py::gil_scoped_release release; 6411986Sandreas.sandberg@arm.com return call_go(animal); 6511986Sandreas.sandberg@arm.com }); 6611986Sandreas.sandberg@arm.com 6711986Sandreas.sandberg@arm.com return m.ptr(); 6811986Sandreas.sandberg@arm.com } 6911986Sandreas.sandberg@arm.com 7011986Sandreas.sandberg@arm.com 7111986Sandreas.sandberg@arm.comBinding sequence data types, iterators, the slicing protocol, etc. 7211986Sandreas.sandberg@arm.com================================================================== 7311986Sandreas.sandberg@arm.com 7411986Sandreas.sandberg@arm.comPlease refer to the supplemental example for details. 7511986Sandreas.sandberg@arm.com 7611986Sandreas.sandberg@arm.com.. seealso:: 7711986Sandreas.sandberg@arm.com 7811986Sandreas.sandberg@arm.com The file :file:`tests/test_sequences_and_iterators.cpp` contains a 7911986Sandreas.sandberg@arm.com complete example that shows how to bind a sequence data type, including 8011986Sandreas.sandberg@arm.com length queries (``__len__``), iterators (``__iter__``), the slicing 8111986Sandreas.sandberg@arm.com protocol and other kinds of useful operations. 8211986Sandreas.sandberg@arm.com 8311986Sandreas.sandberg@arm.com 8411986Sandreas.sandberg@arm.comPartitioning code over multiple extension modules 8511986Sandreas.sandberg@arm.com================================================= 8611986Sandreas.sandberg@arm.com 8711986Sandreas.sandberg@arm.comIt's straightforward to split binding code over multiple extension modules, 8811986Sandreas.sandberg@arm.comwhile referencing types that are declared elsewhere. Everything "just" works 8911986Sandreas.sandberg@arm.comwithout any special precautions. One exception to this rule occurs when 9011986Sandreas.sandberg@arm.comextending a type declared in another extension module. Recall the basic example 9111986Sandreas.sandberg@arm.comfrom Section :ref:`inheritance`. 9211986Sandreas.sandberg@arm.com 9311986Sandreas.sandberg@arm.com.. code-block:: cpp 9411986Sandreas.sandberg@arm.com 9511986Sandreas.sandberg@arm.com py::class_<Pet> pet(m, "Pet"); 9611986Sandreas.sandberg@arm.com pet.def(py::init<const std::string &>()) 9711986Sandreas.sandberg@arm.com .def_readwrite("name", &Pet::name); 9811986Sandreas.sandberg@arm.com 9911986Sandreas.sandberg@arm.com py::class_<Dog>(m, "Dog", pet /* <- specify parent */) 10011986Sandreas.sandberg@arm.com .def(py::init<const std::string &>()) 10111986Sandreas.sandberg@arm.com .def("bark", &Dog::bark); 10211986Sandreas.sandberg@arm.com 10311986Sandreas.sandberg@arm.comSuppose now that ``Pet`` bindings are defined in a module named ``basic``, 10411986Sandreas.sandberg@arm.comwhereas the ``Dog`` bindings are defined somewhere else. The challenge is of 10511986Sandreas.sandberg@arm.comcourse that the variable ``pet`` is not available anymore though it is needed 10611986Sandreas.sandberg@arm.comto indicate the inheritance relationship to the constructor of ``class_<Dog>``. 10711986Sandreas.sandberg@arm.comHowever, it can be acquired as follows: 10811986Sandreas.sandberg@arm.com 10911986Sandreas.sandberg@arm.com.. code-block:: cpp 11011986Sandreas.sandberg@arm.com 11111986Sandreas.sandberg@arm.com py::object pet = (py::object) py::module::import("basic").attr("Pet"); 11211986Sandreas.sandberg@arm.com 11311986Sandreas.sandberg@arm.com py::class_<Dog>(m, "Dog", pet) 11411986Sandreas.sandberg@arm.com .def(py::init<const std::string &>()) 11511986Sandreas.sandberg@arm.com .def("bark", &Dog::bark); 11611986Sandreas.sandberg@arm.com 11711986Sandreas.sandberg@arm.comAlternatively, you can specify the base class as a template parameter option to 11811986Sandreas.sandberg@arm.com``class_``, which performs an automated lookup of the corresponding Python 11911986Sandreas.sandberg@arm.comtype. Like the above code, however, this also requires invoking the ``import`` 12011986Sandreas.sandberg@arm.comfunction once to ensure that the pybind11 binding code of the module ``basic`` 12111986Sandreas.sandberg@arm.comhas been executed: 12211986Sandreas.sandberg@arm.com 12311986Sandreas.sandberg@arm.com.. code-block:: cpp 12411986Sandreas.sandberg@arm.com 12511986Sandreas.sandberg@arm.com py::module::import("basic"); 12611986Sandreas.sandberg@arm.com 12711986Sandreas.sandberg@arm.com py::class_<Dog, Pet>(m, "Dog") 12811986Sandreas.sandberg@arm.com .def(py::init<const std::string &>()) 12911986Sandreas.sandberg@arm.com .def("bark", &Dog::bark); 13011986Sandreas.sandberg@arm.com 13111986Sandreas.sandberg@arm.comNaturally, both methods will fail when there are cyclic dependencies. 13211986Sandreas.sandberg@arm.com 13311986Sandreas.sandberg@arm.comNote that compiling code which has its default symbol visibility set to 13411986Sandreas.sandberg@arm.com*hidden* (e.g. via the command line flag ``-fvisibility=hidden`` on GCC/Clang) can interfere with the 13511986Sandreas.sandberg@arm.comability to access types defined in another extension module. Workarounds 13611986Sandreas.sandberg@arm.cominclude changing the global symbol visibility (not recommended, because it will 13711986Sandreas.sandberg@arm.comlead unnecessarily large binaries) or manually exporting types that are 13811986Sandreas.sandberg@arm.comaccessed by multiple extension modules: 13911986Sandreas.sandberg@arm.com 14011986Sandreas.sandberg@arm.com.. code-block:: cpp 14111986Sandreas.sandberg@arm.com 14211986Sandreas.sandberg@arm.com #ifdef _WIN32 14311986Sandreas.sandberg@arm.com # define EXPORT_TYPE __declspec(dllexport) 14411986Sandreas.sandberg@arm.com #else 14511986Sandreas.sandberg@arm.com # define EXPORT_TYPE __attribute__ ((visibility("default"))) 14611986Sandreas.sandberg@arm.com #endif 14711986Sandreas.sandberg@arm.com 14811986Sandreas.sandberg@arm.com class EXPORT_TYPE Dog : public Animal { 14911986Sandreas.sandberg@arm.com ... 15011986Sandreas.sandberg@arm.com }; 15111986Sandreas.sandberg@arm.com 15211986Sandreas.sandberg@arm.comNote also that it is possible (although would rarely be required) to share arbitrary 15311986Sandreas.sandberg@arm.comC++ objects between extension modules at runtime. Internal library data is shared 15411986Sandreas.sandberg@arm.combetween modules using capsule machinery [#f6]_ which can be also utilized for 15511986Sandreas.sandberg@arm.comstoring, modifying and accessing user-defined data. Note that an extension module 15611986Sandreas.sandberg@arm.comwill "see" other extensions' data if and only if they were built with the same 15711986Sandreas.sandberg@arm.compybind11 version. Consider the following example: 15811986Sandreas.sandberg@arm.com 15911986Sandreas.sandberg@arm.com.. code-block:: cpp 16011986Sandreas.sandberg@arm.com 16111986Sandreas.sandberg@arm.com auto data = (MyData *) py::get_shared_data("mydata"); 16211986Sandreas.sandberg@arm.com if (!data) 16311986Sandreas.sandberg@arm.com data = (MyData *) py::set_shared_data("mydata", new MyData(42)); 16411986Sandreas.sandberg@arm.com 16511986Sandreas.sandberg@arm.comIf the above snippet was used in several separately compiled extension modules, 16611986Sandreas.sandberg@arm.comthe first one to be imported would create a ``MyData`` instance and associate 16711986Sandreas.sandberg@arm.coma ``"mydata"`` key with a pointer to it. Extensions that are imported later 16811986Sandreas.sandberg@arm.comwould be then able to access the data behind the same pointer. 16911986Sandreas.sandberg@arm.com 17011986Sandreas.sandberg@arm.com.. [#f6] https://docs.python.org/3/extending/extending.html#using-capsules 17111986Sandreas.sandberg@arm.com 17211986Sandreas.sandberg@arm.com 17311986Sandreas.sandberg@arm.comGenerating documentation using Sphinx 17411986Sandreas.sandberg@arm.com===================================== 17511986Sandreas.sandberg@arm.com 17611986Sandreas.sandberg@arm.comSphinx [#f4]_ has the ability to inspect the signatures and documentation 17711986Sandreas.sandberg@arm.comstrings in pybind11-based extension modules to automatically generate beautiful 17811986Sandreas.sandberg@arm.comdocumentation in a variety formats. The python_example repository [#f5]_ contains a 17911986Sandreas.sandberg@arm.comsimple example repository which uses this approach. 18011986Sandreas.sandberg@arm.com 18111986Sandreas.sandberg@arm.comThere are two potential gotchas when using this approach: first, make sure that 18211986Sandreas.sandberg@arm.comthe resulting strings do not contain any :kbd:`TAB` characters, which break the 18311986Sandreas.sandberg@arm.comdocstring parsing routines. You may want to use C++11 raw string literals, 18411986Sandreas.sandberg@arm.comwhich are convenient for multi-line comments. Conveniently, any excess 18511986Sandreas.sandberg@arm.comindentation will be automatically be removed by Sphinx. However, for this to 18611986Sandreas.sandberg@arm.comwork, it is important that all lines are indented consistently, i.e.: 18711986Sandreas.sandberg@arm.com 18811986Sandreas.sandberg@arm.com.. code-block:: cpp 18911986Sandreas.sandberg@arm.com 19011986Sandreas.sandberg@arm.com // ok 19111986Sandreas.sandberg@arm.com m.def("foo", &foo, R"mydelimiter( 19211986Sandreas.sandberg@arm.com The foo function 19311986Sandreas.sandberg@arm.com 19411986Sandreas.sandberg@arm.com Parameters 19511986Sandreas.sandberg@arm.com ---------- 19611986Sandreas.sandberg@arm.com )mydelimiter"); 19711986Sandreas.sandberg@arm.com 19811986Sandreas.sandberg@arm.com // *not ok* 19911986Sandreas.sandberg@arm.com m.def("foo", &foo, R"mydelimiter(The foo function 20011986Sandreas.sandberg@arm.com 20111986Sandreas.sandberg@arm.com Parameters 20211986Sandreas.sandberg@arm.com ---------- 20311986Sandreas.sandberg@arm.com )mydelimiter"); 20411986Sandreas.sandberg@arm.com 20511986Sandreas.sandberg@arm.comBy default, pybind11 automatically generates and prepends a signature to the docstring of a function 20611986Sandreas.sandberg@arm.comregistered with ``module::def()`` and ``class_::def()``. Sometimes this 20711986Sandreas.sandberg@arm.combehavior is not desirable, because you want to provide your own signature or remove 20811986Sandreas.sandberg@arm.comthe docstring completely to exclude the function from the Sphinx documentation. 20911986Sandreas.sandberg@arm.comThe class ``options`` allows you to selectively suppress auto-generated signatures: 21011986Sandreas.sandberg@arm.com 21111986Sandreas.sandberg@arm.com.. code-block:: cpp 21211986Sandreas.sandberg@arm.com 21311986Sandreas.sandberg@arm.com PYBIND11_PLUGIN(example) { 21411986Sandreas.sandberg@arm.com py::module m("example", "pybind11 example plugin"); 21511986Sandreas.sandberg@arm.com 21611986Sandreas.sandberg@arm.com py::options options; 21711986Sandreas.sandberg@arm.com options.disable_function_signatures(); 21811986Sandreas.sandberg@arm.com 21911986Sandreas.sandberg@arm.com m.def("add", [](int a, int b) { return a + b; }, "A function which adds two numbers"); 22011986Sandreas.sandberg@arm.com 22111986Sandreas.sandberg@arm.com return m.ptr(); 22211986Sandreas.sandberg@arm.com } 22311986Sandreas.sandberg@arm.com 22411986Sandreas.sandberg@arm.comNote that changes to the settings affect only function bindings created during the 22511986Sandreas.sandberg@arm.comlifetime of the ``options`` instance. When it goes out of scope at the end of the module's init function, 22611986Sandreas.sandberg@arm.comthe default settings are restored to prevent unwanted side effects. 22711986Sandreas.sandberg@arm.com 22811986Sandreas.sandberg@arm.com.. [#f4] http://www.sphinx-doc.org 22911986Sandreas.sandberg@arm.com.. [#f5] http://github.com/pybind/python_example 230