misc.rst revision 11986:c12e4625ab56
1Miscellaneous 2############# 3 4.. _macro_notes: 5 6General notes regarding convenience macros 7========================================== 8 9pybind11 provides a few convenience macros such as 10:func:`PYBIND11_MAKE_OPAQUE` and :func:`PYBIND11_DECLARE_HOLDER_TYPE`, and 11``PYBIND11_OVERLOAD_*``. Since these are "just" macros that are evaluated 12in the preprocessor (which has no concept of types), they *will* get confused 13by commas in a template argument such as ``PYBIND11_OVERLOAD(MyReturnValue<T1, 14T2>, myFunc)``. In this case, the preprocessor assumes that the comma indicates 15the beginning of the next parameter. Use a ``typedef`` to bind the template to 16another name and use it in the macro to avoid this problem. 17 18 19Global Interpreter Lock (GIL) 20============================= 21 22The classes :class:`gil_scoped_release` and :class:`gil_scoped_acquire` can be 23used to acquire and release the global interpreter lock in the body of a C++ 24function call. In this way, long-running C++ code can be parallelized using 25multiple Python threads. Taking :ref:`overriding_virtuals` as an example, this 26could be realized as follows (important changes highlighted): 27 28.. code-block:: cpp 29 :emphasize-lines: 8,9,33,34 30 31 class PyAnimal : public Animal { 32 public: 33 /* Inherit the constructors */ 34 using Animal::Animal; 35 36 /* Trampoline (need one for each virtual function) */ 37 std::string go(int n_times) { 38 /* Acquire GIL before calling Python code */ 39 py::gil_scoped_acquire acquire; 40 41 PYBIND11_OVERLOAD_PURE( 42 std::string, /* Return type */ 43 Animal, /* Parent class */ 44 go, /* Name of function */ 45 n_times /* Argument(s) */ 46 ); 47 } 48 }; 49 50 PYBIND11_PLUGIN(example) { 51 py::module m("example", "pybind11 example plugin"); 52 53 py::class_<Animal, PyAnimal> animal(m, "Animal"); 54 animal 55 .def(py::init<>()) 56 .def("go", &Animal::go); 57 58 py::class_<Dog>(m, "Dog", animal) 59 .def(py::init<>()); 60 61 m.def("call_go", [](Animal *animal) -> std::string { 62 /* Release GIL before calling into (potentially long-running) C++ code */ 63 py::gil_scoped_release release; 64 return call_go(animal); 65 }); 66 67 return m.ptr(); 68 } 69 70 71Binding sequence data types, iterators, the slicing protocol, etc. 72================================================================== 73 74Please refer to the supplemental example for details. 75 76.. seealso:: 77 78 The file :file:`tests/test_sequences_and_iterators.cpp` contains a 79 complete example that shows how to bind a sequence data type, including 80 length queries (``__len__``), iterators (``__iter__``), the slicing 81 protocol and other kinds of useful operations. 82 83 84Partitioning code over multiple extension modules 85================================================= 86 87It's straightforward to split binding code over multiple extension modules, 88while referencing types that are declared elsewhere. Everything "just" works 89without any special precautions. One exception to this rule occurs when 90extending a type declared in another extension module. Recall the basic example 91from Section :ref:`inheritance`. 92 93.. code-block:: cpp 94 95 py::class_<Pet> pet(m, "Pet"); 96 pet.def(py::init<const std::string &>()) 97 .def_readwrite("name", &Pet::name); 98 99 py::class_<Dog>(m, "Dog", pet /* <- specify parent */) 100 .def(py::init<const std::string &>()) 101 .def("bark", &Dog::bark); 102 103Suppose now that ``Pet`` bindings are defined in a module named ``basic``, 104whereas the ``Dog`` bindings are defined somewhere else. The challenge is of 105course that the variable ``pet`` is not available anymore though it is needed 106to indicate the inheritance relationship to the constructor of ``class_<Dog>``. 107However, it can be acquired as follows: 108 109.. code-block:: cpp 110 111 py::object pet = (py::object) py::module::import("basic").attr("Pet"); 112 113 py::class_<Dog>(m, "Dog", pet) 114 .def(py::init<const std::string &>()) 115 .def("bark", &Dog::bark); 116 117Alternatively, you can specify the base class as a template parameter option to 118``class_``, which performs an automated lookup of the corresponding Python 119type. Like the above code, however, this also requires invoking the ``import`` 120function once to ensure that the pybind11 binding code of the module ``basic`` 121has been executed: 122 123.. code-block:: cpp 124 125 py::module::import("basic"); 126 127 py::class_<Dog, Pet>(m, "Dog") 128 .def(py::init<const std::string &>()) 129 .def("bark", &Dog::bark); 130 131Naturally, both methods will fail when there are cyclic dependencies. 132 133Note that compiling code which has its default symbol visibility set to 134*hidden* (e.g. via the command line flag ``-fvisibility=hidden`` on GCC/Clang) can interfere with the 135ability to access types defined in another extension module. Workarounds 136include changing the global symbol visibility (not recommended, because it will 137lead unnecessarily large binaries) or manually exporting types that are 138accessed by multiple extension modules: 139 140.. code-block:: cpp 141 142 #ifdef _WIN32 143 # define EXPORT_TYPE __declspec(dllexport) 144 #else 145 # define EXPORT_TYPE __attribute__ ((visibility("default"))) 146 #endif 147 148 class EXPORT_TYPE Dog : public Animal { 149 ... 150 }; 151 152Note also that it is possible (although would rarely be required) to share arbitrary 153C++ objects between extension modules at runtime. Internal library data is shared 154between modules using capsule machinery [#f6]_ which can be also utilized for 155storing, modifying and accessing user-defined data. Note that an extension module 156will "see" other extensions' data if and only if they were built with the same 157pybind11 version. Consider the following example: 158 159.. code-block:: cpp 160 161 auto data = (MyData *) py::get_shared_data("mydata"); 162 if (!data) 163 data = (MyData *) py::set_shared_data("mydata", new MyData(42)); 164 165If the above snippet was used in several separately compiled extension modules, 166the first one to be imported would create a ``MyData`` instance and associate 167a ``"mydata"`` key with a pointer to it. Extensions that are imported later 168would be then able to access the data behind the same pointer. 169 170.. [#f6] https://docs.python.org/3/extending/extending.html#using-capsules 171 172 173Generating documentation using Sphinx 174===================================== 175 176Sphinx [#f4]_ has the ability to inspect the signatures and documentation 177strings in pybind11-based extension modules to automatically generate beautiful 178documentation in a variety formats. The python_example repository [#f5]_ contains a 179simple example repository which uses this approach. 180 181There are two potential gotchas when using this approach: first, make sure that 182the resulting strings do not contain any :kbd:`TAB` characters, which break the 183docstring parsing routines. You may want to use C++11 raw string literals, 184which are convenient for multi-line comments. Conveniently, any excess 185indentation will be automatically be removed by Sphinx. However, for this to 186work, it is important that all lines are indented consistently, i.e.: 187 188.. code-block:: cpp 189 190 // ok 191 m.def("foo", &foo, R"mydelimiter( 192 The foo function 193 194 Parameters 195 ---------- 196 )mydelimiter"); 197 198 // *not ok* 199 m.def("foo", &foo, R"mydelimiter(The foo function 200 201 Parameters 202 ---------- 203 )mydelimiter"); 204 205By default, pybind11 automatically generates and prepends a signature to the docstring of a function 206registered with ``module::def()`` and ``class_::def()``. Sometimes this 207behavior is not desirable, because you want to provide your own signature or remove 208the docstring completely to exclude the function from the Sphinx documentation. 209The class ``options`` allows you to selectively suppress auto-generated signatures: 210 211.. code-block:: cpp 212 213 PYBIND11_PLUGIN(example) { 214 py::module m("example", "pybind11 example plugin"); 215 216 py::options options; 217 options.disable_function_signatures(); 218 219 m.def("add", [](int a, int b) { return a + b; }, "A function which adds two numbers"); 220 221 return m.ptr(); 222 } 223 224Note that changes to the settings affect only function bindings created during the 225lifetime of the ``options`` instance. When it goes out of scope at the end of the module's init function, 226the default settings are restored to prevent unwanted side effects. 227 228.. [#f4] http://www.sphinx-doc.org 229.. [#f5] http://github.com/pybind/python_example 230