misc.rst revision 12037:d28054ac6ec9
1Miscellaneous 2############# 3 4.. _macro_notes: 5 6General notes regarding convenience macros 7========================================== 8 9pybind11 provides a few convenience macros such as 10:func:`PYBIND11_MAKE_OPAQUE` and :func:`PYBIND11_DECLARE_HOLDER_TYPE`, and 11``PYBIND11_OVERLOAD_*``. Since these are "just" macros that are evaluated 12in the preprocessor (which has no concept of types), they *will* get confused 13by commas in a template argument such as ``PYBIND11_OVERLOAD(MyReturnValue<T1, 14T2>, myFunc)``. In this case, the preprocessor assumes that the comma indicates 15the beginning of the next parameter. Use a ``typedef`` to bind the template to 16another name and use it in the macro to avoid this problem. 17 18 19Global Interpreter Lock (GIL) 20============================= 21 22When calling a C++ function from Python, the GIL is always held. 23The classes :class:`gil_scoped_release` and :class:`gil_scoped_acquire` can be 24used to acquire and release the global interpreter lock in the body of a C++ 25function call. In this way, long-running C++ code can be parallelized using 26multiple Python threads. Taking :ref:`overriding_virtuals` as an example, this 27could be realized as follows (important changes highlighted): 28 29.. code-block:: cpp 30 :emphasize-lines: 8,9,33,34 31 32 class PyAnimal : public Animal { 33 public: 34 /* Inherit the constructors */ 35 using Animal::Animal; 36 37 /* Trampoline (need one for each virtual function) */ 38 std::string go(int n_times) { 39 /* Acquire GIL before calling Python code */ 40 py::gil_scoped_acquire acquire; 41 42 PYBIND11_OVERLOAD_PURE( 43 std::string, /* Return type */ 44 Animal, /* Parent class */ 45 go, /* Name of function */ 46 n_times /* Argument(s) */ 47 ); 48 } 49 }; 50 51 PYBIND11_PLUGIN(example) { 52 py::module m("example", "pybind11 example plugin"); 53 54 py::class_<Animal, PyAnimal> animal(m, "Animal"); 55 animal 56 .def(py::init<>()) 57 .def("go", &Animal::go); 58 59 py::class_<Dog>(m, "Dog", animal) 60 .def(py::init<>()); 61 62 m.def("call_go", [](Animal *animal) -> std::string { 63 /* Release GIL before calling into (potentially long-running) C++ code */ 64 py::gil_scoped_release release; 65 return call_go(animal); 66 }); 67 68 return m.ptr(); 69 } 70 71 72Binding sequence data types, iterators, the slicing protocol, etc. 73================================================================== 74 75Please refer to the supplemental example for details. 76 77.. seealso:: 78 79 The file :file:`tests/test_sequences_and_iterators.cpp` contains a 80 complete example that shows how to bind a sequence data type, including 81 length queries (``__len__``), iterators (``__iter__``), the slicing 82 protocol and other kinds of useful operations. 83 84 85Partitioning code over multiple extension modules 86================================================= 87 88It's straightforward to split binding code over multiple extension modules, 89while referencing types that are declared elsewhere. Everything "just" works 90without any special precautions. One exception to this rule occurs when 91extending a type declared in another extension module. Recall the basic example 92from Section :ref:`inheritance`. 93 94.. code-block:: cpp 95 96 py::class_<Pet> pet(m, "Pet"); 97 pet.def(py::init<const std::string &>()) 98 .def_readwrite("name", &Pet::name); 99 100 py::class_<Dog>(m, "Dog", pet /* <- specify parent */) 101 .def(py::init<const std::string &>()) 102 .def("bark", &Dog::bark); 103 104Suppose now that ``Pet`` bindings are defined in a module named ``basic``, 105whereas the ``Dog`` bindings are defined somewhere else. The challenge is of 106course that the variable ``pet`` is not available anymore though it is needed 107to indicate the inheritance relationship to the constructor of ``class_<Dog>``. 108However, it can be acquired as follows: 109 110.. code-block:: cpp 111 112 py::object pet = (py::object) py::module::import("basic").attr("Pet"); 113 114 py::class_<Dog>(m, "Dog", pet) 115 .def(py::init<const std::string &>()) 116 .def("bark", &Dog::bark); 117 118Alternatively, you can specify the base class as a template parameter option to 119``class_``, which performs an automated lookup of the corresponding Python 120type. Like the above code, however, this also requires invoking the ``import`` 121function once to ensure that the pybind11 binding code of the module ``basic`` 122has been executed: 123 124.. code-block:: cpp 125 126 py::module::import("basic"); 127 128 py::class_<Dog, Pet>(m, "Dog") 129 .def(py::init<const std::string &>()) 130 .def("bark", &Dog::bark); 131 132Naturally, both methods will fail when there are cyclic dependencies. 133 134Note that compiling code which has its default symbol visibility set to 135*hidden* (e.g. via the command line flag ``-fvisibility=hidden`` on GCC/Clang) can interfere with the 136ability to access types defined in another extension module. Workarounds 137include changing the global symbol visibility (not recommended, because it will 138lead unnecessarily large binaries) or manually exporting types that are 139accessed by multiple extension modules: 140 141.. code-block:: cpp 142 143 #ifdef _WIN32 144 # define EXPORT_TYPE __declspec(dllexport) 145 #else 146 # define EXPORT_TYPE __attribute__ ((visibility("default"))) 147 #endif 148 149 class EXPORT_TYPE Dog : public Animal { 150 ... 151 }; 152 153Note also that it is possible (although would rarely be required) to share arbitrary 154C++ objects between extension modules at runtime. Internal library data is shared 155between modules using capsule machinery [#f6]_ which can be also utilized for 156storing, modifying and accessing user-defined data. Note that an extension module 157will "see" other extensions' data if and only if they were built with the same 158pybind11 version. Consider the following example: 159 160.. code-block:: cpp 161 162 auto data = (MyData *) py::get_shared_data("mydata"); 163 if (!data) 164 data = (MyData *) py::set_shared_data("mydata", new MyData(42)); 165 166If the above snippet was used in several separately compiled extension modules, 167the first one to be imported would create a ``MyData`` instance and associate 168a ``"mydata"`` key with a pointer to it. Extensions that are imported later 169would be then able to access the data behind the same pointer. 170 171.. [#f6] https://docs.python.org/3/extending/extending.html#using-capsules 172 173Module Destructors 174================== 175 176pybind11 does not provide an explicit mechanism to invoke cleanup code at 177module destruction time. In rare cases where such functionality is required, it 178is possible to emulate it using Python capsules with a destruction callback. 179 180.. code-block:: cpp 181 182 auto cleanup_callback = []() { 183 // perform cleanup here -- this function is called with the GIL held 184 }; 185 186 m.add_object("_cleanup", py::capsule(cleanup_callback)); 187 188Generating documentation using Sphinx 189===================================== 190 191Sphinx [#f4]_ has the ability to inspect the signatures and documentation 192strings in pybind11-based extension modules to automatically generate beautiful 193documentation in a variety formats. The python_example repository [#f5]_ contains a 194simple example repository which uses this approach. 195 196There are two potential gotchas when using this approach: first, make sure that 197the resulting strings do not contain any :kbd:`TAB` characters, which break the 198docstring parsing routines. You may want to use C++11 raw string literals, 199which are convenient for multi-line comments. Conveniently, any excess 200indentation will be automatically be removed by Sphinx. However, for this to 201work, it is important that all lines are indented consistently, i.e.: 202 203.. code-block:: cpp 204 205 // ok 206 m.def("foo", &foo, R"mydelimiter( 207 The foo function 208 209 Parameters 210 ---------- 211 )mydelimiter"); 212 213 // *not ok* 214 m.def("foo", &foo, R"mydelimiter(The foo function 215 216 Parameters 217 ---------- 218 )mydelimiter"); 219 220By default, pybind11 automatically generates and prepends a signature to the docstring of a function 221registered with ``module::def()`` and ``class_::def()``. Sometimes this 222behavior is not desirable, because you want to provide your own signature or remove 223the docstring completely to exclude the function from the Sphinx documentation. 224The class ``options`` allows you to selectively suppress auto-generated signatures: 225 226.. code-block:: cpp 227 228 PYBIND11_PLUGIN(example) { 229 py::module m("example", "pybind11 example plugin"); 230 231 py::options options; 232 options.disable_function_signatures(); 233 234 m.def("add", [](int a, int b) { return a + b; }, "A function which adds two numbers"); 235 236 return m.ptr(); 237 } 238 239Note that changes to the settings affect only function bindings created during the 240lifetime of the ``options`` instance. When it goes out of scope at the end of the module's init function, 241the default settings are restored to prevent unwanted side effects. 242 243.. [#f4] http://www.sphinx-doc.org 244.. [#f5] http://github.com/pybind/python_example 245