faq.rst revision 11986:c12e4625ab56
1Frequently asked questions 2########################## 3 4"ImportError: dynamic module does not define init function" 5=========================================================== 6 71. Make sure that the name specified in ``pybind::module`` and 8 ``PYBIND11_PLUGIN`` is consistent and identical to the filename of the 9 extension library. The latter should not contain any extra prefixes (e.g. 10 ``test.so`` instead of ``libtest.so``). 11 122. If the above did not fix your issue, then you are likely using an 13 incompatible version of Python (for instance, the extension library was 14 compiled against Python 2, while the interpreter is running on top of some 15 version of Python 3, or vice versa) 16 17"Symbol not found: ``__Py_ZeroStruct`` / ``_PyInstanceMethod_Type``" 18======================================================================== 19 20See item 2 of the first answer. 21 22"SystemError: dynamic module not initialized properly" 23====================================================== 24 25See item 2 of the first answer. 26 27The Python interpreter immediately crashes when importing my module 28=================================================================== 29 30See item 2 of the first answer. 31 32CMake doesn't detect the right Python version 33============================================= 34 35The CMake-based build system will try to automatically detect the installed 36version of Python and link against that. When this fails, or when there are 37multiple versions of Python and it finds the wrong one, delete 38``CMakeCache.txt`` and then invoke CMake as follows: 39 40.. code-block:: bash 41 42 cmake -DPYTHON_EXECUTABLE:FILEPATH=<path-to-python-executable> . 43 44Limitations involving reference arguments 45========================================= 46 47In C++, it's fairly common to pass arguments using mutable references or 48mutable pointers, which allows both read and write access to the value 49supplied by the caller. This is sometimes done for efficiency reasons, or to 50realize functions that have multiple return values. Here are two very basic 51examples: 52 53.. code-block:: cpp 54 55 void increment(int &i) { i++; } 56 void increment_ptr(int *i) { (*i)++; } 57 58In Python, all arguments are passed by reference, so there is no general 59issue in binding such code from Python. 60 61However, certain basic Python types (like ``str``, ``int``, ``bool``, 62``float``, etc.) are **immutable**. This means that the following attempt 63to port the function to Python doesn't have the same effect on the value 64provided by the caller -- in fact, it does nothing at all. 65 66.. code-block:: python 67 68 def increment(i): 69 i += 1 # nope.. 70 71pybind11 is also affected by such language-level conventions, which means that 72binding ``increment`` or ``increment_ptr`` will also create Python functions 73that don't modify their arguments. 74 75Although inconvenient, one workaround is to encapsulate the immutable types in 76a custom type that does allow modifications. 77 78An other alternative involves binding a small wrapper lambda function that 79returns a tuple with all output arguments (see the remainder of the 80documentation for examples on binding lambda functions). An example: 81 82.. code-block:: cpp 83 84 int foo(int &i) { i++; return 123; } 85 86and the binding code 87 88.. code-block:: cpp 89 90 m.def("foo", [](int i) { int rv = foo(i); return std::make_tuple(rv, i); }); 91 92 93How can I reduce the build time? 94================================ 95 96It's good practice to split binding code over multiple files, as in the 97following example: 98 99:file:`example.cpp`: 100 101.. code-block:: cpp 102 103 void init_ex1(py::module &); 104 void init_ex2(py::module &); 105 /* ... */ 106 107 PYBIND11_PLUGIN(example) { 108 py::module m("example", "pybind example plugin"); 109 110 init_ex1(m); 111 init_ex2(m); 112 /* ... */ 113 114 return m.ptr(); 115 } 116 117:file:`ex1.cpp`: 118 119.. code-block:: cpp 120 121 void init_ex1(py::module &m) { 122 m.def("add", [](int a, int b) { return a + b; }); 123 } 124 125:file:`ex2.cpp`: 126 127.. code-block:: cpp 128 129 void init_ex1(py::module &m) { 130 m.def("sub", [](int a, int b) { return a - b; }); 131 } 132 133:command:`python`: 134 135.. code-block:: pycon 136 137 >>> import example 138 >>> example.add(1, 2) 139 3 140 >>> example.sub(1, 1) 141 0 142 143As shown above, the various ``init_ex`` functions should be contained in 144separate files that can be compiled independently from one another, and then 145linked together into the same final shared object. Following this approach 146will: 147 1481. reduce memory requirements per compilation unit. 149 1502. enable parallel builds (if desired). 151 1523. allow for faster incremental builds. For instance, when a single class 153 definition is changed, only a subset of the binding code will generally need 154 to be recompiled. 155 156"recursive template instantiation exceeded maximum depth of 256" 157================================================================ 158 159If you receive an error about excessive recursive template evaluation, try 160specifying a larger value, e.g. ``-ftemplate-depth=1024`` on GCC/Clang. The 161culprit is generally the generation of function signatures at compile time 162using C++14 template metaprogramming. 163 164 165.. _`faq:symhidden`: 166 167How can I create smaller binaries? 168================================== 169 170To do its job, pybind11 extensively relies on a programming technique known as 171*template metaprogramming*, which is a way of performing computation at compile 172time using type information. Template metaprogamming usually instantiates code 173involving significant numbers of deeply nested types that are either completely 174removed or reduced to just a few instructions during the compiler's optimization 175phase. However, due to the nested nature of these types, the resulting symbol 176names in the compiled extension library can be extremely long. For instance, 177the included test suite contains the following symbol: 178 179.. only:: html 180 181 .. code-block:: none 182 183 __ZN8pybind1112cpp_functionC1Iv8Example2JRNSt3__16vectorINS3_12basic_stringIwNS3_11char_traitsIwEENS3_9allocatorIwEEEENS8_ISA_EEEEEJNS_4nameENS_7siblingENS_9is_methodEA28_cEEEMT0_FT_DpT1_EDpRKT2_ 184 185.. only:: not html 186 187 .. code-block:: cpp 188 189 __ZN8pybind1112cpp_functionC1Iv8Example2JRNSt3__16vectorINS3_12basic_stringIwNS3_11char_traitsIwEENS3_9allocatorIwEEEENS8_ISA_EEEEEJNS_4nameENS_7siblingENS_9is_methodEA28_cEEEMT0_FT_DpT1_EDpRKT2_ 190 191which is the mangled form of the following function type: 192 193.. code-block:: cpp 194 195 pybind11::cpp_function::cpp_function<void, Example2, std::__1::vector<std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> >, std::__1::allocator<std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> > > >&, pybind11::name, pybind11::sibling, pybind11::is_method, char [28]>(void (Example2::*)(std::__1::vector<std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> >, std::__1::allocator<std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> > > >&), pybind11::name const&, pybind11::sibling const&, pybind11::is_method const&, char const (&) [28]) 196 197The memory needed to store just the mangled name of this function (196 bytes) 198is larger than the actual piece of code (111 bytes) it represents! On the other 199hand, it's silly to even give this function a name -- after all, it's just a 200tiny cog in a bigger piece of machinery that is not exposed to the outside 201world. So we'll generally only want to export symbols for those functions which 202are actually called from the outside. 203 204This can be achieved by specifying the parameter ``-fvisibility=hidden`` to GCC 205and Clang, which sets the default symbol visibility to *hidden*. It's best to 206do this only for release builds, since the symbol names can be helpful in 207debugging sessions. On Visual Studio, symbols are already hidden by default, so 208nothing needs to be done there. Needless to say, this has a tremendous impact 209on the final binary size of the resulting extension library. 210 211Another aspect that can require a fair bit of code are function signature 212descriptions. pybind11 automatically generates human-readable function 213signatures for docstrings, e.g.: 214 215.. code-block:: none 216 217 | __init__(...) 218 | __init__(*args, **kwargs) 219 | Overloaded function. 220 | 221 | 1. __init__(example.Example1) -> NoneType 222 | 223 | Docstring for overload #1 goes here 224 | 225 | 2. __init__(example.Example1, int) -> NoneType 226 | 227 | Docstring for overload #2 goes here 228 | 229 | 3. __init__(example.Example1, example.Example1) -> NoneType 230 | 231 | Docstring for overload #3 goes here 232 233 234In C++11 mode, these are generated at run time using string concatenation, 235which can amount to 10-20% of the size of the resulting binary. If you can, 236enable C++14 language features (using ``-std=c++14`` for GCC/Clang), in which 237case signatures are efficiently pre-generated at compile time. Unfortunately, 238Visual Studio's C++14 support (``constexpr``) is not good enough as of April 2392016, so it always uses the more expensive run-time approach. 240 241Working with ancient Visual Studio 2009 builds on Windows 242========================================================= 243 244The official Windows distributions of Python are compiled using truly 245ancient versions of Visual Studio that lack good C++11 support. Some users 246implicitly assume that it would be impossible to load a plugin built with 247Visual Studio 2015 into a Python distribution that was compiled using Visual 248Studio 2009. However, no such issue exists: it's perfectly legitimate to 249interface DLLs that are built with different compilers and/or C libraries. 250Common gotchas to watch out for involve not ``free()``-ing memory region 251that that were ``malloc()``-ed in another shared library, using data 252structures with incompatible ABIs, and so on. pybind11 is very careful not 253to make these types of mistakes. 254