compiling.rst revision 14299
112391Sjason@lowepower.com.. _compiling: 212391Sjason@lowepower.com 311986Sandreas.sandberg@arm.comBuild systems 411986Sandreas.sandberg@arm.com############# 511986Sandreas.sandberg@arm.com 611986Sandreas.sandberg@arm.comBuilding with setuptools 711986Sandreas.sandberg@arm.com======================== 811986Sandreas.sandberg@arm.com 911986Sandreas.sandberg@arm.comFor projects on PyPI, building with setuptools is the way to go. Sylvain Corlay 1011986Sandreas.sandberg@arm.comhas kindly provided an example project which shows how to set up everything, 1111986Sandreas.sandberg@arm.comincluding automatic generation of documentation using Sphinx. Please refer to 1211986Sandreas.sandberg@arm.comthe [python_example]_ repository. 1311986Sandreas.sandberg@arm.com 1411986Sandreas.sandberg@arm.com.. [python_example] https://github.com/pybind/python_example 1511986Sandreas.sandberg@arm.com 1611986Sandreas.sandberg@arm.comBuilding with cppimport 1711986Sandreas.sandberg@arm.com======================== 1811986Sandreas.sandberg@arm.com 1912391Sjason@lowepower.com[cppimport]_ is a small Python import hook that determines whether there is a C++ 2012391Sjason@lowepower.comsource file whose name matches the requested module. If there is, the file is 2112391Sjason@lowepower.comcompiled as a Python extension using pybind11 and placed in the same folder as 2212391Sjason@lowepower.comthe C++ source file. Python is then able to find the module and load it. 2311986Sandreas.sandberg@arm.com 2411986Sandreas.sandberg@arm.com.. [cppimport] https://github.com/tbenthompson/cppimport 2511986Sandreas.sandberg@arm.com 2611986Sandreas.sandberg@arm.com.. _cmake: 2711986Sandreas.sandberg@arm.com 2811986Sandreas.sandberg@arm.comBuilding with CMake 2911986Sandreas.sandberg@arm.com=================== 3011986Sandreas.sandberg@arm.com 3111986Sandreas.sandberg@arm.comFor C++ codebases that have an existing CMake-based build system, a Python 3211986Sandreas.sandberg@arm.comextension module can be created with just a few lines of code: 3311986Sandreas.sandberg@arm.com 3411986Sandreas.sandberg@arm.com.. code-block:: cmake 3511986Sandreas.sandberg@arm.com 3611986Sandreas.sandberg@arm.com cmake_minimum_required(VERSION 2.8.12) 3711986Sandreas.sandberg@arm.com project(example) 3811986Sandreas.sandberg@arm.com 3911986Sandreas.sandberg@arm.com add_subdirectory(pybind11) 4011986Sandreas.sandberg@arm.com pybind11_add_module(example example.cpp) 4111986Sandreas.sandberg@arm.com 4211986Sandreas.sandberg@arm.comThis assumes that the pybind11 repository is located in a subdirectory named 4311986Sandreas.sandberg@arm.com:file:`pybind11` and that the code is located in a file named :file:`example.cpp`. 4412037Sandreas.sandberg@arm.comThe CMake command ``add_subdirectory`` will import the pybind11 project which 4512037Sandreas.sandberg@arm.comprovides the ``pybind11_add_module`` function. It will take care of all the 4612037Sandreas.sandberg@arm.comdetails needed to build a Python extension module on any platform. 4711986Sandreas.sandberg@arm.com 4811986Sandreas.sandberg@arm.comA working sample project, including a way to invoke CMake from :file:`setup.py` for 4911986Sandreas.sandberg@arm.comPyPI integration, can be found in the [cmake_example]_ repository. 5011986Sandreas.sandberg@arm.com 5111986Sandreas.sandberg@arm.com.. [cmake_example] https://github.com/pybind/cmake_example 5211986Sandreas.sandberg@arm.com 5312037Sandreas.sandberg@arm.compybind11_add_module 5412037Sandreas.sandberg@arm.com------------------- 5511986Sandreas.sandberg@arm.com 5612037Sandreas.sandberg@arm.comTo ease the creation of Python extension modules, pybind11 provides a CMake 5712037Sandreas.sandberg@arm.comfunction with the following signature: 5812037Sandreas.sandberg@arm.com 5912037Sandreas.sandberg@arm.com.. code-block:: cmake 6012037Sandreas.sandberg@arm.com 6112037Sandreas.sandberg@arm.com pybind11_add_module(<name> [MODULE | SHARED] [EXCLUDE_FROM_ALL] 6214299Sbbruce@ucdavis.edu [NO_EXTRAS] [SYSTEM] [THIN_LTO] source1 [source2 ...]) 6312037Sandreas.sandberg@arm.com 6412037Sandreas.sandberg@arm.comThis function behaves very much like CMake's builtin ``add_library`` (in fact, 6512037Sandreas.sandberg@arm.comit's a wrapper function around that command). It will add a library target 6612037Sandreas.sandberg@arm.comcalled ``<name>`` to be built from the listed source files. In addition, it 6712037Sandreas.sandberg@arm.comwill take care of all the Python-specific compiler and linker flags as well 6812037Sandreas.sandberg@arm.comas the OS- and Python-version-specific file extension. The produced target 6912037Sandreas.sandberg@arm.com``<name>`` can be further manipulated with regular CMake commands. 7012037Sandreas.sandberg@arm.com 7112037Sandreas.sandberg@arm.com``MODULE`` or ``SHARED`` may be given to specify the type of library. If no 7212037Sandreas.sandberg@arm.comtype is given, ``MODULE`` is used by default which ensures the creation of a 7312037Sandreas.sandberg@arm.comPython-exclusive module. Specifying ``SHARED`` will create a more traditional 7412037Sandreas.sandberg@arm.comdynamic library which can also be linked from elsewhere. ``EXCLUDE_FROM_ALL`` 7512037Sandreas.sandberg@arm.comremoves this target from the default build (see CMake docs for details). 7612037Sandreas.sandberg@arm.com 7712037Sandreas.sandberg@arm.comSince pybind11 is a template library, ``pybind11_add_module`` adds compiler 7812037Sandreas.sandberg@arm.comflags to ensure high quality code generation without bloat arising from long 7912391Sjason@lowepower.comsymbol names and duplication of code in different translation units. It 8012391Sjason@lowepower.comsets default visibility to *hidden*, which is required for some pybind11 8112391Sjason@lowepower.comfeatures and functionality when attempting to load multiple pybind11 modules 8212391Sjason@lowepower.comcompiled under different pybind11 versions. It also adds additional flags 8312391Sjason@lowepower.comenabling LTO (Link Time Optimization) and strip unneeded symbols. See the 8412391Sjason@lowepower.com:ref:`FAQ entry <faq:symhidden>` for a more detailed explanation. These 8512391Sjason@lowepower.comlatter optimizations are never applied in ``Debug`` mode. If ``NO_EXTRAS`` is 8612391Sjason@lowepower.comgiven, they will always be disabled, even in ``Release`` mode. However, this 8712391Sjason@lowepower.comwill result in code bloat and is generally not recommended. 8812037Sandreas.sandberg@arm.com 8914299Sbbruce@ucdavis.eduBy default, pybind11 and Python headers will be included with ``-I``. In order 9014299Sbbruce@ucdavis.eduto include pybind11 as system library, e.g. to avoid warnings in downstream 9114299Sbbruce@ucdavis.educode with warn-levels outside of pybind11's scope, set the option ``SYSTEM``. 9214299Sbbruce@ucdavis.edu 9312037Sandreas.sandberg@arm.comAs stated above, LTO is enabled by default. Some newer compilers also support 9412037Sandreas.sandberg@arm.comdifferent flavors of LTO such as `ThinLTO`_. Setting ``THIN_LTO`` will cause 9512037Sandreas.sandberg@arm.comthe function to prefer this flavor if available. The function falls back to 9612037Sandreas.sandberg@arm.comregular LTO if ``-flto=thin`` is not available. 9712037Sandreas.sandberg@arm.com 9812037Sandreas.sandberg@arm.com.. _ThinLTO: http://clang.llvm.org/docs/ThinLTO.html 9912037Sandreas.sandberg@arm.com 10012037Sandreas.sandberg@arm.comConfiguration variables 10112037Sandreas.sandberg@arm.com----------------------- 10212037Sandreas.sandberg@arm.com 10312391Sjason@lowepower.comBy default, pybind11 will compile modules with the C++14 standard, if available 10412391Sjason@lowepower.comon the target compiler, falling back to C++11 if C++14 support is not 10512391Sjason@lowepower.comavailable. Note, however, that this default is subject to change: future 10612391Sjason@lowepower.compybind11 releases are expected to migrate to newer C++ standards as they become 10712391Sjason@lowepower.comavailable. To override this, the standard flag can be given explicitly in 10812391Sjason@lowepower.com``PYBIND11_CPP_STANDARD``: 10912037Sandreas.sandberg@arm.com 11012037Sandreas.sandberg@arm.com.. code-block:: cmake 11112037Sandreas.sandberg@arm.com 11212391Sjason@lowepower.com # Use just one of these: 11312391Sjason@lowepower.com # GCC/clang: 11412037Sandreas.sandberg@arm.com set(PYBIND11_CPP_STANDARD -std=c++11) 11512391Sjason@lowepower.com set(PYBIND11_CPP_STANDARD -std=c++14) 11612391Sjason@lowepower.com set(PYBIND11_CPP_STANDARD -std=c++1z) # Experimental C++17 support 11712391Sjason@lowepower.com # MSVC: 11812391Sjason@lowepower.com set(PYBIND11_CPP_STANDARD /std:c++14) 11912391Sjason@lowepower.com set(PYBIND11_CPP_STANDARD /std:c++latest) # Enables some MSVC C++17 features 12012391Sjason@lowepower.com 12112037Sandreas.sandberg@arm.com add_subdirectory(pybind11) # or find_package(pybind11) 12212037Sandreas.sandberg@arm.com 12312037Sandreas.sandberg@arm.comNote that this and all other configuration variables must be set **before** the 12412391Sjason@lowepower.comcall to ``add_subdirectory`` or ``find_package``. The variables can also be set 12512037Sandreas.sandberg@arm.comwhen calling CMake from the command line using the ``-D<variable>=<value>`` flag. 12612037Sandreas.sandberg@arm.com 12712037Sandreas.sandberg@arm.comThe target Python version can be selected by setting ``PYBIND11_PYTHON_VERSION`` 12812037Sandreas.sandberg@arm.comor an exact Python installation can be specified with ``PYTHON_EXECUTABLE``. 12912037Sandreas.sandberg@arm.comFor example: 13012037Sandreas.sandberg@arm.com 13112037Sandreas.sandberg@arm.com.. code-block:: bash 13212037Sandreas.sandberg@arm.com 13312037Sandreas.sandberg@arm.com cmake -DPYBIND11_PYTHON_VERSION=3.6 .. 13412037Sandreas.sandberg@arm.com # or 13512037Sandreas.sandberg@arm.com cmake -DPYTHON_EXECUTABLE=path/to/python .. 13612037Sandreas.sandberg@arm.com 13712037Sandreas.sandberg@arm.comfind_package vs. add_subdirectory 13812037Sandreas.sandberg@arm.com--------------------------------- 13912037Sandreas.sandberg@arm.com 14012037Sandreas.sandberg@arm.comFor CMake-based projects that don't include the pybind11 repository internally, 14112037Sandreas.sandberg@arm.coman external installation can be detected through ``find_package(pybind11)``. 14212037Sandreas.sandberg@arm.comSee the `Config file`_ docstring for details of relevant CMake variables. 14311986Sandreas.sandberg@arm.com 14411986Sandreas.sandberg@arm.com.. code-block:: cmake 14511986Sandreas.sandberg@arm.com 14611986Sandreas.sandberg@arm.com cmake_minimum_required(VERSION 2.8.12) 14711986Sandreas.sandberg@arm.com project(example) 14811986Sandreas.sandberg@arm.com 14911986Sandreas.sandberg@arm.com find_package(pybind11 REQUIRED) 15011986Sandreas.sandberg@arm.com pybind11_add_module(example example.cpp) 15111986Sandreas.sandberg@arm.com 15214299Sbbruce@ucdavis.eduNote that ``find_package(pybind11)`` will only work correctly if pybind11 15314299Sbbruce@ucdavis.eduhas been correctly installed on the system, e. g. after downloading or cloning 15414299Sbbruce@ucdavis.eduthe pybind11 repository : 15514299Sbbruce@ucdavis.edu 15614299Sbbruce@ucdavis.edu.. code-block:: bash 15714299Sbbruce@ucdavis.edu 15814299Sbbruce@ucdavis.edu cd pybind11 15914299Sbbruce@ucdavis.edu mkdir build 16014299Sbbruce@ucdavis.edu cd build 16114299Sbbruce@ucdavis.edu cmake .. 16214299Sbbruce@ucdavis.edu make install 16314299Sbbruce@ucdavis.edu 16412037Sandreas.sandberg@arm.comOnce detected, the aforementioned ``pybind11_add_module`` can be employed as 16512037Sandreas.sandberg@arm.combefore. The function usage and configuration variables are identical no matter 16612037Sandreas.sandberg@arm.comif pybind11 is added as a subdirectory or found as an installed package. You 16712037Sandreas.sandberg@arm.comcan refer to the same [cmake_example]_ repository for a full sample project 16812037Sandreas.sandberg@arm.com-- just swap out ``add_subdirectory`` for ``find_package``. 16911986Sandreas.sandberg@arm.com 17012037Sandreas.sandberg@arm.com.. _Config file: https://github.com/pybind/pybind11/blob/master/tools/pybind11Config.cmake.in 17112037Sandreas.sandberg@arm.com 17212037Sandreas.sandberg@arm.comAdvanced: interface library target 17312037Sandreas.sandberg@arm.com---------------------------------- 17412037Sandreas.sandberg@arm.com 17512037Sandreas.sandberg@arm.comWhen using a version of CMake greater than 3.0, pybind11 can additionally 17612037Sandreas.sandberg@arm.combe used as a special *interface library* . The target ``pybind11::module`` 17712037Sandreas.sandberg@arm.comis available with pybind11 headers, Python headers and libraries as needed, 17812037Sandreas.sandberg@arm.comand C++ compile definitions attached. This target is suitable for linking 17912037Sandreas.sandberg@arm.comto an independently constructed (through ``add_library``, not 18012037Sandreas.sandberg@arm.com``pybind11_add_module``) target in the consuming project. 18111986Sandreas.sandberg@arm.com 18211986Sandreas.sandberg@arm.com.. code-block:: cmake 18311986Sandreas.sandberg@arm.com 18411986Sandreas.sandberg@arm.com cmake_minimum_required(VERSION 3.0) 18511986Sandreas.sandberg@arm.com project(example) 18611986Sandreas.sandberg@arm.com 18712037Sandreas.sandberg@arm.com find_package(pybind11 REQUIRED) # or add_subdirectory(pybind11) 18812037Sandreas.sandberg@arm.com 18911986Sandreas.sandberg@arm.com add_library(example MODULE main.cpp) 19012037Sandreas.sandberg@arm.com target_link_libraries(example PRIVATE pybind11::module) 19111986Sandreas.sandberg@arm.com set_target_properties(example PROPERTIES PREFIX "${PYTHON_MODULE_PREFIX}" 19211986Sandreas.sandberg@arm.com SUFFIX "${PYTHON_MODULE_EXTENSION}") 19311986Sandreas.sandberg@arm.com 19411986Sandreas.sandberg@arm.com.. warning:: 19511986Sandreas.sandberg@arm.com 19611986Sandreas.sandberg@arm.com Since pybind11 is a metatemplate library, it is crucial that certain 19711986Sandreas.sandberg@arm.com compiler flags are provided to ensure high quality code generation. In 19811986Sandreas.sandberg@arm.com contrast to the ``pybind11_add_module()`` command, the CMake interface 19911986Sandreas.sandberg@arm.com library only provides the *minimal* set of parameters to ensure that the 20011986Sandreas.sandberg@arm.com code using pybind11 compiles, but it does **not** pass these extra compiler 20111986Sandreas.sandberg@arm.com flags (i.e. this is up to you). 20211986Sandreas.sandberg@arm.com 20311986Sandreas.sandberg@arm.com These include Link Time Optimization (``-flto`` on GCC/Clang/ICPC, ``/GL`` 20412391Sjason@lowepower.com and ``/LTCG`` on Visual Studio) and .OBJ files with many sections on Visual 20512391Sjason@lowepower.com Studio (``/bigobj``). The :ref:`FAQ <faq:symhidden>` contains an 20611986Sandreas.sandberg@arm.com explanation on why these are needed. 20711986Sandreas.sandberg@arm.com 20812391Sjason@lowepower.comEmbedding the Python interpreter 20912391Sjason@lowepower.com-------------------------------- 21012391Sjason@lowepower.com 21112391Sjason@lowepower.comIn addition to extension modules, pybind11 also supports embedding Python into 21212391Sjason@lowepower.coma C++ executable or library. In CMake, simply link with the ``pybind11::embed`` 21312391Sjason@lowepower.comtarget. It provides everything needed to get the interpreter running. The Python 21412391Sjason@lowepower.comheaders and libraries are attached to the target. Unlike ``pybind11::module``, 21512391Sjason@lowepower.comthere is no need to manually set any additional properties here. For more 21612391Sjason@lowepower.cominformation about usage in C++, see :doc:`/advanced/embedding`. 21712391Sjason@lowepower.com 21812391Sjason@lowepower.com.. code-block:: cmake 21912391Sjason@lowepower.com 22012391Sjason@lowepower.com cmake_minimum_required(VERSION 3.0) 22112391Sjason@lowepower.com project(example) 22212391Sjason@lowepower.com 22312391Sjason@lowepower.com find_package(pybind11 REQUIRED) # or add_subdirectory(pybind11) 22412391Sjason@lowepower.com 22512391Sjason@lowepower.com add_executable(example main.cpp) 22612391Sjason@lowepower.com target_link_libraries(example PRIVATE pybind11::embed) 22712391Sjason@lowepower.com 22812391Sjason@lowepower.com.. _building_manually: 22912391Sjason@lowepower.com 23012391Sjason@lowepower.comBuilding manually 23112391Sjason@lowepower.com================= 23212391Sjason@lowepower.com 23312391Sjason@lowepower.compybind11 is a header-only library, hence it is not necessary to link against 23412391Sjason@lowepower.comany special libraries and there are no intermediate (magic) translation steps. 23512391Sjason@lowepower.com 23612391Sjason@lowepower.comOn Linux, you can compile an example such as the one given in 23712391Sjason@lowepower.com:ref:`simple_example` using the following command: 23812391Sjason@lowepower.com 23912391Sjason@lowepower.com.. code-block:: bash 24012391Sjason@lowepower.com 24112391Sjason@lowepower.com $ c++ -O3 -Wall -shared -std=c++11 -fPIC `python3 -m pybind11 --includes` example.cpp -o example`python3-config --extension-suffix` 24212391Sjason@lowepower.com 24312391Sjason@lowepower.comThe flags given here assume that you're using Python 3. For Python 2, just 24412391Sjason@lowepower.comchange the executable appropriately (to ``python`` or ``python2``). 24512391Sjason@lowepower.com 24612391Sjason@lowepower.comThe ``python3 -m pybind11 --includes`` command fetches the include paths for 24712391Sjason@lowepower.comboth pybind11 and Python headers. This assumes that pybind11 has been installed 24812391Sjason@lowepower.comusing ``pip`` or ``conda``. If it hasn't, you can also manually specify 24912391Sjason@lowepower.com``-I <path-to-pybind11>/include`` together with the Python includes path 25012391Sjason@lowepower.com``python3-config --includes``. 25112391Sjason@lowepower.com 25212391Sjason@lowepower.comNote that Python 2.7 modules don't use a special suffix, so you should simply 25312391Sjason@lowepower.comuse ``example.so`` instead of ``example`python3-config --extension-suffix```. 25412391Sjason@lowepower.comBesides, the ``--extension-suffix`` option may or may not be available, depending 25512391Sjason@lowepower.comon the distribution; in the latter case, the module extension can be manually 25612391Sjason@lowepower.comset to ``.so``. 25712391Sjason@lowepower.com 25812391Sjason@lowepower.comOn Mac OS: the build command is almost the same but it also requires passing 25912391Sjason@lowepower.comthe ``-undefined dynamic_lookup`` flag so as to ignore missing symbols when 26012391Sjason@lowepower.combuilding the module: 26112391Sjason@lowepower.com 26212391Sjason@lowepower.com.. code-block:: bash 26312391Sjason@lowepower.com 26412391Sjason@lowepower.com $ c++ -O3 -Wall -shared -std=c++11 -undefined dynamic_lookup `python3 -m pybind11 --includes` example.cpp -o example`python3-config --extension-suffix` 26512391Sjason@lowepower.com 26612391Sjason@lowepower.comIn general, it is advisable to include several additional build parameters 26712391Sjason@lowepower.comthat can considerably reduce the size of the created binary. Refer to section 26812391Sjason@lowepower.com:ref:`cmake` for a detailed example of a suitable cross-platform CMake-based 26912391Sjason@lowepower.combuild system that works on all platforms including Windows. 27012391Sjason@lowepower.com 27112391Sjason@lowepower.com.. note:: 27212391Sjason@lowepower.com 27312391Sjason@lowepower.com On Linux and macOS, it's better to (intentionally) not link against 27412391Sjason@lowepower.com ``libpython``. The symbols will be resolved when the extension library 27512391Sjason@lowepower.com is loaded into a Python binary. This is preferable because you might 27612391Sjason@lowepower.com have several different installations of a given Python version (e.g. the 27712391Sjason@lowepower.com system-provided Python, and one that ships with a piece of commercial 27812391Sjason@lowepower.com software). In this way, the plugin will work with both versions, instead 27912391Sjason@lowepower.com of possibly importing a second Python library into a process that already 28012391Sjason@lowepower.com contains one (which will lead to a segfault). 28112391Sjason@lowepower.com 28212037Sandreas.sandberg@arm.comGenerating binding code automatically 28312037Sandreas.sandberg@arm.com===================================== 28411986Sandreas.sandberg@arm.com 28512037Sandreas.sandberg@arm.comThe ``Binder`` project is a tool for automatic generation of pybind11 binding 28612037Sandreas.sandberg@arm.comcode by introspecting existing C++ codebases using LLVM/Clang. See the 28712037Sandreas.sandberg@arm.com[binder]_ documentation for details. 28812037Sandreas.sandberg@arm.com 28912037Sandreas.sandberg@arm.com.. [binder] http://cppbinder.readthedocs.io/en/latest/about.html 290