1STL containers
2##############
3
4Automatic conversion
5====================
6
7When including the additional header file :file:`pybind11/stl.h`, conversions
8between ``std::vector<>``/``std::deque<>``/``std::list<>``/``std::array<>``,
9``std::set<>``/``std::unordered_set<>``, and
10``std::map<>``/``std::unordered_map<>`` and the Python ``list``, ``set`` and
11``dict`` data structures are automatically enabled. The types ``std::pair<>``
12and ``std::tuple<>`` are already supported out of the box with just the core
13:file:`pybind11/pybind11.h` header.
14
15The major downside of these implicit conversions is that containers must be
16converted (i.e. copied) on every Python->C++ and C++->Python transition, which
17can have implications on the program semantics and performance. Please read the
18next sections for more details and alternative approaches that avoid this.
19
20.. note::
21
22    Arbitrary nesting of any of these types is possible.
23
24.. seealso::
25
26    The file :file:`tests/test_stl.cpp` contains a complete
27    example that demonstrates how to pass STL data types in more detail.
28
29.. _cpp17_container_casters:
30
31C++17 library containers
32========================
33
34The :file:`pybind11/stl.h` header also includes support for ``std::optional<>``
35and ``std::variant<>``. These require a C++17 compiler and standard library.
36In C++14 mode, ``std::experimental::optional<>`` is supported if available.
37
38Various versions of these containers also exist for C++11 (e.g. in Boost).
39pybind11 provides an easy way to specialize the ``type_caster`` for such
40types:
41
42.. code-block:: cpp
43
44    // `boost::optional` as an example -- can be any `std::optional`-like container
45    namespace pybind11 { namespace detail {
46        template <typename T>
47        struct type_caster<boost::optional<T>> : optional_caster<boost::optional<T>> {};
48    }}
49
50The above should be placed in a header file and included in all translation units
51where automatic conversion is needed. Similarly, a specialization can be provided
52for custom variant types:
53
54.. code-block:: cpp
55
56    // `boost::variant` as an example -- can be any `std::variant`-like container
57    namespace pybind11 { namespace detail {
58        template <typename... Ts>
59        struct type_caster<boost::variant<Ts...>> : variant_caster<boost::variant<Ts...>> {};
60
61        // Specifies the function used to visit the variant -- `apply_visitor` instead of `visit`
62        template <>
63        struct visit_helper<boost::variant> {
64            template <typename... Args>
65            static auto call(Args &&...args) -> decltype(boost::apply_visitor(args...)) {
66                return boost::apply_visitor(args...);
67            }
68        };
69    }} // namespace pybind11::detail
70
71The ``visit_helper`` specialization is not required if your ``name::variant`` provides
72a ``name::visit()`` function. For any other function name, the specialization must be
73included to tell pybind11 how to visit the variant.
74
75.. note::
76
77    pybind11 only supports the modern implementation of ``boost::variant``
78    which makes use of variadic templates. This requires Boost 1.56 or newer.
79    Additionally, on Windows, MSVC 2017 is required because ``boost::variant``
80    falls back to the old non-variadic implementation on MSVC 2015.
81
82.. _opaque:
83
84Making opaque types
85===================
86
87pybind11 heavily relies on a template matching mechanism to convert parameters
88and return values that are constructed from STL data types such as vectors,
89linked lists, hash tables, etc. This even works in a recursive manner, for
90instance to deal with lists of hash maps of pairs of elementary and custom
91types, etc.
92
93However, a fundamental limitation of this approach is that internal conversions
94between Python and C++ types involve a copy operation that prevents
95pass-by-reference semantics. What does this mean?
96
97Suppose we bind the following function
98
99.. code-block:: cpp
100
101    void append_1(std::vector<int> &v) {
102       v.push_back(1);
103    }
104
105and call it from Python, the following happens:
106
107.. code-block:: pycon
108
109   >>> v = [5, 6]
110   >>> append_1(v)
111   >>> print(v)
112   [5, 6]
113
114As you can see, when passing STL data structures by reference, modifications
115are not propagated back the Python side. A similar situation arises when
116exposing STL data structures using the ``def_readwrite`` or ``def_readonly``
117functions:
118
119.. code-block:: cpp
120
121    /* ... definition ... */
122
123    class MyClass {
124        std::vector<int> contents;
125    };
126
127    /* ... binding code ... */
128
129    py::class_<MyClass>(m, "MyClass")
130        .def(py::init<>())
131        .def_readwrite("contents", &MyClass::contents);
132
133In this case, properties can be read and written in their entirety. However, an
134``append`` operation involving such a list type has no effect:
135
136.. code-block:: pycon
137
138   >>> m = MyClass()
139   >>> m.contents = [5, 6]
140   >>> print(m.contents)
141   [5, 6]
142   >>> m.contents.append(7)
143   >>> print(m.contents)
144   [5, 6]
145
146Finally, the involved copy operations can be costly when dealing with very
147large lists. To deal with all of the above situations, pybind11 provides a
148macro named ``PYBIND11_MAKE_OPAQUE(T)`` that disables the template-based
149conversion machinery of types, thus rendering them *opaque*. The contents of
150opaque objects are never inspected or extracted, hence they *can* be passed by
151reference. For instance, to turn ``std::vector<int>`` into an opaque type, add
152the declaration
153
154.. code-block:: cpp
155
156    PYBIND11_MAKE_OPAQUE(std::vector<int>);
157
158before any binding code (e.g. invocations to ``class_::def()``, etc.). This
159macro must be specified at the top level (and outside of any namespaces), since
160it instantiates a partial template overload. If your binding code consists of
161multiple compilation units, it must be present in every file (typically via a
162common header) preceding any usage of ``std::vector<int>``. Opaque types must
163also have a corresponding ``class_`` declaration to associate them with a name
164in Python, and to define a set of available operations, e.g.:
165
166.. code-block:: cpp
167
168    py::class_<std::vector<int>>(m, "IntVector")
169        .def(py::init<>())
170        .def("clear", &std::vector<int>::clear)
171        .def("pop_back", &std::vector<int>::pop_back)
172        .def("__len__", [](const std::vector<int> &v) { return v.size(); })
173        .def("__iter__", [](std::vector<int> &v) {
174           return py::make_iterator(v.begin(), v.end());
175        }, py::keep_alive<0, 1>()) /* Keep vector alive while iterator is used */
176        // ....
177
178.. seealso::
179
180    The file :file:`tests/test_opaque_types.cpp` contains a complete
181    example that demonstrates how to create and expose opaque types using
182    pybind11 in more detail.
183
184.. _stl_bind:
185
186Binding STL containers
187======================
188
189The ability to expose STL containers as native Python objects is a fairly
190common request, hence pybind11 also provides an optional header file named
191:file:`pybind11/stl_bind.h` that does exactly this. The mapped containers try
192to match the behavior of their native Python counterparts as much as possible.
193
194The following example showcases usage of :file:`pybind11/stl_bind.h`:
195
196.. code-block:: cpp
197
198    // Don't forget this
199    #include <pybind11/stl_bind.h>
200
201    PYBIND11_MAKE_OPAQUE(std::vector<int>);
202    PYBIND11_MAKE_OPAQUE(std::map<std::string, double>);
203
204    // ...
205
206    // later in binding code:
207    py::bind_vector<std::vector<int>>(m, "VectorInt");
208    py::bind_map<std::map<std::string, double>>(m, "MapStringDouble");
209
210When binding STL containers pybind11 considers the types of the container's
211elements to decide whether the container should be confined to the local module
212(via the :ref:`module_local` feature).  If the container element types are
213anything other than already-bound custom types bound without
214``py::module_local()`` the container binding will have ``py::module_local()``
215applied.  This includes converting types such as numeric types, strings, Eigen
216types; and types that have not yet been bound at the time of the stl container
217binding.  This module-local binding is designed to avoid potential conflicts
218between module bindings (for example, from two separate modules each attempting
219to bind ``std::vector<int>`` as a python type).
220
221It is possible to override this behavior to force a definition to be either
222module-local or global.  To do so, you can pass the attributes
223``py::module_local()`` (to make the binding module-local) or
224``py::module_local(false)`` (to make the binding global) into the
225``py::bind_vector`` or ``py::bind_map`` arguments:
226
227.. code-block:: cpp
228
229    py::bind_vector<std::vector<int>>(m, "VectorInt", py::module_local(false));
230
231Note, however, that such a global binding would make it impossible to load this
232module at the same time as any other pybind module that also attempts to bind
233the same container type (``std::vector<int>`` in the above example).
234
235See :ref:`module_local` for more details on module-local bindings.
236
237.. seealso::
238
239    The file :file:`tests/test_stl_binders.cpp` shows how to use the
240    convenience STL container wrappers.
241