stl.rst revision 12037:d28054ac6ec9 
1STL containers
2##############
3
4Automatic conversion
5====================
6
7When including the additional header file :file:`pybind11/stl.h`, conversions
8between ``std::vector<>``/``std::list<>``/``std::array<>``,
9``std::set<>``/``std::unordered_set<>``, and
10``std::map<>``/``std::unordered_map<>`` and the Python ``list``, ``set`` and
11``dict`` data structures are automatically enabled. The types ``std::pair<>``
12and ``std::tuple<>`` are already supported out of the box with just the core
13:file:`pybind11/pybind11.h` header.
14
15The major downside of these implicit conversions is that containers must be
16converted (i.e. copied) on every Python->C++ and C++->Python transition, which
17can have implications on the program semantics and performance. Please read the
18next sections for more details and alternative approaches that avoid this.
19
20.. note::
21
22    Arbitrary nesting of any of these types is possible.
23
24.. seealso::
25
26    The file :file:`tests/test_python_types.cpp` contains a complete
27    example that demonstrates how to pass STL data types in more detail.
28
29.. _opaque:
30
31Making opaque types
32===================
33
34pybind11 heavily relies on a template matching mechanism to convert parameters
35and return values that are constructed from STL data types such as vectors,
36linked lists, hash tables, etc. This even works in a recursive manner, for
37instance to deal with lists of hash maps of pairs of elementary and custom
38types, etc.
39
40However, a fundamental limitation of this approach is that internal conversions
41between Python and C++ types involve a copy operation that prevents
42pass-by-reference semantics. What does this mean?
43
44Suppose we bind the following function
45
46.. code-block:: cpp
47
48    void append_1(std::vector<int> &v) {
49       v.push_back(1);
50    }
51
52and call it from Python, the following happens:
53
54.. code-block:: pycon
55
56   >>> v = [5, 6]
57   >>> append_1(v)
58   >>> print(v)
59   [5, 6]
60
61As you can see, when passing STL data structures by reference, modifications
62are not propagated back the Python side. A similar situation arises when
63exposing STL data structures using the ``def_readwrite`` or ``def_readonly``
64functions:
65
66.. code-block:: cpp
67
68    /* ... definition ... */
69
70    class MyClass {
71        std::vector<int> contents;
72    };
73
74    /* ... binding code ... */
75
76    py::class_<MyClass>(m, "MyClass")
77        .def(py::init<>())
78        .def_readwrite("contents", &MyClass::contents);
79
80In this case, properties can be read and written in their entirety. However, an
81``append`` operation involving such a list type has no effect:
82
83.. code-block:: pycon
84
85   >>> m = MyClass()
86   >>> m.contents = [5, 6]
87   >>> print(m.contents)
88   [5, 6]
89   >>> m.contents.append(7)
90   >>> print(m.contents)
91   [5, 6]
92
93Finally, the involved copy operations can be costly when dealing with very
94large lists. To deal with all of the above situations, pybind11 provides a
95macro named ``PYBIND11_MAKE_OPAQUE(T)`` that disables the template-based
96conversion machinery of types, thus rendering them *opaque*. The contents of
97opaque objects are never inspected or extracted, hence they *can* be passed by
98reference. For instance, to turn ``std::vector<int>`` into an opaque type, add
99the declaration
100
101.. code-block:: cpp
102
103    PYBIND11_MAKE_OPAQUE(std::vector<int>);
104
105before any binding code (e.g. invocations to ``class_::def()``, etc.). This
106macro must be specified at the top level (and outside of any namespaces), since
107it instantiates a partial template overload. If your binding code consists of
108multiple compilation units, it must be present in every file preceding any
109usage of ``std::vector<int>``. Opaque types must also have a corresponding
110``class_`` declaration to associate them with a name in Python, and to define a
111set of available operations, e.g.:
112
113.. code-block:: cpp
114
115    py::class_<std::vector<int>>(m, "IntVector")
116        .def(py::init<>())
117        .def("clear", &std::vector<int>::clear)
118        .def("pop_back", &std::vector<int>::pop_back)
119        .def("__len__", [](const std::vector<int> &v) { return v.size(); })
120        .def("__iter__", [](std::vector<int> &v) {
121           return py::make_iterator(v.begin(), v.end());
122        }, py::keep_alive<0, 1>()) /* Keep vector alive while iterator is used */
123        // ....
124
125The ability to expose STL containers as native Python objects is a fairly
126common request, hence pybind11 also provides an optional header file named
127:file:`pybind11/stl_bind.h` that does exactly this. The mapped containers try
128to match the behavior of their native Python counterparts as much as possible.
129
130The following example showcases usage of :file:`pybind11/stl_bind.h`:
131
132.. code-block:: cpp
133
134    // Don't forget this
135    #include <pybind11/stl_bind.h>
136
137    PYBIND11_MAKE_OPAQUE(std::vector<int>);
138    PYBIND11_MAKE_OPAQUE(std::map<std::string, double>);
139
140    // ...
141
142    // later in binding code:
143    py::bind_vector<std::vector<int>>(m, "VectorInt");
144    py::bind_map<std::map<std::string, double>>(m, "MapStringDouble");
145
146Please take a look at the :ref:`macro_notes` before using the
147``PYBIND11_MAKE_OPAQUE`` macro.
148
149.. seealso::
150
151    The file :file:`tests/test_opaque_types.cpp` contains a complete
152    example that demonstrates how to create and expose opaque types using
153    pybind11 in more detail.
154
155    The file :file:`tests/test_stl_binders.cpp` shows how to use the
156    convenience STL container wrappers.
157