numpy.rst (11986:c12e4625ab56) numpy.rst (12037:d28054ac6ec9)
1.. _numpy:
2
3NumPy
4#####
5
6Buffer protocol
7===============
8

--- 19 unchanged lines hidden (view full) ---

28
29The following binding code exposes the ``Matrix`` contents as a buffer object,
30making it possible to cast Matrices into NumPy arrays. It is even possible to
31completely avoid copy operations with Python expressions like
32``np.array(matrix_instance, copy = False)``.
33
34.. code-block:: cpp
35
1.. _numpy:
2
3NumPy
4#####
5
6Buffer protocol
7===============
8

--- 19 unchanged lines hidden (view full) ---

28
29The following binding code exposes the ``Matrix`` contents as a buffer object,
30making it possible to cast Matrices into NumPy arrays. It is even possible to
31completely avoid copy operations with Python expressions like
32``np.array(matrix_instance, copy = False)``.
33
34.. code-block:: cpp
35
36 py::class_(m, "Matrix")
36 py::class_<Matrix>(m, "Matrix", py::buffer_protocol())
37 .def_buffer([](Matrix &m) -> py::buffer_info {
38 return py::buffer_info(
39 m.data(), /* Pointer to buffer */
40 sizeof(float), /* Size of one scalar */
41 py::format_descriptor<float>::format(), /* Python struct-style format descriptor */
42 2, /* Number of dimensions */
43 { m.rows(), m.cols() }, /* Buffer dimensions */
44 { sizeof(float) * m.rows(), /* Strides (in bytes) for each index */
45 sizeof(float) }
46 );
47 });
48
37 .def_buffer([](Matrix &m) -> py::buffer_info {
38 return py::buffer_info(
39 m.data(), /* Pointer to buffer */
40 sizeof(float), /* Size of one scalar */
41 py::format_descriptor<float>::format(), /* Python struct-style format descriptor */
42 2, /* Number of dimensions */
43 { m.rows(), m.cols() }, /* Buffer dimensions */
44 { sizeof(float) * m.rows(), /* Strides (in bytes) for each index */
45 sizeof(float) }
46 );
47 });
48
49The snippet above binds a lambda function, which can create ``py::buffer_info``
50description records on demand describing a given matrix. The contents of
51``py::buffer_info`` mirror the Python buffer protocol specification.
49Supporting the buffer protocol in a new type involves specifying the special
50``py::buffer_protocol()`` tag in the ``py::class_`` constructor and calling the
51``def_buffer()`` method with a lambda function that creates a
52``py::buffer_info`` description record on demand describing a given matrix
53instance. The contents of ``py::buffer_info`` mirror the Python buffer protocol
54specification.
52
53.. code-block:: cpp
54
55 struct buffer_info {
56 void *ptr;
57 size_t itemsize;
58 std::string format;
59 int ndim;

--- 12 unchanged lines hidden (view full) ---

72.. code-block:: cpp
73
74 /* Bind MatrixXd (or some other Eigen type) to Python */
75 typedef Eigen::MatrixXd Matrix;
76
77 typedef Matrix::Scalar Scalar;
78 constexpr bool rowMajor = Matrix::Flags & Eigen::RowMajorBit;
79
55
56.. code-block:: cpp
57
58 struct buffer_info {
59 void *ptr;
60 size_t itemsize;
61 std::string format;
62 int ndim;

--- 12 unchanged lines hidden (view full) ---

75.. code-block:: cpp
76
77 /* Bind MatrixXd (or some other Eigen type) to Python */
78 typedef Eigen::MatrixXd Matrix;
79
80 typedef Matrix::Scalar Scalar;
81 constexpr bool rowMajor = Matrix::Flags & Eigen::RowMajorBit;
82
80 py::class_(m, "Matrix")
83 py::class_<Matrix>(m, "Matrix", py::buffer_protocol())
81 .def("__init__", [](Matrix &m, py::buffer b) {
82 typedef Eigen::Stride<Eigen::Dynamic, Eigen::Dynamic> Strides;
83
84 /* Request a buffer descriptor from Python */
85 py::buffer_info info = b.request();
86
87 /* Some sanity checks ... */
88 if (info.format != py::format_descriptor<Scalar>::format())

--- 58 unchanged lines hidden (view full) ---

147
148.. code-block:: cpp
149
150 void f(py::array_t<double> array);
151
152When it is invoked with a different type (e.g. an integer or a list of
153integers), the binding code will attempt to cast the input into a NumPy array
154of the requested type. Note that this feature requires the
84 .def("__init__", [](Matrix &m, py::buffer b) {
85 typedef Eigen::Stride<Eigen::Dynamic, Eigen::Dynamic> Strides;
86
87 /* Request a buffer descriptor from Python */
88 py::buffer_info info = b.request();
89
90 /* Some sanity checks ... */
91 if (info.format != py::format_descriptor<Scalar>::format())

--- 58 unchanged lines hidden (view full) ---

150
151.. code-block:: cpp
152
153 void f(py::array_t<double> array);
154
155When it is invoked with a different type (e.g. an integer or a list of
156integers), the binding code will attempt to cast the input into a NumPy array
157of the requested type. Note that this feature requires the
155:file:``pybind11/numpy.h`` header to be included.
158:file:`pybind11/numpy.h` header to be included.
156
157Data in NumPy arrays is not guaranteed to packed in a dense manner;
158furthermore, entries can be separated by arbitrary column and row strides.
159Sometimes, it can be useful to require a function to only accept dense arrays
160using either the C (row-major) or Fortran (column-major) ordering. This can be
161accomplished via a second template argument with values ``py::array::c_style``
162or ``py::array::f_style``.
163

--- 4 unchanged lines hidden (view full) ---

168The ``py::array::forcecast`` argument is the default value of the second
169template parameter, and it ensures that non-conforming arguments are converted
170into an array satisfying the specified requirements instead of trying the next
171function overload.
172
173Structured types
174================
175
159
160Data in NumPy arrays is not guaranteed to packed in a dense manner;
161furthermore, entries can be separated by arbitrary column and row strides.
162Sometimes, it can be useful to require a function to only accept dense arrays
163using either the C (row-major) or Fortran (column-major) ordering. This can be
164accomplished via a second template argument with values ``py::array::c_style``
165or ``py::array::f_style``.
166

--- 4 unchanged lines hidden (view full) ---

171The ``py::array::forcecast`` argument is the default value of the second
172template parameter, and it ensures that non-conforming arguments are converted
173into an array satisfying the specified requirements instead of trying the next
174function overload.
175
176Structured types
177================
178
176In order for ``py::array_t`` to work with structured (record) types, we first need
177to register the memory layout of the type. This can be done via ``PYBIND11_NUMPY_DTYPE``
178macro which expects the type followed by field names:
179In order for ``py::array_t`` to work with structured (record) types, we first
180need to register the memory layout of the type. This can be done via
181``PYBIND11_NUMPY_DTYPE`` macro, called in the plugin definition code, which
182expects the type followed by field names:
179
180.. code-block:: cpp
181
182 struct A {
183 int x;
184 double y;
185 };
186
187 struct B {
188 int z;
189 A a;
190 };
191
183
184.. code-block:: cpp
185
186 struct A {
187 int x;
188 double y;
189 };
190
191 struct B {
192 int z;
193 A a;
194 };
195
192 PYBIND11_NUMPY_DTYPE(A, x, y);
193 PYBIND11_NUMPY_DTYPE(B, z, a);
196 // ...
197 PYBIND11_PLUGIN(test) {
198 // ...
194
199
195 /* now both A and B can be used as template arguments to py::array_t */
200 PYBIND11_NUMPY_DTYPE(A, x, y);
201 PYBIND11_NUMPY_DTYPE(B, z, a);
202 /* now both A and B can be used as template arguments to py::array_t */
203 }
196
197Vectorizing functions
198=====================
199
200Suppose we want to bind a function with the following signature to Python so
201that it can process arbitrary NumPy array arguments (vectors, matrices, general
202N-D arrays) in addition to its normal arguments:
203

--- 88 unchanged lines hidden (view full) ---

292 m.def("add_arrays", &add_arrays, "Add two NumPy arrays");
293 return m.ptr();
294 }
295
296.. seealso::
297
298 The file :file:`tests/test_numpy_vectorize.cpp` contains a complete
299 example that demonstrates using :func:`vectorize` in more detail.
204
205Vectorizing functions
206=====================
207
208Suppose we want to bind a function with the following signature to Python so
209that it can process arbitrary NumPy array arguments (vectors, matrices, general
210N-D arrays) in addition to its normal arguments:
211

--- 88 unchanged lines hidden (view full) ---

300 m.def("add_arrays", &add_arrays, "Add two NumPy arrays");
301 return m.ptr();
302 }
303
304.. seealso::
305
306 The file :file:`tests/test_numpy_vectorize.cpp` contains a complete
307 example that demonstrates using :func:`vectorize` in more detail.
308
309Direct access
310=============
311
312For performance reasons, particularly when dealing with very large arrays, it
313is often desirable to directly access array elements without internal checking
314of dimensions and bounds on every access when indices are known to be already
315valid. To avoid such checks, the ``array`` class and ``array_t<T>`` template
316class offer an unchecked proxy object that can be used for this unchecked
317access through the ``unchecked<N>`` and ``mutable_unchecked<N>`` methods,
318where ``N`` gives the required dimensionality of the array:
319
320.. code-block:: cpp
321
322 m.def("sum_3d", [](py::array_t<double> x) {
323 auto r = x.unchecked<3>(); // x must have ndim = 3; can be non-writeable
324 double sum = 0;
325 for (size_t i = 0; i < r.shape(0); i++)
326 for (size_t j = 0; j < r.shape(1); j++)
327 for (size_t k = 0; k < r.shape(2); k++)
328 sum += r(i, j, k);
329 return sum;
330 });
331 m.def("increment_3d", [](py::array_t<double> x) {
332 auto r = x.mutable_unchecked<3>(); // Will throw if ndim != 3 or flags.writeable is false
333 for (size_t i = 0; i < r.shape(0); i++)
334 for (size_t j = 0; j < r.shape(1); j++)
335 for (size_t k = 0; k < r.shape(2); k++)
336 r(i, j, k) += 1.0;
337 }, py::arg().noconvert());
338
339To obtain the proxy from an ``array`` object, you must specify both the data
340type and number of dimensions as template arguments, such as ``auto r =
341myarray.mutable_unchecked<float, 2>()``.
342
343If the number of dimensions is not known at compile time, you can omit the
344dimensions template parameter (i.e. calling ``arr_t.unchecked()`` or
345``arr.unchecked<T>()``. This will give you a proxy object that works in the
346same way, but results in less optimizable code and thus a small efficiency
347loss in tight loops.
348
349Note that the returned proxy object directly references the array's data, and
350only reads its shape, strides, and writeable flag when constructed. You must
351take care to ensure that the referenced array is not destroyed or reshaped for
352the duration of the returned object, typically by limiting the scope of the
353returned instance.
354
355The returned proxy object supports some of the same methods as ``py::array`` so
356that it can be used as a drop-in replacement for some existing, index-checked
357uses of ``py::array``:
358
359- ``r.ndim()`` returns the number of dimensions
360
361- ``r.data(1, 2, ...)`` and ``r.mutable_data(1, 2, ...)``` returns a pointer to
362 the ``const T`` or ``T`` data, respectively, at the given indices. The
363 latter is only available to proxies obtained via ``a.mutable_unchecked()``.
364
365- ``itemsize()`` returns the size of an item in bytes, i.e. ``sizeof(T)``.
366
367- ``ndim()`` returns the number of dimensions.
368
369- ``shape(n)`` returns the size of dimension ``n``
370
371- ``size()`` returns the total number of elements (i.e. the product of the shapes).
372
373- ``nbytes()`` returns the number of bytes used by the referenced elements
374 (i.e. ``itemsize()`` times ``size()``).
375
376.. seealso::
377
378 The file :file:`tests/test_numpy_array.cpp` contains additional examples
379 demonstrating the use of this feature.