(CMake) Examples

Basic example (from readme)

Consider the following C++ code:

main.cpp

#include <numeric>
#include <xtensor.hpp>
#include <pybind11/pybind11.h>
#define FORCE_IMPORT_ARRAY
#include <xtensor-python/pyarray.hpp>

double sum_of_sines(xt::pyarray<double>& m)
{
    auto sines = xt::sin(m);  // sines does not actually hold values.
    return std::accumulate(sines.begin(), sines.end(), 0.0);
}

PYBIND11_MODULE(mymodule, m)
{
    xt::import_numpy();
    m.doc() = "Test module for xtensor python bindings";
    m.def("sum_of_sines", sum_of_sines, "Sum the sines of the input values");
}

There are several options to build the module, whereby we will use CMake here with the following CMakeLists.txt:

CMakeLists.txt

cmake_minimum_required(VERSION 3.18..3.20)

project(mymodule)

find_package(Python REQUIRED COMPONENTS Interpreter Development NumPy)
find_package(pybind11 REQUIRED CONFIG)
find_package(xtensor REQUIRED)
find_package(xtensor-python REQUIRED)

pybind11_add_module(mymodule main.cpp)
target_link_libraries(mymodule PUBLIC pybind11::module xtensor-python Python::NumPy)

target_compile_definitions(mymodule PRIVATE VERSION_INFO=0.1.0)

Tip

There is a potential pitfall here, centered around the fact that CMake has a ‘new’ FindPython and a ‘classic’ FindPythonLibs. We here use FindPython because of its ability to find the NumPy headers, that we need for xtensor-python.

This has the consequence that when we want to force CMake to use a specific Python executable, we have to use something like

cmake -Bbuild -DPython_EXECUTABLE=`which python`

whereby it is crucial that one uses the correct case Python_EXECUTABLE, as:

Python_EXECUTABLE   <->   FindPython
PYTHON_EXECUTABLE   <->   FindPythonLibs

(remember that CMake is case-sensitive!).

Now, since we use FindPython because of xtensor-python we also want pybind11 to use FindPython (and not the classic FindPythonLibs, since we want to specify the Python executable only once). To this end we have to make sure to do things in the correct order, which is

find_package(Python REQUIRED COMPONENTS Interpreter Development NumPy)
find_package(pybind11 REQUIRED CONFIG)

(i.e. one finds Python before pybind11). See the pybind11 documentation.

In addition, be sure to use a quite recent CMake version, by starting your CMakeLists.txt for example with

cmake_minimum_required(VERSION 3.18..3.20)

Then we can test the module:

example.py

import mymodule
import numpy as np

a = np.array([1, 2, 3])
assert np.isclose(np.sum(np.sin(a)), mymodule.sum_of_sines(a))

Note

Since we did not install the module, we should compile and run the example from the same folder. To install, please consult this *pybind11* / *CMake* example.

Type restriction with SFINAE

See also

Medium post by Johan Mabille This example covers “Option 4”.

In this example we will design a module with a function that accepts an xt::xtensor as argument, but in such a way that an xt::pyxtensor can be accepted in the Python module. This is done by having a templated function

template <class T>
void times_dimension(T& t);

As this might be a bit too permissive for your liking, we will show you how to limit the scope to xtensor types, and allow other overloads using the principle of SFINAE (Substitution Failure Is Not An Error). In particular:

mymodule.hpp

#include <xtensor/xtensor.hpp>

namespace mymodule {

template <class T>
struct is_std_vector
{
    static const bool value = false;
};

template <class T>
struct is_std_vector<std::vector<T> >
{
    static const bool value = true;
};

// any xtensor object
template <class T, std::enable_if_t<xt::is_xexpression<T>::value, bool> = true>
void times_dimension(T& t)
{
    using value_type = typename T::value_type;
    t *= (value_type)(t.dimension());
}

// an std::vector
template <class T, std::enable_if_t<is_std_vector<T>::value, bool> = true>
void times_dimension(T& t)
{
    // do nothing
}

}

Consequently from C++, the interaction with the module’s function is trivial

main.cpp

#include "mymodule.hpp"
#include <xtensor/xio.hpp>

int main()
{
    xt::xtensor<size_t, 2> a = xt::arange<size_t>(2 * 3).reshape({2, 3});
    mymodule::times_dimension(a);
    std::cout << a << std::endl;
    return 0;
}

For the Python module we just have to specify the template to be xt::pyarray or xt::pytensor. E.g.

src/python.cpp

#include "mymodule.hpp"
#include <pybind11/pybind11.h>
#define FORCE_IMPORT_ARRAY
#include <xtensor-python/pyarray.hpp>

PYBIND11_MODULE(mymodule, m)
{
    xt::import_numpy();
    m.doc() = "Test module for xtensor python bindings";
    m.def("times_dimension", &mymodule::times_dimension<xt::pyarray<double>>);
}

We will again use CMake to compile, with the following CMakeLists.txt:

CMakeLists.txt

cmake_minimum_required(VERSION 3.18..3.20)

project(mymodule)

find_package(Python REQUIRED COMPONENTS Interpreter Development NumPy)
find_package(pybind11 REQUIRED CONFIG)
find_package(xtensor REQUIRED)
find_package(xtensor-python REQUIRED)

pybind11_add_module(mymodule python.cpp)
target_link_libraries(mymodule PUBLIC pybind11::module xtensor-python Python::NumPy)

target_compile_definitions(mymodule PRIVATE VERSION_INFO=0.1.0)

add_executable(myexec main.cpp)
target_link_libraries(myexec PUBLIC xtensor)

(see CMake tip above).

Then we can test the module:

example.py

import mymodule
import numpy as np

a = np.array([1, 2, 3])
assert np.isclose(np.sum(np.sin(a)), mymodule.sum_of_sines(a))

Note

Since we did not install the module, we should compile and run the example from the same folder. To install, please consult this pybind11 / CMake example. Tip: take care to modify that example with the correct CMake case Python_EXECUTABLE.

Fall-back cast

The previous example showed you how to design your module to be flexible in accepting data. From C++ we used xt::xarray<double>, whereas for the Python API we used xt::pyarray<double> to operate directly on the memory of a NumPy array from Python (without copying the data).

Sometimes, you might not have the flexibility to design your module’s methods with template parameters. This might occur when you want to override functions (though it is recommended to use CRTP to still use templates). In this case we can still bind the module in Python using xtensor-python, however, we have to copy the data from a (NumPy) array. This means that although the following signatures are quite different when used from C++, as follows:

Constant reference: read from the data, without copying it.
```
void foo(const xt::xarray<double>& a);
```
Reference: read from and/or write to the data, without copying it.
```
void foo(xt::xarray<double>& a);
```
Copy: copy the data.
```
void foo(xt::xarray<double> a);
```

The Python will all cases result in a copy to a temporary variable (though the last signature will lead to a copy to a temporary variable, and another copy to a). On the one hand, this is more costly than when using xt::pyarray and xt::pyxtensor, on the other hand, it means that all changes you make to a reference, are made to the temporary copy, and are thus lost.

Still, it might be a convenient way to create Python bindings, using a minimal effort. Consider this example:

main.cpp

#include <numeric>
#include <xtensor.hpp>
#include <pybind11/pybind11.h>
#define FORCE_IMPORT_ARRAY
#include <xtensor-python/pyarray.hpp>

template <class T>
double sum_of_sines(T& m)
{
    auto sines = xt::sin(m);  // sines does not actually hold values.
    return std::accumulate(sines.begin(), sines.end(), 0.0);
}

// In the Python API this a reference to a temporary variable
double sum_of_cosines(const xt::xarray<double>& m)
{
    auto cosines = xt::cos(m);  // cosines does not actually hold values.
    return std::accumulate(cosines.begin(), cosines.end(), 0.0);
}

PYBIND11_MODULE(mymodule, m)
{
    xt::import_numpy();
    m.doc() = "Test module for xtensor python bindings";
    m.def("sum_of_sines", sum_of_sines<xt::pyarray<double>>, "Sum the sines of the input values");
    m.def("sum_of_cosines", sum_of_cosines, "Sum the cosines of the input values");
}