return numpy array of std::variant in c++

I'm trying to create a pybind11 interface which returns a tuple of two numpy arrays from C++ to Python. The subfunction lttb inside the lambda expression returns two vectors of type std::vector<double> and std::vector<std::variant<uint32_t, in64_t, double>>.

If I directly return the vectors, the pybind11/stl.h header takes care of the conversion just fine. The creation of the double numpy array also works flawlessly. However, when trying to create a numpy array of std::variant, I get the following error

..\subprojects\pybind11\include\pybind11/numpy.h(1159): error C2338: Attempt to use a non-POD or unimplemented POD type as a numpy dtype

There is a way to register structured types (https://pybind11.readthedocs.io/en/stable/advanced/pycpp/numpy.html#structured-types) in order to prevent this error message, however I only see a way to make this work with a struct not with an std::variant. Does anyone know how to create a numpy array of std::variant? Should not be that hard as the conversion of an std::vector<std::variant<...>> works out of the box. Here's the code.

    .def("lttb",
         [&](ScopeDataElement& data,
             const std::pair<double, double>& limit,
             int64_t masterTick,
             long double samplingTime,
             size_t destinationSize) {
           auto&& [x, y] =
             data.lttb(limit, masterTick, samplingTime, destinationSize);
           using T = std::decay_t<decltype(x)>::value_type;
           using U = std::decay_t<decltype(y)>::value_type;
           auto arrx = py::array_t<T>(x.size(), x.data());
           auto arry = py::array_t<U>(y.size(), y.data());
           return py::make_tuple(arrx, arry);
         })