Cython: Converting a Cython function with `except` into C like function

user_10177420 0
Viewed 359

I am basically trying to wrap a C library and want to pass Python functions in wrapped C functions using Cython. I am using global variables to store the Python function, calling it in a Cython wrapper and then passing the Cython wrapper as a C function to some other C function. I want to capture the errors in the Python function when called by the Cython wrapper, so, I add a except? -1 at the end of the Cython wrapper declaration.

The problem is when I pass a Cython function with except keyword, it fails to convert it into C like function. If I don't, I can't handle errors occuring in the Python function (I need to do this as the function is called in a loop)

A MRE is shown below.

# file: mre.pyx

# cython: language_level=3

import numpy as np
cimport numpy as np

_glob_params = None
_glob_func = None

cdef extern from "mre.h":
    double eval_func(double (*func)(double, double *, int))

cdef double _func_wrapper(double x, double *params, int n) except? -1:
    return _glob_func(x, *_glob_params)

def run_func(func, params=(), Py_ssize_t size=100):
    cdef Py_ssize_t i
    cdef np.ndarray[np.float64_t, ndim=1] out = np.empty(size, dtype=np.float64)
    global _glob_func, _glob_params

    _glob_func = func
    _glob_params = params

    for i in range(size):
        out[i] = eval_func(_func_wrapper)

    return out

// file : mre.h

#pragma once

#include <stdlib.h>
#include <stdio.h>

double eval_func(double (*func)(double, double *, int)) {
    double res = func(1.0, NULL, 0);
    if ( res < 0. ) {
        fprintf(stderr, "ERROR: func < 0. -> not allowed!\n");
    }
    return res;
}

Error:

Error compiling Cython file:
------------------------------------------------------------
...

    _glob_func = func
    _glob_params = params

    for i in range(size):
        out[i] = eval_func(_func_wrapper)
                          ^
------------------------------------------------------------

mre.pyx:24:27: Cannot assign type 'double (double, double *, int) except? -1.0' to 'double (*)(double, double *, int)'

Without except? -1:

>>> from mre import run_func
>>> run_func(None)
TypeError: 'NoneType' object is not callable
Exception ignored in: 'mre._func_wrapper'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'NoneType' object is not callable
TypeError: 'NoneType' object is not callable
Exception ignored in: 'mre._func_wrapper'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
             ...
             ...
             ...
             ...
TypeError: 'NoneType' object is not callable
array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])

So, is there a way to handle exceptions in Cython without except? I tried to use ctypes to wrap the Python function but it is very slow in practice.

cython c python
1 Answers

The way exceptions work in Python, once an exception is handled, the global stay must be changed, i.e. error must be cleared (e.g. via PyErr_Clear()), otherwise the interpreter is in an inconsistent state and strange things can happen.

Because your C code is not aware of Python, it cannot clear the error indicator, thus it must happen in Cython code. One possibility would be to add an additional wrapper:

cdef double wrapper_noraise(double x, double *params, int n):
    try:
        return _func_wrapper(val)
    except:
        return -1.0

and now pass wrapper_noraise to C code.

Or, in your case, you could directly change _func_wrapper to:

cdef double _func_wrapper(double x, double *params, int n):
    try:
       return _glob_func(x, *_glob_params)
    except:
       return -1.0
user_57694630
Related