Python: type-hinting CTypes "pointer to X" types in a way that MyPy accepts

user_7801407 0
Viewed 237

I have a large Python binding over a C library, with complex memory management. To help with that, I have drawn up the following aliases for strings (here, you a minimal reproducible example of my definitions and the subsequent problem).

from typing import Type, TypeAlias
from ctypes import POINTER, pointer, c_char, c_char_p

#normal python string
p_useable_p_str = str

#the result of a my_str.decode("utf-8")
c_useable_p_str = bytes

#used for string literals returned by the C, which should not be freed
p_useable_c_str = c_char_p

#used for allocated strings returned by the C, which need to be freed later  
c_useable_c_str = POINTER(c_char)  #the problematic line

def example(hello: c_useable_c_str):  # source of the MyPy error
    pass

My code runs fine (memory freed properly, consistent inheritance of the above if necessary, etc) when following the above definitions+usage conventions). POINTER(c_char) has the intended behavior in the rest of the code.

However, analyzing the above with MyPy, I get:

playground.py:10: error: Variable "playground.c_useable_c_str" is not valid as a type
playground.py:10: note: See https://mypy.readthedocs.io/en/latest/common_issues.html#variables-vs-type-aliases
Found 1 error in 1 file (checked 1 source file)

I get this error for anything that uses the c_useable_c_str alias. I, of course, read the section in the documentation linked above, and tried using Type and TypeAlias in a bunch of different ways - to no avail.

The only syntax that seems to make MyPy happy is

c_useable_c_str = pointer[c_char]

However, when actually running the code with this definition of the type alias, I get the following error (not seen by MyPy, so I suspect a bug on MyPy's end, or a lack of typing in the standard):

Traceback (most recent call last):
  File "/home/fulguritude/ProfessionalWork/LEDR/Orchestra-AvesTerra/Python_binding/playground2.py", line 92, in <module>
    c_useable_c_str = pointer[c_char]
TypeError: 'builtin_function_or_method' object is not subscriptable

Any ideas as to how I'm supposed to make things consistent ?

TLDR: what's the valid way to type hint, and alias, a "pointer to X" with MyPy and CTypes ?

mypy type-hinting ctypes python
2 Answers

I think you can use something like:

if TYPE_CHECKING: # valid for mypy
    c_useable_c_str = pointer[c_char]  # the problematic line
else: # valid at run time
    c_useable_c_str = pointer
user_41111300

I found an imperfect (but good enough) solution, based on what I found here: https://github.com/python/mypy/issues/7540 (comment by thijsmie, commented on May 21, 2021).

if not TYPE_CHECKING:
    # Monkeypatch typed pointer from typeshed into ctypes
    # NB: files that wish to import ctypes.pointer should all import `pointer` from this file instead
    class pointer_fix:
        @classmethod
        def __class_getitem__(cls, item):
            return POINTER(item)
    pointer = pointer_fix

It does solve MyPy's complaints concerning c_useable_c_str, and functions that use c_useable_c_str still get proper typing errors when the argument provided isn't of a compatible type, so that's good.

Well, EXCEPT for the following case:

class t_string(c_useable_c_str):
    pass

class t_string_p(pointer[t_string]):
    pass

where MyPy does not seem to be able to understand that t_string_p is compatible with the type pointer[pointer[c_char]]. For these cases, I used a type union; not ideal, but it got the job done.

I'll be leaving this question unanswered for now, in case someone, someday, can find a better solution.

user_78014070
Related