Python 3 __getattr__ behaving differently than in Python 2?

user_1091116 0
Viewed 4335

I need to write a class that implements 32-bit unsigned integers the same way they work in C programming language. What I care about most are the binary shifts, but I generally want my class to:

  1. Have the same interface int has and works with int properly
  2. Any operation with my U32 class (int + U32, U32 + int etc) also return U32
  3. Be pure-python - I don't want to use NumPy, ctypes, etc.

As can be found in this answer, I got a solution that works under Python 2. Recently I tried to run it under Python 3 and noticed that while the following test code works fine under older versions of Python, Python 3 raises an error:

class U32:
    """Emulates 32-bit unsigned int known from C programming language."""

    def __init__(self, num=0, base=None):
        """Creates the U32 object.

        Args:
            num: the integer/string to use as the initial state
            base: the base of the integer use if the num given was a string
        """
        if base is None:
            self.int_ = int(num) % 2**32
        else:
            self.int_ = int(num, base) % 2**32

    def __coerce__(self, ignored):
        return None

    def __str__(self):
        return "<U32 instance at 0x%x, int=%d>" % (id(self), self.int_)

    def __getattr__(self, attribute_name):
        print("getattr called, attribute_name=%s" % attribute_name)
        # you might want to take a look here:
        # https://stackoverflow.com/q/19611001/1091116
        r = getattr(self.int_, attribute_name)
        if callable(r):  # return a wrapper if integer's function was requested
            def f(*args, **kwargs):
                if args and isinstance(args[0], U32):
                    args = (args[0].int_, ) + args[1:]
                ret = r(*args, **kwargs)
                if ret is NotImplemented:
                    return ret
                if attribute_name in ['__str__', '__repr__', '__index__']:
                    return ret
                ret %= 2**32
                return U32(ret)
            return f
        return r

print(U32(4) / 2)
print(4 / U32(2))
print(U32(4) / U32(2))

And here's the error:

Traceback (most recent call last):
  File "u32.py", line 41, in <module>
    print(U32(4) / 2)
TypeError: unsupported operand type(s) for /: 'U32' and 'int'

It looks like the getattr trick doesn't get called at all in Python 3. Why is that? How can I get this code working both under Python 2 and 3?

python
2 Answers

Inherit from int and replace all operators you want to use:

(tested with Python 3.7)

class U32(int):
    MAXVALUE = 0xffffffff

    def __new__(cls, value):
        return int.__new__(cls, value & cls.MAXVALUE)

    def __add__(self, *args, **kwargs):
        return self.__new__(type(self),int.__add__(self, *args, **kwargs))

    def __radd__(self, *args, **kwargs):
        return self.__new__(type(self),int.__radd__(self, *args, **kwargs))

    def __sub__(self, *args, **kwargs):
        return self.__new__(type(self), int.__sub__(self, *args, **kwargs))

    def __rsub__(self,*args, **kwargs):
        return self.__new__(type(self),int.__rsub__(self, *args, **kwargs))

    def __mul__(self, *args, **kwargs):
        return self.__new__(type(self),int.__mul__(self, *args, **kwargs))

    def __rmul__(self, *args, **kwargs):
        return self.__new__(type(self),int.__rmul__(self, *args, **kwargs))

    def __div__(self, *args, **kwargs):
        return self.__new__(type(self),int.__floordiv__(self, *args, **kwargs))

    def __rdiv__(self, *args, **kwargs):
        return self.__new__(type(self),int.__rfloordiv__(self, *args, **kwargs))

    def __truediv__(self, *args, **kwargs):
        return self.__new__(type(self),int.__floordiv__(self, *args, **kwargs))

    def __rtruediv__(self, *args, **kwargs):
        return self.__new__(type(self),int.__rfloordiv__(self, *args, **kwargs))

    def __pow__(self, *args, **kwargs):
        return self.__new__(type(self),int.__pow__(self, *args, **kwargs))

    def __rpow__(self, *args, **kwargs):
        return self.__new__(type(self),int.__rpow__(self, *args, **kwargs))

    def __lshift__(self, *args, **kwargs):
        return self.__new__(type(self),int.__lshift__(self, *args, **kwargs))

    def __rlshift__(self, *args, **kwargs):
        return self.__new__(type(self),int.__rlshift__(self, *args, **kwargs))

    def __rshift__(self, *args, **kwargs):
        return self.__new__(type(self),int.__rshift__(self, *args, **kwargs))

    def __rrshift__(self, *args, **kwargs):
        return self.__new__(type(self),int.__rrshift__(self, *args, **kwargs))

    def __and__(self, *args, **kwargs):
        return self.__new__(type(self),int.__and__(self, *args, **kwargs))

    def __rand__(self, *args, **kwargs):
        return self.__new__(type(self),int.__rand__(self, *args, **kwargs))

    def __or__(self, *args, **kwargs):
        return self.__new__(type(self),int.__ror__(self, *args, **kwargs))

    def __ror__(self, *args, **kwargs):
        return self.__new__(type(self),int.__ror__(self, *args, **kwargs))

    def __xor__(self, *args, **kwargs):
        return self.__new__(type(self),int.__rxor__(self, *args, **kwargs))

    def __rxor__(self, *args, **kwargs):
        return self.__new__(type(self),int.__rxor__(self, *args, **kwargs))

Now it's easy to inherit other uint types:

class U16(U32):
    def __new__(cls, value):
        cls.MAXVALUE = 0xFFFF
        return int.__new__(cls, value&cls.MAXVALUE)


class U8(U32):
    def __new__(cls, value):
        cls.MAXVALUE = 0xFF
        return int.__new__(cls, value&cls.MAXVALUE)

Outputs

type( U8(0) + 1 ) = <class '__main__.U8'>
U8(0xcde) = 0xde
U8(2) + 0xff = 0x1
U8(0) + 0xfff = 0xff
U8(0) - 2 = 0xfe
U8(0xf) * 32 = 0xe0
U8(0x7)**3 = 0x57
U8(8) / 3 = 0x2
U8(0xff)>>4 = 0xf
U8(0xff)<<4 = 0xf0
type( 1 + U8(0) ) = <class '__main__.U8'>

Example printer to get the output examples

def exampleprinter(vec):
    for v in vec:
        result = eval(v)
        if issubclass(type(result), int):
            result = hex(result)
        print(v,'=' , result)

examples = [
            # results are of type U8
            'type( U8(0) + 1 )',
            # Correct tranform on over / underflow
            'U8(0xcde)',
            'U8(2) + 0xff',
            'U8(0) + 0xfff',
            'U8(0) - 2',
            'U8(0xf) * 32',
            'U8(0x7)**3',  # 7 ** 3 = 0x157 -> 0x157 & 0xff = 0x57
            # division will floor any remainder
            'U8(8) / 3',
            # Shifts
            'U8(0xff)>>4',
            'U8(0xff)<<4',
            # when swap operations are defined (the 'r...' ones) returns also U8 on reverse use
            'type( 1 + U8(0) )',
            ]

exampleprinter(examples)

I created these to take over some calculations of existing cryptogrphic-algorithm that relied heavily on bit-shifts on uint32.

A noticeable option is the reflective (or swapping) operator functions. These are the ones with the r-prefix, like __radd__, __rsub__, etc. They come into action when our uint type is the second operator, like in 1 + U8(0xff). If __radd__ is implemented the result will be 0x0 and of type U8, if is not implemented the result is 0x100 and of type int. While U8(0xff)+1 = 0 remains the same (since __add__ is defined). For my use-case it was best to define the reflective functions also to ensure all results were of type U32.

user_67003580
Related