Monkey patching class and instance in Python

user_12118546 0
Viewed 56

I am confused with following difference. Say I have this class with some use case:

class C:
    def f(self, a, b, c=None):
        print(f"Real f called with {a=}, {b=} and {c=}.")


my_c = C()
my_c.f(1, 2, c=3)  # Output: Real f called with a=1, b=2 and c=3.

I can monkey patch it for purpose of testing like this:

class C:
    def f(self, a, b, c=None):
        print(f"Real f called with {a=}, {b=} and {c=}.")


def f_monkey_patched(self, *args, **kwargs):
    print(f"Patched f called with {args=} and {kwargs=}.")


C.f = f_monkey_patched
my_c = C()
my_c.f(1, 2, c=3)  # Output: Patched f called with args=(1, 2) and kwargs={'c': 3}.

So far so good. But I would like to patch only one single instance and it somehow consumes first argument:

class C:
    def f(self, a, b, c=None):
        print(f"Real f called with {a=}, {b=} and {c=}.")


def f_monkey_patched(self, *args, **kwargs):
    print(f"Patched f called with {args=} and {kwargs=}.")


my_c = C()
my_c.f = f_monkey_patched
my_c.f(1, 2, c=3)  # Output: Patched f called with args=(2,) and kwargs={'c': 3}.

Why has been first argument consumed as self instead of the instance itself?

monkeypatching python
3 Answers

Functions in Python are descriptors; when they're attached to a class, but looked up on an instance of the class, the descriptor protocol gets invoked, producing a bound method on your behalf (so my_c.f, where f is defined on the class, is distinct from the actual function f you originally defined, and implicitly passes my_c as self).

If you want to make a replacement that shadows the class f only for a specific instance, but still passes along the instance as self like you expect, you need to manually bind the instance to the function to create the bound method using the (admittedly terribly documented) types.MethodType:

from types import MethodType  # The class implementing bound methods in Python 3

# ... Definition of C and f_monkey_patched unchanged

my_c = C()
my_c.f = MethodType(f_monkey_patched, my_c)  # Creates a pre-bound method from the function and
                                             # the instance to bind to

Being bound, my_c.f will now behave as a function that does not accept self from the caller, but when called self will be received as the instance bound to my_c at the time the MethodType was constructed.

Update with performance comparisons:

Looks like, performance-wise, all the solutions are similar enough as to be irrelevant performance-wise (Kedar's explicit use of the descriptor protocol and my use of MethodType are equivalent, and the fastest, but the percentage difference over functools.partial is so small that it won't matter under the weight of any useful work you're doing):

>>> # ... define C as per OP
>>> def f_monkey_patched(self, a):  # Reduce argument count to reduce unrelated overhead
...     pass

>>> from types import MethodType
>>> from functools import partial
>>> partial_c, mtype_c, desc_c = C(), C(), C()
>>> partial_c.f = partial(f_monkey_patched, partial_c)
>>> mtype_c.f = MethodType(f_monkey_patched, mtype_c)
>>> desc_c.f = f_monkey_patched.__get__(desc_c, C)
>>> %%timeit x = partial_c  # Swapping in partial_c, mtype_c or desc_c
... x.f(1)
...

I'm not even going to give exact timing outputs for the IPython %%timeit magic, as it varied across runs, even on a desktop without CPU throttling involved. All I could say for sure is that partial was reliably a little slower, but only by a matter of ~1 ns (the other two typically ran in 56-56.5 ns, the partial solution typically took 56.5-57.5), and it took quite a lot of paring of extraneous stuff (e.g. switching from %timeit reading the names from global scope causing dict lookups to caching to a local name in %%timeit to use simple array lookups) to even get the differences that predictable.

Point is, any of them work, performance-wise. I'd personally recommend either my MethodType or Kedar's explicit use of descriptor protocol approach (they are identical in end result AFAICT; both produce the same bound method class), whichever one looks prettier to you, as it means the bound method is actually a bound method (so you can extract .__self__ and .__func__ like you would on any bound method constructed the normal way, where partial requires you to switch to .args[0] and .func to get the same info).

user_3646960

You can convert the function to bound method by calling its __get__ method (since all function as descriptors as well, thus have this method)

def t(*args, **kwargs):
    print(args)
    print(kwargs)

class Test():
    pass
Test.t = t.__get__(Test(), Test) # binding to the instance of Test

For example

Test().t(1,2, x=1, y=2)
(<__main__.Test object at 0x7fd7f6d845f8>, 1, 2)
{'y': 2, 'x': 1}

Note that the instance is also passed as an positional argument. That is if you want you function to be instance method, the function should have been written in such a way that first argument behaves as instance of the class. Else, you can bind the function to None instance and the class, which will be like staticmethod.

Test.tt = t.__get__(None, Test)
Test.tt(1,2,x=1, y=2)
(1, 2)
{'y': 2, 'x': 1}

Furthermore, to make it a classmethod (first argument is class):

Test.ttt = t.__get__(Test, None) # bind to class
Test.ttt(1,2, x=1, y=2)
(<class '__main__.Test'>, 1, 2)
{'y': 2, 'x': 1}
user_135421520

When you do C.f = f_monkey_patched, and later instantiate an object of C, the function is bound to that object, effectively doing something like

obj.f = functools.partial(C.f, obj)

When you call obj.f(...), you are actually calling the partially bound function, i.e. f_monkey_patched(obj, ...)

On the other hand, doing my_c.f = f_monkey_patched, you assign the function as-is to the attribute my_c.f. When you call my_c.f(...), those arguments are passed to the function as-is, so self is the first argument you passed, i.e. 1, and the remaining arguments go to *args

user_8439530
Related