How to override the copy/deepcopy operations for a Python object?

user_159658 0
Viewed 87979

I understand the difference between copy vs. deepcopy in the copy module. I've used copy.copy and copy.deepcopy before successfully, but this is the first time I've actually gone about overloading the __copy__ and __deepcopy__ methods. I've already Googled around and looked through the built-in Python modules to look for instances of the __copy__ and __deepcopy__ functions (e.g. sets.py, decimal.py, and fractions.py), but I'm still not 100% sure I've got it right.

Here's my scenario:

I have a configuration object. Initially, I'm going to instantiate one configuration object with a default set of values. This configuration will be handed off to multiple other objects (to ensure all objects start with the same configuration). However, once user interaction starts, each object needs to tweak its configurations independently without affecting each other's configurations (which says to me I'll need to make deepcopys of my initial configuration to hand around).

Here's a sample object:

class ChartConfig(object):

    def __init__(self):

        #Drawing properties (Booleans/strings)
        self.antialiased = None
        self.plot_style = None
        self.plot_title = None
        self.autoscale = None

        #X axis properties (strings/ints)
        self.xaxis_title = None
        self.xaxis_tick_rotation = None
        self.xaxis_tick_align = None

        #Y axis properties (strings/ints)
        self.yaxis_title = None
        self.yaxis_tick_rotation = None
        self.yaxis_tick_align = None

        #A list of non-primitive objects
        self.trace_configs = []

    def __copy__(self):
        pass

    def __deepcopy__(self, memo):
        pass

What is the right way to implement the copy and deepcopy methods on this object to ensure copy.copy and copy.deepcopy give me the proper behavior?

python-internals python
10 Answers

Building on Antony Hatchkins' clean answer, here's my version where the class in question derives from another custom class (s.t. we need to call super):

class Foo(FooBase):
    def __init__(self, param1, param2):
        self._base_params = [param1, param2]
        super(Foo, result).__init__(*self._base_params)

    def __copy__(self):
        cls = self.__class__
        result = cls.__new__(cls)
        result.__dict__.update(self.__dict__)
        super(Foo, result).__init__(*self._base_params)
        return result

    def __deepcopy__(self, memo):
        cls = self.__class__
        result = cls.__new__(cls)
        memo[id(self)] = result
        for k, v in self.__dict__.items():
            setattr(result, k, copy.deepcopy(v, memo))
        super(Foo, result).__init__(*self._base_params)
        return result
user_37659050

Peter's and Eino Gourdin's answers are clever and useful, but they have a very subtle bug!

Python methods are bound to their object. When you do cp.__deepcopy__ = deepcopy_method, you are actually giving the object cp a reference to __deepcopy__ on the original object. Any calls to cp.__deepcopy__ will return a copy of the original! If you deepcopy your object and then deepcopy that copy, the output is a NOT a copy of the copy!

Here's a minimal example of the behavior, along with my fixed implementation where you copy the __deepcopy__ implementation and then bind it to the new object:

from copy import deepcopy
import types


class Good:
    def __init__(self):
        self.i = 0

    def __deepcopy__(self, memo):
        deepcopy_method = self.__deepcopy__
        self.__deepcopy__ = None
        cp = deepcopy(self, memo)
        self.__deepcopy__ = deepcopy_method
        # Copy the function object
        func = types.FunctionType(
            deepcopy_method.__code__,
            deepcopy_method.__globals__,
            deepcopy_method.__name__,
            deepcopy_method.__defaults__,
            deepcopy_method.__closure__,
        )
        # Bind to cp and set
        bound_method = func.__get__(cp, cp.__class__)
        cp.__deepcopy__ = bound_method

        return cp


class Bad:
    def __init__(self):
        self.i = 0

    def __deepcopy__(self, memo):
        deepcopy_method = self.__deepcopy__
        self.__deepcopy__ = None
        cp = deepcopy(self, memo)
        self.__deepcopy__ = deepcopy_method
        cp.__deepcopy__ = deepcopy_method
        return cp


x = Bad()
copy = deepcopy(x)
copy.i = 1
copy_of_copy = deepcopy(copy)
print(copy_of_copy.i)  # 0

x = Good()
copy = deepcopy(x)
copy.i = 1
copy_of_copy = deepcopy(copy)
print(copy_of_copy.i)  # 1
user_164914860

I came here for performance reasons. Using the default copy.deepcopy() function was slowing down my code by up to 30 times. Using the answer by @Anthony Hatchkins as a starting point, I realized that copy.deepcopy() is really slow for e.g. lists. I replaced the setattr loop with simple [:] slicing to copy whole lists. For anyone concerned with performance it is worthwhile doing timeit.timeit() comparisons and replacing the calls to copy.deepcopy() by faster alternatives.

setup = 'import copy; l = [1, 2, 3, 4, 5, 6, 7, 8, 9, 0]'
timeit.timeit(setup = setup, stmt='m=l[:]')
timeit.timeit(setup = setup, stmt='m=l.copy()')
timeit.timeit(setup = setup, stmt='m=copy.deepcopy(l)')

will give these results:

0.11505379999289289
0.09126630000537261
6.423627900003339
user_154678610

Similar with Zach Price's thoughts, there is a simpler way to achieve that goal, i.e. unbind the original __deepcopy__ method then bind it to cp

from copy import deepcopy
import types


class Good:
    def __init__(self):
        self.i = 0

    def __deepcopy__(self, memo):
        deepcopy_method = self.__deepcopy__
        self.__deepcopy__ = None
        cp = deepcopy(self, memo)
        self.__deepcopy__ = deepcopy_method
        
        # Bind to cp by types.MethodType
        cp.__deepcopy__ = types.MethodType(deepcopy_method.__func__, cp)

        return cp
user_120656960
Related