How to "perfectly" override a dict?

user_104021 0
Viewed 160214

How can I make as "perfect" a subclass of dict as possible? The end goal is to have a simple dict in which the keys are lowercase.

It would seem that there should be some tiny set of primitives I can override to make this work, but according to all my research and attempts it seem like this isn't the case:

Here is my first go at it, get() doesn't work and no doubt there are many other minor problems:

class arbitrary_dict(dict):
    """A dictionary that applies an arbitrary key-altering function
       before accessing the keys."""

    def __keytransform__(self, key):
        return key

    # Overridden methods. List from 
    # https://stackoverflow.com/questions/2390827/how-to-properly-subclass-dict

    def __init__(self, *args, **kwargs):
        self.update(*args, **kwargs)

    # Note: I'm using dict directly, since super(dict, self) doesn't work.
    # I'm not sure why, perhaps dict is not a new-style class.

    def __getitem__(self, key):
        return dict.__getitem__(self, self.__keytransform__(key))

    def __setitem__(self, key, value):
        return dict.__setitem__(self, self.__keytransform__(key), value)

    def __delitem__(self, key):
        return dict.__delitem__(self, self.__keytransform__(key))

    def __contains__(self, key):
        return dict.__contains__(self, self.__keytransform__(key))


class lcdict(arbitrary_dict):
    def __keytransform__(self, key):
        return str(key).lower()
get set dictionary inheritance python
6 Answers

After trying out both of the top two suggestions, I've settled on a shady-looking middle route for Python 2.7. Maybe 3 is saner, but for me:

class MyDict(MutableMapping):
   # ... the few __methods__ that mutablemapping requires
   # and then this monstrosity
   @property
   def __class__(self):
       return dict

which I really hate, but seems to fit my needs, which are:

  • can override **my_dict
    • if you inherit from dict, this bypasses your code. try it out.
    • this makes #2 unacceptable for me at all times, as this is quite common in python code
  • masquerades as isinstance(my_dict, dict)
    • rules out MutableMapping alone, so #1 is not enough
    • I heartily recommend #1 if you don't need this, it's simple and predictable
  • fully controllable behavior
    • so I cannot inherit from dict

If you need to tell yourself apart from others, personally I use something like this (though I'd recommend better names):

def __am_i_me(self):
  return True

@classmethod
def __is_it_me(cls, other):
  try:
    return other.__am_i_me()
  except Exception:
    return False

As long as you only need to recognize yourself internally, this way it's harder to accidentally call __am_i_me due to python's name-munging (this is renamed to _MyDict__am_i_me from anything calling outside this class). Slightly more private than _methods, both in practice and culturally.

So far I have no complaints, aside from the seriously-shady-looking __class__ override. I'd be thrilled to hear of any problems that others encounter with this though, I don't fully understand the consequences. But so far I've had no problems whatsoever, and this allowed me to migrate a lot of middling-quality code in lots of locations without needing any changes.

As evidence: https://repl.it/repls/TraumaticToughCockatoo

Basically: copy the current #2 option, add print 'method_name' lines to every method, and then try this and watch the output:

d = LowerDict()  # prints "init", or whatever your print statement said
print '------'
splatted = dict(**d)  # note that there are no prints here

You'll see similar behavior for other scenarios. Say your fake-dict is a wrapper around some other datatype, so there's no reasonable way to store the data in the backing-dict; **your_dict will be empty, regardless of what every other method does.

This works correctly for MutableMapping, but as soon as you inherit from dict it becomes uncontrollable.

Edit: as an update, this has been running without a single issue for almost two years now, on several hundred thousand (eh, might be a couple million) lines of complicated, legacy-ridden python. So I'm pretty happy with it :)

Edit 2: apparently I mis-copied this or something long ago. @classmethod __class__ does not work for isinstance checks - @property __class__ does: https://repl.it/repls/UnitedScientificSequence

user_162950

All you will have to do is 

class BatchCollection(dict):
    def __init__(self, *args, **kwargs):
        dict.__init__(*args, **kwargs)

OR

class BatchCollection(dict):
    def __init__(self, inpt={}):
        super(BatchCollection, self).__init__(inpt)

A sample usage for my personal use

### EXAMPLE
class BatchCollection(dict):
    def __init__(self, inpt={}):
        dict.__init__(*args, **kwargs)

    def __setitem__(self, key, item):
        if (isinstance(key, tuple) and len(key) == 2
                and isinstance(item, collections.Iterable)):
            # self.__dict__[key] = item
            super(BatchCollection, self).__setitem__(key, item)
        else:
            raise Exception(
                "Valid key should be a tuple (database_name, table_name) "
                "and value should be iterable")

Note: tested only in python3

user_9483010

collections.UserDict is often the simplest option when you need a custom dict.

As shown in the other answer, it's very tricky to overwrite dict correctly, while UserDict makes it easy. To answer the original question, you can get a dict with lower keys:

import collections

class LowercaseDict(collections.UserDict):

  def __getitem__(self, key):
    return super().__getitem__(key.lower())

  def __setitem__(self, key, value):
    return super().__setitem__(key.lower(), value)

  def __delitem__(self, key):
    return super().__delitem__(key.lower())

  # Unfortunately, __contains__ is required currently due to
  # https://github.com/python/cpython/issues/91784
  def __contains__(self, key):
    return key.lower() in self.data


d = LowercaseDict(MY_KEY=0)  # Keys normalized in .__init__
d.update({'OTHER_KEY': 1})  # Keys normalized in .update
d['Hello'] = d['other_KEY']
assert 'HELLO' in d
print(d)  # All keys normalized {'my_key': 0, 'other_key': 1, 'hello': 1}

And contrary to collections.abc.MutableMapping, you don't need __iter__, __len__, __init__,... Subclassing UserDict is much easier.

However UserDict is a MutableMapping, not a dict, so:

assert not isinstance(collections.UserDict(), dict)
assert isinstance(collections.UserDict(), collections.abc.MutableMapping)
user_41726850
Related