Common pitfalls in Python

When you need a population of arrays you might be tempted to type something like this:

>>> a=[[1,2,3,4,5]]*4

And sure enough it will give you what you expect when you look at it

>>> from pprint import pprint
>>> pprint(a)

[[1, 2, 3, 4, 5],
 [1, 2, 3, 4, 5],
 [1, 2, 3, 4, 5],
 [1, 2, 3, 4, 5]]

But don't expect the elements of your population to be seperate objects:

>>> a[0][0] = 2
>>> pprint(a)

[[2, 2, 3, 4, 5],
 [2, 2, 3, 4, 5],
 [2, 2, 3, 4, 5],
 [2, 2, 3, 4, 5]]

Unless this is what you need...

It is worth mentioning a workaround:

a = [[1,2,3,4,5] for _ in range(4)]

Python Language Gotchas -- things that fail in very obscure ways

• Using mutable default arguments.

• Leading zeroes mean octal. 09 is a very obscure syntax error in Python 2.x

• Misspelling overridden method names in a superclass or subclass. The superclass misspelling mistake is worse, because none of the subclasses override it correctly.

Python Design Gotchas

• Spending time on introspection (e.g. trying to automatically determine types or superclass identity or other stuff). First, it's obvious from reading the source. More importantly, time spent on weird Python introspection usually indicates a fundamental failure to grasp polymorphism. 80% of the Python introspection questions on SO are failure to get Polymorphism.

• Spending time on code golf. Just because your mental model of your application is four keywords ("do", "what", "I", "mean"), doesn't mean you should build a hyper-complex introspective decorator-driven framework to do that. Python allows you to take DRY to a level that is silliness. The rest of the Python introspection questions on SO attempts to reduce complex problems to code golf exercises.

• Monkeypatching.

• Failure to actually read through the standard library, and reinventing the wheel.

• Conflating interactive type-as-you go Python with a proper program. While you're typing interactively, you may lose track of a variable and have to use globals(). Also, while you're typing, almost everything is global. In proper programs, you'll never "lose track of" a variable, and nothing will be global.

Avoid using keywords as your own identifiers.

Also, it's always good to not use from somemodule import *.

Surprised that nobody said this:

Mix tab and spaces when indenting.

Really, it's a killer. Believe me. In particular, if it runs.

Code Like a Pythonista: Idiomatic Python

Using the %s formatter in error messages. In almost every circumstance, %r should be used.

For example, imagine code like this:

try:
    get_person(person)
except NoSuchPerson:
    logger.error("Person %s not found." %(person))

Printed this error:

ERROR: Person wolever not found.

It's impossible to tell if the person variable is the string "wolever", the unicode string u"wolever" or an instance of the Person class (which has __str__ defined as def __str__(self): return self.name). Whereas, if %r was used, there would be three different error messages:

...
logger.error("Person %r not found." %(person))

Would produce the much more helpful errors:

ERROR: Person 'wolever' not found.
ERROR: Person u'wolever' not found.
ERROR: Person  not found.

Another good reason for this is that paths are a whole lot easier to copy/paste. Imagine:

try:
    stuff = open(path).read()
except IOError:
    logger.error("Could not open %s" %(path))

If path is some path/with 'strange' "characters", the error message will be:

ERROR: Could not open some path/with 'strange' "characters"

Which is hard to visually parse and hard to copy/paste into a shell.

Whereas, if %r is used, the error would be:

ERROR: Could not open 'some path/with \'strange\' "characters"'

Easy to visually parse, easy to copy-paste, all around better.

• don't write large output messages to standard output
• strings are immutable - build them not using "+" operator but rather using str.join() function.
• read those articles:
  • python gotchas
  • things to avoid
  • Gotchas for Python users
  • Python Landmines

Last link is the original one, this SO question is an duplicate.

I would stop using deprecated methods in 2.6, so that your app or script will be ready and easier to convert to Python 3.

A bad habit I had to train myself out of was using X and Y or Z for inline logic.

Unless you can 100% always guarantee that Y will be a true value, even when your code changes in 18 months time, you set yourself up for some unexpected behaviour.

Thankfully, in later versions you can use Y if X else Z.

Some personal opinions, but I find it best NOT to:

• use deprecated modules (use warnings for them)

• overuse classes and inheritance (typical of static languages legacy maybe)

• explicitly use declarative algorithms (as iteration with for vs use of itertools)

• reimplement functions from the standard lib, "because I don't need all of those features"

• using features for the sake of it (reducing compatibility with older Python versions)

• using metaclasses when you really don't have to and more generally make things too "magic"

• avoid using generators

• (more personal) try to micro-optimize CPython code on a low-level basis. Better spend time on algorithms and then optimize by making a small C shared lib called by ctypes (it's so easy to gain 5x perf boosts on an inner loop)

• use unnecessary lists when iterators would suffice

• code a project directly for 3.x before the libs you need are all available (this point may be a bit controversial now!)

++n and --n may not work as expected by people coming from C or Java background.

++n is positive of a positive number, which is simply n.

--n is negative of a negative number, which is simply n.

import this    

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than right now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

import not_this

Write ugly code.
Write implicit code.
Write complex code.
Write nested code.
Write dense code.
Write unreadable code.
Write special cases.
Strive for purity.
Ignore errors and exceptions.
Write optimal code before releasing.
Every implementation needs a flowchart.
Don't use namespaces.

Somewhat related to the default mutable argument, how one checks for the "missing" case results in differences when an empty list is passed:

def func1(toc=None):
    if not toc:
        toc = []
    toc.append('bar')

def func2(toc=None):
    if toc is None:
        toc = []
    toc.append('bar')

def demo(toc, func):
    print func.__name__
    print '  before:', toc
    func(toc)
    print '  after:', toc

demo([], func1)
demo([], func2)

Here's the output:

func1
  before: []
  after: []
func2
  before: []
  after: ['bar']

You've mentioned default arguments... One that's almost as bad as mutable default arguments: default values which aren't None.

Consider a function which will cook some food:

def cook(breakfast="spam"):
    arrange_ingredients_for(breakfast)
    heat_ingredients_for(breakfast)
    serve(breakfast)

Because it specifies a default value for breakfast, it is impossible for some other function to say "cook your default breakfast" without a special-case:

def order(breakfast=None):
    if breakfast is None:
        cook()
    else:
        cook(breakfast)

However, this could be avoided if cook used None as a default value:

def cook(breakfast=None):
    if breakfast is None:
        breakfast = "spam"

def order(breakfast=None):
    cook(breakfast)

A good example of this is Django bug #6988. Django's caching module had a "save to cache" function which looked like this:

def set(key, value, timeout=0):
    if timeout == 0:
        timeout = settings.DEFAULT_TIMEOUT
    _caching_backend.set(key, value, timeout)

But, for the memcached backend, a timeout of 0 means "never timeout"… Which, as you can see, would be impossible to specify.