The unbearable opaqueness of time.struct_time

Question

Why do pylint and the intellisense features of IDEs have trouble recognizing instances of time.struct_time? The following code contains some trivial tests of existent/non-existent attributes of classes, named tuples and the named-tuple-like time.struct_time. Everything works as expected in pylint, IntelliJ and VSCode - the access to missing attributes is reported in each case except for time.struct_time - it generates no warnings or errors in any of these tools. Why can't they tell what it is and what its attributes are?

import time
from collections import namedtuple

t = time.localtime()
e = t.tm_mday
e = t.bad # this is not reported by linters or IDEs. 

class Clz:
    cvar = 'whee'

    def __init__(self):
        self.ivar = 'whaa'

o = Clz()
e = Clz.cvar
e = o.ivar
e = Clz.bad
e = o.bad

Ntup = namedtuple('Ntup', 'thing')
n = Ntup(thing=3)
e = n.thing
e = n.bad

The context of the question is the following recent bug in pipenv -

# Halloween easter-egg.          
if ((now.tm_mon == 10) and (now.tm_day == 30))

Obviously, the pass path was never tested but it seems the typical static analysis tools would not have helped here either. This is odd for a type from the standard library.

(Fix can be seen in full at https://github.com/kennethreitz/pipenv/commit/033b969d094ba2d80f8ae217c8c604bc40160b03)

user_100297 · Answer

time.struct_time is an object defined in C, which means it can't be introspected statically. The autocompletion software can parse Python code and make a reasonable guess as to what classes and namedtuples support, but they can't do this for C-defined objects.

The work-around most systems use is to generate stub files; usually by introspecting the object at runtime (importing the module and recording the attributes found). For example, CodeIntel (part of the Komodo IDE), uses an XML file format called CIX. However, this is a little more error-prone so such systems then err on the side of caution, and will not explicitly mark unknown attributes as wrong.

If you are coding in Python 3, you could look into using type hinting. For C extensions you still need stub files, but the community is pretty good at maintaining these now. The standard library stub files are maintained in a project called typeshed.

You'd have to add type hints to your project:

#!/usr/bin/env python3
import time
from collections import namedtuple


t: time.struct_time = time.localtime()
e: int = t.tm_mday
e = t.bad  # this is not reported by linters or IDEs.


class Clz:
    cvar: str = 'whee'
    ivar: str

    def __init__(self) -> None:
        self.ivar = 'whaa'


o = Clz()
s = Clz.cvar
s = o.ivar
s = Clz.bad
s = o.bad

Ntup = namedtuple('Ntup', 'thing')
n = Ntup(thing=3)
e = n.thing
e = n.bad

but then the flake8 tool combined with the flake8-mypy plugin will detect the bad attributes:

$ flake8 test.py
test.py:8:5: T484 "struct_time" has no attribute "bad"
test.py:22:5: T484 "Clz" has no attribute "bad"
test.py:23:5: T484 "Clz" has no attribute "bad"
test.py:28:5: T484 "Ntup" has no attribute "bad"

PyCharm builds on this work too, and perhaps can detect the same invalid use. It certainly directly supports pyi files.

The unbearable opaqueness of time.struct_time

1 Answers