Get variable name from NameError object

user_12074845 0
Viewed 394

if I catch a NameError exception using except:

try:
    print(unknownVar)
except NameError as ne:
    print(ne)

I get a string like :

NameError: name 'unknownVar' is not defined

I work in the context of eval'ed expressions and it whould be a useful information to me if I could obtain only the variable name (here "unknownVar" alone) and not the full string. I did not find an attribute for example in the NameError object to get it (perhaps does it exists, but I did not find it). Is there something better than parsing this string to do to get the information I need ?

Best Regards
Mikhaël

python
3 Answers

Extract it from the string:

ne.args[0].split()[1].strip("'")

user_131706900

You can extract it using regex:

import re
try:
    print(unknownVar)
except NameError as ne:
    var_name = re.findall(r"'([^']*)'", str(ne))[0] 
    print(var_name) # output: unknownVar
user_124002140

Unfortunately, error messages are not exactly Python's strong suit. However, there is actually an alternative to parsing the string, but it is quite "hacky" and only works with CPython (i.e. this will fail with PyPy, Jython, etc.).

The idea is to extract the name of whatever you wanted to load from the underlying code object.

import sys
import opcode

def extract_name():
    tb = sys.exc_info()[2]        # get the traceback
    while tb.tb_next is not None:
        tb = tb.tb_next
    instr_pos = tb.tb_lasti       # the index of the "current" instruction
    frame = tb.tb_frame           
    code = frame.f_code           # the code object
    instruction = opcode.opname[code.co_code[instr_pos]]
    arg = code.co_code[instr_pos + 1]
    if instruction == 'LOAD_FAST':
        return code.co_varnames[arg]
    else:
        return code.co_names[arg]

def test(s):
    try:
        exec(s)
    except NameError:
        name = extract_name()
        print(name)

test("print(x + y)")

1. The Background of Code Object

Python compiles the original Python source code into bytecode and then executes that bytecode. The code is stored in "code objects", which are (partly) documented here. For our purpose, the following will suffice:

class CodeObject:
    co_code: bytes       # the bytecode instructions
    co_varnames: tuple   # names of local variables and parameters
    co_names: tuple      # all other names

If some code produces a NameError, it failed to load a specific name. That name must be either in the co_names or co_varnames tuple. All we have to figure out is which one.

While the code objects desribe the code statically, we also need a dynamic object that tells us the value of local variables and which instruction we are currently executing. This role is fulfilled by the "frame" (leaving out irrelevant details):

class Frame:
    f_code: CodeObject     # the code object (see above)
    f_lasti: int           # the instruction currently executed

You could think of the interpreter as basically doing the following:

def runCode(code):
    frame = create_new_frame(code)
    while True:
        i = frame.f_lasti
        opcode = frame.f_code.co_code[i]
        arg    = frame.f_code.co_code[i+1]
        exec_opcode(opcode, arg)
        frame.f_lasti += 2

The code to load a name then has a form like this:

  • LOAD_NAME 3 (the actual name is co_names[3])
  • LOAD_GLOBAL 3 (the actual name is co_names[3])
  • LOAD_FAST 3 (the actual name is co_varnames[3])

You can see that we have to distinguish between LOAD_FAST (i.e. load a local variable) and all other LOAD_X opcodes.

2. Getting The Right Name

When an error occurs, we need to go through the stacktrace/traceback until we find the frame in which the error occurred. From the frame we then get the code object with the list of all names and instructions, extract the instruction and argument that led to the error and thus the name.

We retrieve the traceback with sys.exc_info()[2]. The actual frame and traceback we are interested in is the very last one (this is what you can read in the line Traceback (most recent call last): whenever a runtime error occurs):

    tb = sys.exc_info()[2]        # get the traceback
    while tb.tb_next is not None:
        tb = tb.tb_next

This traceback object then contains two information of importance to us: the frame tb_frame and the instruction pointer tb_last where the error occurred. From the frame we then extract the code object:

    instr_pos = tb.tb_lasti       # the index of the "current" instruction
    frame = tb.tb_frame           
    code = frame.f_code           # the code object

Since the byte encoding the instruction can change with different Python versions, we want to get the human-readable form, which is more stable. We need that so that we can distinguish between local variables all others:

    instruction = opcode.opname[code.co_code[instr_pos]]
    arg = code.co_code[instr_pos + 1]
    if instruction == 'LOAD_FAST':
        return code.co_varnames[arg]
    else:
        return code.co_names[arg]

3. Caveat

If the code object uses more than 255 names, a single byte will no longer be enough as index into the tuples with all names. In that case, the bytecode allows for an extension prefix, which is not taken into account here. But for most code objects, this should work just fine.

As mentioned in the beginning, this is a rather hacky method that is based on internals of Python that might change (although this is rather unlikely). Nonetheless, it is fun taking Python apart this way, isn't it ;-).

user_135497090
Related