Run a dynamic multi-line string of code in python at runtime

user_8958754 0
Viewed 3150

How can I run a multi-line string of code from inside python, additionally the string is generated at runtime and stored in either a string or an array.

Background

  • I have a function which dynamically generates random code snippets in high volume.
  • I wish to pass this code to a variable/string and evaluate syntax/return code.
  • Code can not be imported from file/api request etc as focus is on high throughput and speed.

Example

# my function returns random code snippets
myCode = myRandomCodeGenerator()

# I wish to perform some in fight validation (ideally RC or syntax)
someTestingFunction(myCode)

My attempts so far

I have seen solutions such as the one below, however since my code is generated dynamically I have issues formatting. I have tried generating \n and \r or adding """ in the string to compensate with no luck. 

code = """
def multiply(x,y):
    return x*y

print('Multiply of 2 and 3 is: ',multiply(2,3))
"""

exec(code)

I have also tried using the call function on a string yields similar formatting issues.

So far the best I can do is to perform a line by line syntax check and feed it a line at a time as shown below. 

import codeop
def is_valid_code(line):
    try:
        codeop.compile_command(line)
    except SyntaxError:
        return False
    else:
        return True

I suspect there may be some syntax tricks I could apply to preserve formatting of indentation and returns. I am also aware of the risk of generating dynamic code in runtime, and have a filter of allowed terms in the function.

dynamic python-3.x python
3 Answers

Another way instead of eval and exec is to compile the code before and then to exec it:

Example:

import contextlib,sys
from io import StringIO 
@contextlib.contextmanager
def stdoutIO(stdout=None):
    old = sys.stdout
    if stdout is None:
        stdout = StringIO()
    sys.stdout = stdout
    yield stdout
    sys.stdout = old


def run_code(override_kale_blocks):
    compiled123 = []
    for b123 in override_kale_blocks:
        compiled123.append(compile(b123,"<string>","exec"))
    
    with stdoutIO() as s:
        for c123 in compiled123:
            exec(c123)
    return s.getvalue()
    


block0='''
import time
a=5
b=6
b=a+b
'''
block1='''
b=a+b
'''
block2="print(b)"
blocksleep='''
print('startsleep')
time.sleep(1)
print('donesleep')
'''
pc  = (block0,blocksleep,block1,block2)
cb = []
print('before')
output= run_code(pc)
print(output)
print('after')
print("Hello World!\n")

source: https://stackoverflow.com/a/3906390/1211174

user_12111740

Question was answered by Joran Beasley.

Essentially, my issue was I was adding a space after \n as other compilers get confused, python however takes this into account. Thus if you add a space after a return carriage in a def statement, it will produce syntax error.

I have kept this incase others encounter similar issues.

user_89587540

Try using '\t' instead of placing tabs and spaces for the indentation, that may preserve your formatting.

code = """
def multiply(x,y):
\t return x*y

print('Multiply of 2 and 3 is: ',multiply(2,3))
"""
exec(code)
user_81427340
Related