saving and reloading variables in Python, preserving names

user_15239769 0
Viewed 834

I'm just coming to Python after many years of Matlab use. I'm still learning all the differences. I'm using Jupyter Notebooks (by necessity as it's with someone else who uses it)

Something I can't get my head round is saving variables so that they can be loaded in another script. I wan to run a script that processes some data, producing several variables. I then want to save those variables to file and be able to reload them in another script.

say my variable names are, 'wbl','wbr','body_ang'

In Matlab this is trivial, with something like:

save(filename,'wbl','wbr','body_ang')

I can then just load back the variables with:

load(filename)

I can't find anything that has the same functionality in Python. You can save the variables, using e.g. numpy.save, but it doesn't preserve the variable names. So unless you know the order you saved them in you can't recreate them.

I want them as separate variables, not just as part of an array (unless they can be easily recreated after loading).

It seems like such a simple and obvious thing, but I can't work out how to do it.

jupyter-notebook save matlab python
4 Answers

Not a Python thing, but a Jupyter Notebook thing is the %store command that let's you save a variable in one notebook to retrieve in an other:

Store:

x = 10
%store x

Stored 'x' (int)

Retrieve:

%store -r x
x

10
user_86355470

As Epsi95 said, you can use the pickle module in Python, which saves Python objects into files.

If you want to record the values of global variables, then use these functions (they work just like the examples you gave from Matlab):

import pickle

def save(filename, *args):
    # Get global dictionary
    glob = globals()
    d = {}
    for v in args:
        # Copy over desired values
        d[v] = glob[v]
    with open(filename, 'wb') as f:
        # Put them in the file 
        pickle.dump(d, f)

def load(filename):
    # Get global dictionary
    glob = globals()
    with open(filename, 'rb') as f:
        for k, v in pickle.load(f).items():
            # Set each global variable to the value from the file
            glob[k] = v

You can call them just the same:

save(filename,'wbl','wbr','body_ang')

load(filename)

This will only work with global variables, so the following won't work:

def foo():
    u = 8
    # "u" is not a global variable, so it can't be saved
    # This will raise an Exception 
    save("bar", "u")

foo()
# Even if we did save "u", where do we put it now?
# The function isn't running, so we'd have it put it in globals 
load("bar")

Using the global keyword can help you, though:

def foo():
    global u = 8
    # "u" is a global variable, so everything will work fine
    save("bar", "u")
    # Delete it if you don't want "u" to stick around after function
    del u

foo()
# Put "u" back in globals
load("bar")

If you want to save other variables besides global variables, you will have do something like pickle a dictionary with all your values mapped inside it, and retrieve the whole dictionary later as well.

EDIT: Fixed some bugs in the code

user_16573550
user_152398380

Here's a little helper module for storing/loading variables in json format:

import json
import inspect

def save_vars(fname, *args):
    d = {}
    caller_vars = inspect.stack()[1].frame.f_locals
    for arg in args:
        if arg in caller_vars:
            d[arg] = caller_vars[arg]
    with open(fname, 'w') as f:
        f.write(json.dumps(d))

def load_vars(fname):
    caller_vars = inspect.stack()[1].frame.f_locals
    with open(fname, 'r') as f:
        d = json.loads(f.read())
    for k,v in d.items():
        caller_vars[k] = v

Keep this module in your current directory as, say, myvars.py .

Here's how you can use it:

sgal@diff$ python3
Python 3.9.0 (default, Nov 30 2020, 15:21:09)
>>> from myvars import *
>>> a,b,c=1,2,3
>>> save_vars('vars','a','b','c')

sgal@diff$ cat vars
{"a": 1, "b": 2, "c": 3}

sgal@diff$ python3
Python 3.9.0 (default, Nov 30 2020, 15:21:09)
>>> from myvars import *
>>> load_vars('vars')
>>> a
1
>>> b
2
>>> c
3
user_28894780
Related