how does the magic store commands for dataframe work?

user_10829044 0
Viewed 416

I created two dataframes (df1 and df2) in two jupyter notebooks (N1 and N2) respectively.

On day 1, I used the below store command to use the df1 and its variables in N2 jupyter notebook

%store -r df1

But on day 25, I created a new jupyter notebook N3 and used the below store command again

%store -r df1

And it seemed to easily pull all the details of dataframe df1 into N3 jupyter notebook easily?

How does this work?

Aren't they valid only for that specific jupyter notebook session?

Then instead of storing all dataframes as files, can we just execute a store command and store/retrieve them easily anytime?

jupyter-notebook pandas dataframe python-import python
2 Answers

Storemagic is an IPython feature that "Stores variables, aliases and macros in IPython’s database". Because it is an IPython feature rather than exclusive to Jupyter you can store and restore variables across many IPython and Jupyter sessions.

In my environment (IPython 7.19.0) the variables get stored in the directory:

$HOME/.ipython/profile_default/db/autorestore

They are stored one per file when stored using %store <name>. The files themselves are the pickled representation of the stored variables. You could manually load the variables by using the following:

import pickle

# Name of the previously stored variable
stored_var = 'test'

# myvar will contain the variable previously stored with "%store test"
myvar_filename = get_ipython().ipython_dir + '/profile_default/db/autorestore/' + stored_var
with open(myvar_filename, 'rb') as f:
    myvar = pickle.load(f)
user_163429640

%store magic command stores variables in IPython’s database (it's essentially using pickle at the backend to store) so that you recover it later across separate notebooks or sessions.

e.g.

>>> myvar = "store this string"
>>> %store myvar
Stored 'myvar' (str)
>>> %store
Stored variables and their in-db values:
myvar             -> 'store this string'

Now, you can recover this variable

>>> %store -r myvar
>>> myvar
'store this string'
user_62108070
Related