Store numpy.array in cells of a Pandas.DataFrame

user_420892 0
Viewed 111222

I have a dataframe in which I would like to store 'raw' numpy.array:

df['COL_ARRAY'] = df.apply(lambda r: np.array(do_something_with_r), axis=1)

but it seems that pandas tries to 'unpack' the numpy.array.

Is there a workaround? Other than using a wrapper (see edit below)?

I tried reduce=False with no success.

EDIT

This works, but I have to use the 'dummy' Data class to wrap around the array, which is unsatisfactory and not very elegant.

class Data:
    def __init__(self, v):
        self.v = v

meas = pd.read_excel(DATA_FILE)
meas['DATA'] = meas.apply(
    lambda r: Data(np.array(pd.read_csv(r['filename'])))),
    axis=1
)
pandas dataframe numpy python
6 Answers

If you first set a column to have type object, you can insert an array without any wrapping:

df = pd.DataFrame(columns=[1])
df[1] = df[1].astype(object)
df.loc[1, 1] = np.array([5, 6, 7, 8])
df

Output:

    1
1   [5, 6, 7, 8]
user_49307440

Suppose you have a DataFrame ds and it has a column named as 'class'. If ds['class'] contains strings or numbers, and you want to change them with numpy.ndarrays or lists, the following code would help. In the code, class2vector is a numpy.ndarray or list and ds_class is a filter condition.

ds['class'] = ds['class'].map(lambda x: class2vector if (isinstance(x, str) and (x == ds_class)) else x)

user_116376630

choose eval buildin function is easy to use and easy to read.

# First ensure use object store str
df['col2'] = self.df['col2'].astype(object)
# read
arr_obj = eval(df.at[df[df.col_1=='xyz'].index[0], 'col2']))
# write
df.at[df[df.col_1=='xyz'].index[0], 'col2'] = str(arr_obj)

real store display perfect human readable value:

col_1,  col_2
xyz,    "['aaa', 'bbb', 'ccc', 'ddd']"
user_97591330

Just wrap what you want to store in a cell to a list object through first apply, and extract it by index 0of that list through second apply:

import pandas as pd
import numpy as np

df = pd.DataFrame({'id': [1, 2, 3, 4],
                   'a': ['on', 'on', 'off', 'off'],
                   'b': ['on', 'off', 'on', 'off']})


df['new'] = df.apply(lambda x: [np.array(x)], axis=1).apply(lambda x: x[0])

df

output:

    id  a       b       new
0   1   on      on      [1, on, on]
1   2   on      off     [2, on, off]
2   3   off     on      [3, off, on]
3   4   off     off     [4, off, off]
user_18514920
Related