Modify the format of a string array after reading a CSV

Carola 0
Viewed 44

I have a DataFrame in python that contains a column with the following information:

                                          nodes  
0                                        [N1, N2]  
1                                        [N2, N2, N3]  
2                                        [N3, N1]

If I save it to CSV and read it back (using pandas read_csv function), the resulting DataFrame looks like this:

                                          nodes  
0                                        ['N1', 'N2']  
1                                        ['N2', 'N2', 'N3']  
2                                        ['N3', 'N1']

I have another function that I devote to apply on those DataFrames, and it works for the first format but not for the second one. I would like to be able to change the format of the second DataFrame (once read from the CSV) to the first one.

Both return the following with dtype:

nodes                    object
dtype: object
dataframe python
1 Answers

So I assume you have a column that holds list of strings. You serialize it as a CSV, and then you reload it. The issue is that those lists will not be loaded as lists of strings again but just as pure strings, which produces the output. You can apply json.loads on the entries of that column, which would transform the strings into lists again. In code that would like

import pandas as pd
import json

df = pd.DataFrame({"nodes":[["N1","N"],["N1","N3"]]})
df.to_csv("your.csv")

df2 = pd.read_csv("your.csv",converters={"nodes": lambda entry: json.loads(entry.replace("'", '"'))})

Note that this entry.replace("'", '"')is required due to having the right quoting in for json loads. Not sure if that is a general thing or just related to my concrete example.

Simon Hawe0
Related