Insert dictionary(JSON) as is in a Pandas Dataframe

Viewed 2900

I have a usecase where I need to convert the existing columns of a dataframe into JSON and store in only on one column.

So far I tried this:

import pandas as pd
import json
df=pd.DataFrame([{'a':'sjdfb','b':'jsfubs'},{'a':'ouhbsdv','b':'cm osdn'}]) #Random data
jsonresult1=df.to_json(orient='records')
# '[{"a":"sjdfb","b":"jsfubs"},{"a":"ouhbsdv","b":"cm osdn"}]'

But I want the data to be just string representation of the dictionary and not the list. So I tried this:

>>>jsonresult2=df.to_dict(orient='records')
>>>jsonresult2
# [{'a': 'sjdfb', 'b': 'jsfubs'}, {'a': 'ouhbsdv', 'b': 'cm osdn'}]

This is what I wanted the data to look like but when I try to make this a dataframe the dataframe again is in the format of 2 columns [a,b]. The string representation of these dictionary objects will insert into the dataframe the column data in the required format.

>>>for i in range(len(jsonresult2)):
...    jsonresult3.append(str(jsonresult2[i]))
...
>>> jsonresult3
["{'a': 'sjdfb', 'b': 'jsfubs'}", "{'a': 'ouhbsdv', 'b': 'cm osdn'}"]

This is exactly what I wanted. And when I push this to a dataframe I get:

 >>> df1
                              0
 ++++++++++++++++++++++++++++++++++++
 0  |   {'a': 'sjdfb', 'b': 'jsfubs'}
 1  |{'a': 'ouhbsdv', 'b': 'cm osdn'}

But I feel this is a very inefficient way. How do I make it look and work in an optimized way? My data can exceed 10M rows. And this is taking too long.

2 Answers
Related