Most efficient way to split a pandas dataframe column into several columns

user_10506005 0
Viewed 105

For example, I have a dataframe column ('x') that contains lists as values. 

import pandas as pd
jk = pd.DataFrame()
jk['x'] = [[1, 2, 3], [1, 4, 2], [27, 1, 3]]

I used the code below to split the values into columns like this. However, my actual data set is very large. I have about 80, 000 rows and 16 values inside every list. Is there a more efficient way to do this? 

jk1 = pd.DataFrame(jk.x.values.tolist(), columns=['a','b','c'])

enter image description here

pandas dataframe python-3.x python performance
1 Answers

No, there isn't a more efficient way

You should avoid creating a series of lists in the first place. As soon as you do this, you are left with an object dtype series with a nested layer of pointers. One layer pointing to each list, and another layer pointing to individual elements within each list. This prohibits vectorised operations.

user_92095460
Related