How am I supposed to use pandas functions?

user_14813170 0
Viewed 43

Let data be a giant pandas dataframe. It has many functions. The functions do not modify in place but return a new dataframe. How then am I supposed to perform multiple operations, to maximize performance?

For example, say I want to do

data = data.method1().method2().method()

where method1 could be set_index, and so on.

Is this the way you are supposed to do it? My wory is that pandas creates a copy every time I call a method, so that there are 3 copies being made of my data in the above, when in reality, all I want is to modify the original one.

So is it faster to say

data = data.method1(inplace=True)
data = data.method2(inplace=True)
data = data.method3(inplace=True)

This is just way too verbose for me?

pandas python
1 Answers

Yes you can do that, i.e applying methods one after the other. Since you overwrite data, you are not creating 3 copies, but only keeping one copy

data = data.method1().method2().method()

However regarding your second example, the general rule is to either write over the existing dataframe or do it "inplace", but not both at the same time

data = data.method1()

or

data.method1(inplace=True)
user_147757360
Related