I am training my data by using multiclass SVM, and this is my reference. Whenever I try to run the line below, it gives me a memory error. How do I assign df.iloc[:,:-1] to a variable without getting any error?
x=df.iloc[:,:-1]
How do I define df.iloc[:,:-1] properly without getting any error?
Viewed 33
1 Answers
You can try this:
x = df
y = x.pop(x.columns[-1])
This will set x to be a reference to df, then cause the rightmost column of df (with contents equivalent to df.iloc[:,-1]) to be assigned to y and deleted from x.
In the pandas source, pop() is equivalent to:
item = x.columns[-1]
y = x[item]
del x[item]
Because del is an in-place operation, pop() should not attempt to make a copy of the input dataframe before mutating it. This may be less memory intensive than OP's code.
Note that the 5.6 GB size of df shown in OP's comment also suggests it may be prudent to try to modify program logic to partition the input and process it in smaller batches.
Related