I have a huge spatial dataframe with 1,4M entries, with X and Y columns. This causes SKlearn random forest algorithm to memory-overflow (32GB RAM)
I'd like to decrease the size of the dataframe to about 350K (or less) entries, with all other values being averaged out spatially. For example, to turn these 4 into a single dataframe row (ignore the fact that Y values are the same, it should be 2 unique X and 2 unique Ys)
X Y dose mezo hght
141668 -8773.349057 30004.7125 12.750189 156.410492 182.800507
141669 -8746.650943 30004.7125 12.742241 156.200867 182.364288
141670 -8719.952830 30004.7125 12.742241 155.998550 181.934616
141672 -8666.556604 30004.7125 12.729473 155.926254 181.166870
This data is going to be fed into sklearn's RandomForest classifier and be used to predict based on other data.