Why is scikit-learn's random forest using so much memory?

user_1937197 0
Viewed 2342

I'm using scikit's Random Forest implementation:

sklearn.ensemble.RandomForestClassifier(n_estimators=100, 
                                        max_features="auto", 
                                        max_depth=10)

After calling rf.fit(...), the process's memory usage increases by 80MB, or 0.8MB per tree (I also tried many other settings with similar results. I used top and psutil to monitor the memory usage)

A binary tree of depth 10 should have, at most, 2^11-1 = 2047 elements, which can all be stored in one dense array, allowing the programmer to find parents and children of any given element easily.

Each element needs an index of the feature used in the split and the cut-off, or 6-16 bytes, depending on how economical the programmer is. This translates into 0.01-0.03MB per tree in my case.

Why is scikit's implementation using 20-60x as much memory to store a tree of a random forest?

scikit-learn decision-tree classification machine-learning random-forest
1 Answers

Each decision (non-leaf) node stores the left and right branch integer indices (2 x 8 bytes), the index of the feature used to split (8 bytes), the float value of the threshold for the decision feature (8 bytes), the decrease in impurity (8 bytes). Furthermore leaf nodes store the constant target value predicted by the leaf.

You can have a look at the Cython class definition in the source code for the details.

user_1637400
Related