Retrieve Decision Boundary Lines (x,y coordinate format) from SKlearn Decision Tree

Question

I am trying to create a surface plot on an external visualization platform. I'm working with the iris data set that is featured on the sklearn decision tree documentation page. I'm also using the same approach to create my decision surface plot. My end goal though is not the matplot lib visual, so from here I input the data to my visualization software. To do this I just called flatten() and tolist() on xx, yy and Z and wrote a JSON file containing these lists.

The trouble is when I try to plot it, my visualization program crashes. It turns out the data is too large. When flattened the length of the list is >86,000. This is due to the fact the step size/plot step is very small .02. So it is essentially taking baby steps across the domain of the data's min and max and plotting/filling as it goes, according to the model's predictions.It's kind of like a pixel-grid; I shrunk the size down to an array of only 2000 and noticed that the coordinates were just lines going back and forth (eventually encompassing the entire coordinate plane).

Question: Can I retrieve the x,y coordinates of the decision boundary lines themselves (as opposed to iterating across the whole plane)? Ideally a list containing only the turning points of each line. Or alternatively, is there maybe some other completely different way to recreate this plot, so that it is more computationally efficient?

This can somewhat be visualized by replacing the contourf() call with countour():

I'm just not sure how to retrieve the data governing those lines (via xx, yy and Z or possibly other means?).

Note: I'm not picky about the exact format of the list/or data structure that contains the lines format as long as its computationally efficient. For instance, for the first plot above, some red areas are actually islands in the prediction space, so that might mean we'd have to handle it like it's its own line. I'm guessing as long as the class is coupled with the x,y coordinates, it shouldn't matter how many arrays (containing coordinates)are used to capture the decision boundaries.

user_1281995 · Answer

For those interested, I had to recently also implement this for higher dimensional data, code was as follow:

number_of_leaves = (tree.tree_.children_left == -1).sum()
features = x.shape[1]
boundaries = np.zeros([number_of_leaves, features, 2])
boundaries[:,:,0] = -np.inf
boundaries[:,:,1] = np.inf

locs = np.where(tree.tree_.children_left == -1)[0]

for k in range(locs.shape[0]):
    idx = locs[k]
    idx_new = idx

    while idx_new != 0:
        i_check = np.where(tree.tree_.children_left == idx_new)[0]
        j_check = np.where(tree.tree_.children_right == idx_new)[0]

        if i_check.shape[0] == 1:
            idx_new = i_check[0]
            feat_ = tree.tree_.feature[idx_new]
            val_ = tree.tree_.value[idx_new]
            boundaries[k,feat_, 0] = val_
        elif j_check.shape[0] == 1:
            idx_new = j_check[0]
            feat_ = tree.tree_.feature[idx_new]
            val_ = tree.tree_.value[idx_new]
            boundaries[k,feat_, 1] = val_ 
        else: 
            print('Fail Case') # for debugging only - never occurs

Essentially I build up a n*d*2 tensor where n is the number of leaves of the tree, d is the dimensionality of the space and the third dimension holds the min and max values. Leaves are stored in tree.tree_.children_left / tree.tree_.children_right as -1, I then loop backwards to find the branch that caused the split onto the leaf and add the splitting criteria to the decision bounds.

Retrieve Decision Boundary Lines (x,y coordinate format) from SKlearn Decision Tree

3 Answers