Partial Dependence Plot for a Keras Neural Network

Viewed 1741

I have a densely connected neural network that was built using the Keras Sequential API. I'm trying to create some partial dependence plots (PDP's) to use for a bit a sensitivity analysis. I am attempting to use the scikit-learn plot_partial_dependence function in order to do this. I've been getting the following error: ValueError: 'estimator' must be a fitted regressor or classifier.. When it first happened, I added the use of KerasClassifier. I've used it successfully in the past to use my Keras model in scikit-learn GridSearchCV. I'm still getting the same error. I've also tried KerasRegressor.

Can anyone tell me what's wrong and how I could fix it? Do I absolutely need to use scikit-learn's decision tree based functions to be able to use the PDP function? If yes, what's the biggest implementation difference between Keras neural networks and decision trees? (I've never used decision trees. My machine learning experience is limited to Keras.)

My relevant code is below and I'm running python on google colab's GPU. I'm sure there are several issues in that last line but I can't get past this one to figure them out.

from sklearn.inspection import plot_partial_dependence
from keras.wrappers.scikit_learn import KerasClassifier
from keras.wrappers.scikit_learn import KerasRegressor

def create_model():
  def swish(x):
    return (x*sigmoid(x))

  from keras.utils.generic_utils import get_custom_objects
  from keras.layers import Activation
  get_custom_objects().update({'swish':(swish)})

  model=Sequential()

  model.add(Dense(1024,activation='swish',input_shape=(6,)))
  model.add(Dropout(.1))

  model.add(Dense(512,activation='swish'))

  model.add(Dense(256,activation='swish'))
  model.add(Dropout(.1))

  model.add(Dense(128,activation='swish'))

  model.add(Dense(64,activation='swish'))
  model.add(Dropout(.1))

  model.add(Dense(32,activation='swish'))

  model.add(Dense(16,activation='swish'))
  model.add(Dropout(.1))

  model.add(Dense(12, activation='softmax'))

  opt=optimizers.Adam(lr=0.05)

  model.compile(loss='categorical_crossentropy',optimizer='adam', metrics=['accuracy'])

  return model

from keras.callbacks import LearningRateScheduler
from keras.callbacks import EarlyStopping
import math
def scheduler(epoch, lr):
  if epoch < 20:
    return lr
  else:
    return lr * math.exp(-0.1)

callback=LearningRateScheduler(scheduler, verbose=1)

weightsCallback=EarlyStopping(patience=30,monitor='accuracy',restore_best_weights=True, min_delta=1*10**-5, verbose=1)

modelClassified=KerasClassifier(build_fn=create_model)

modelClassified.fit(X_train, Y_train, batch_size=50, epochs=50, callbacks=[callback,weightsCallback], verbose=1)

disp=plot_partial_dependence(modelClassified, X_holdout,target=1, verbose =1, features=[0,1,2,3,4,5], feature_names=['aspect ratio','diel inner radius','diel outer radius','diel seperation','diel height','diel constant'])
1 Answers

I found that this error is actually a bug. My program should have, by all means, worked just fine. There is an error in the plot_partial_dependence function source code.

For much more detail and the solution I used to make it work, see this link to another StackOverflow question: https://stackoverflow.com/a/61485502/13822019

Related