Is there a way to implement early stopping in Keras ONLY after the first, say, 100 epochs

My fit function is nonconvex so that the loss function can sometimes worsen before it improves. Given this, I want to use early stopping, but only after the first 100 or more epochs. So far I have this:

# Early stopping
ES = [EarlyStopping(monitor='val_loss',patience=100,verbose=1,mode='auto')]

# fit model
history = model.fit(x_train, y_train, epochs=1000,batch_size=50,verbose=2,shuffle=True,validation_split=.1,callbacks=ES)

Unfortunately, the fit stops very early after 10 or so epochs. I want to wait until the 100th epoch to start early stopping. Any ideas? Any suggestions other than early stopping are also appreciated.