I had some older working code that used keras. I recently dusted it off and tried to use it but with current versions of keras/tensorflow. I am getting a warning/error when calling predict_on_batch:
W tensorflow/core/data/root_dataset.cc:167] Optimization loop failed: Cancelled: Operation was cancelled
I tried to google this and, to my surprise, there doesn't seem to be a good explanation on the web as to what causes this or how to fix it. Here is what I did find:
https://github.com/tensorflow/tensorflow/issues/48689
https://discuss.tensorflow.org/t/optimization-loop-failed-cancelled-operation-was-cancelled/1524
The one answer it lists is to make sure the batch size isn't larger than the whole set. That isn't the case here.
The code is a bit long, so I can't easily show it all. It's a deep reinforcement learning app so the DL code is split up across two main functions which I'll show here:
class DQN(QContract):
def __init__(self, states, actions, lr, DDQN=False):
self.history = []
act_relu = activations.relu
act_linear = activations.linear
top_layer = 150
middle_layer = 120
# Create Network: Default Parameters from https://towardsdatascience.com/solving-lunar-lander-openaigym-reinforcement-learning-785675066197
model = Sequential()
layer = layers.Dense(top_layer, input_dim=states, activation=act_relu)
model.add(layer)
layer = layers.Dense(middle_layer, activation=act_relu)
model.add(layer)
layer = layers.Dense(actions, activation=act_linear)
model.add(layer)
opt = optimizers.Adam(learning_rate=lr)
model.compile(loss='mse', optimizer=opt)
# Create DDQN-like networks
self.modelA = model
#self.modelB = copy.deepcopy(model)
self.batch_size = 100
self.current = "A"
self.count = 0 def Update(self, state, action, reward, new_state, gamma, alpha=None):
# Preform Replay
row_count = self.batch_size
if len(self.history) < row_count: return
# Column names
state = 0
action = 1
reward = 2
next_state = 3
done = 4
# Get samples in mini-batches
samples = random.sample(self.history, row_count)
# Separate into separate arrays
states_array = [sample[state] for sample in samples]
actions_array = [sample[action] for sample in samples]
rewards_array = [sample[reward] for sample in samples]
next_states_array = [sample[next_state] for sample in samples]
done_array = [sample[done] for sample in samples]
# Turn into arrays
states_array = np.array(states_array)
actions_array = np.array(actions_array)
rewards_array = np.array(rewards_array)
next_states_array = np.array(next_states_array)
done_array = (1.0 - np.array(done_array))
# train on states_array
X = states_array
# Create y (i.e. labels for supervised learning)
if self.current == "A":
model1 = self.modelA
model2 = self.modelA
else:
model1 = self.modelA
model2 = self.modelA
predicted_values = self.modelA.predict_on_batch(states_array)
next_predicted_values = self.modelA.predict_on_batch(next_states_array)
actual_values = rewards_array + gamma * np.amax(next_predicted_values, axis=1) * done_array
predicted_values[list(range(row_count)), actions_array] = actual_values
y = predicted_values
# Update network
self.current = "A"
if self.current == "A":
print('Do fit'+str(self.count))
self.count += 1
self.modelA.fit(X, y, epochs=1, verbose=0)
self.current = "B"
else:
self.modelA.fit(X, y, epochs=1, verbose=0)
self.current = "A"At one point I was trying to do a DQN and I'm not right now, so ignore the attempt to have two models. It's currently disabled.
It seems like this would be a fairly straightforward issue, but I can't seem to figure it out. I even tried stepping through the code and I found it didn't happen when stepping through the debugger.