keras dqn agent expecting more dimentions

user_11664078 0
Viewed 176

I have built a custom environment based on the openAI gym on which I aim to train a DQN agent.

In this environment, each observation space is one row, and 75 columns, and so

env.observation_space.shape

(75,)

When I build a model, I use the following:

def build_model(states, actions):
    model = Sequential()
    model.add(Dense(75, activation = 'relu', input_dim = 75))
    model.add(Dense(75, activation = 'relu'))
    model.add(Dense(actions, activation = 'relu'))
    return model

With the output shape of the first layer (none, 75) and the output shape of the final layer (none, 3) for each of the three possible actions.

In building my agent I use:

def build_agent(model, actions):
    policy = BoltzmannQPolicy()
    memory = SequentialMemory(limit=50000, window_length=1)
    dqn = DQNAgent(model=model, memory=memory, policy=policy, 
                  nb_actions=actions, nb_steps_warmup=10, target_model_update=1e-2)
    return dqn

However, fitting the agent throws the following error:

dqn = build_agent(model, actions)
dqn.compile(Adam(lr=1e-3), metrics=['mae'])
dqn.fit(env, nb_steps=50000, visualize=False, verbose=1)

Error when checking input: expected dense_58_input to have 2 dimensions, but got array with shape (1, 1, 75)

I don't understand how an extra dimension has been expected, given my data observations are 75 columns. Do I need to reshape my input or re-define the input layer of my model?

reinforcement-learning keras tensorflow python
2 Answers

From the comment section for the benefit of the community.

currently using rl.agents. I fixed this problem using a flatten layer as the first layer.

user_142906970

I see that you are using ‘relu’ activation in your last layer. DQN usually performs regression to predict q-values for each action and using a relu activation would prevent estimates from reaching their true values. If you could share your entire code, the community should be able to provide more insights.

user_116284370
Related