Build a Keras model that takes a structured array as an input

user_76701 0
Viewed 39

I'm doing Reinforcement Learning to teach agents to accomplish tasks in a 2-dimensional world. A big part of that is to figure out how to represent their environment as neurons.

So far I've represented the world has a 3-d grid of shape (10, 10, 7). The first two 10s are because the size of the grid is 10 in each direction, and the 7 is because I have 7 different kinds of things to say about each space (whether it has food, an enemy, a wall...)

I then used convolutional layers in Keras to process this information and learn from it. It worked and the creatures are successfully walking towards the food.

Now I would like to also add more information that the neural network might figure out how to use. For example, I'd like to encode the last action the agent took. I might also encode the distance or angle to the nearest food. Obviously, this is not 3-d data, this is a sequence of 1-d data.

I want Keras to be able to use that as input together with the 3-d input, and learn from it. I've represented that combined data as a structured array in NumPy:

observation = np.zeros((1,), dtype=[('grid', np.float64, (10, 10, 7,)), ('sequential', np.float64, (7,))])

That way it's possible to access the grid data as observation['grid'] and the sequential data as observation['sequential'].

Unfortunately I don't know how to get Keras to work with this kind of structured array. My reasoning is that I should build a model using the Functional API, and that model will have two "prongs" for the input that'll connect together to a concatenate at some point and get merged to a final output layer.

But, I have no idea how to make Keras figure out that the NumPy structured array should be broken down to the subarrays that it's made of. Is that possible?

If I'm going the wrong way with this, please advise.

reinforcement-learning keras deep-learning machine-learning numpy
1 Answers

in keras you can give different inputs as follow

from keras.models import Input, Model
from keras.layers import Conv2D, Dense, Flatten, concatenate

first_input = Input(shape=(10, 10, 7))
second_input = Input(shape=(7))

c1 = Conv2D(32, (3,3), padding='same') (first_input)
c1 = Flatten() (c1)
d1 = Dense(10) (second_input)

m = concatenate([c1,d1])
m = Dense(5) (m)

model = Model(inputs=[first_input,second_input], outputs=m)
model.compile(optimizer='adam' loss='categorical_crossentropy')

model.fit([observation['grid'], observation['sequential']], Y_train)

this is rough design but it will take your input and concatenate, then produce result

user_142131600
Related