Keras LSTM for continuous input and continuous output

user_5182526 0
Viewed 1022

For example I have binary data, let say: 0, 0, 0, 1, 1, 0, 1, 1. This may continue indefinitely. For each input, there is corresponding output. Let say we use XOR operation. So, output may look like this: 0, 0, 0, 1, 0, 1, 1, 0.

How do I shape Keras input shape? How do I set timesteps? If I declare timesteps 1 are for each 1 timestep considered different case or it can still take account of previous input as sequence or learned memory?

Keras is using LSTM or GRU for it's hidden layer.

I've tried 2 method for this problem, but none seem succeed. Both method stuck at 37.5 acc. In fact, it keep guessing 1.

Method 1:

data = [[[0], [0], [0], [1], [1], [0], [1], [1]]]
output = [[[0], [0], [0], [1], [0], [1], [1], [0]]]

model = Sequential()
model.add(GRU(10, input_shape=(8, 1), return_sequences=True))
model.add(GRU(10, return_sequences=True))
model.add(Dense(1, activation='softmax'))

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['acc'])
model.fit(np.asarray(data), np.asarray(output), epochs=3000)

Method 2:

data = [[[0]], [[0]], [[0]], [[1]], [[1]], [[0]], [[1]], [[1]]]
output = [[0], [0], [0], [1], [0], [1], [1], [0]]

model = Sequential()
model.add(GRU(10, input_shape=(1, 1), return_sequences=True))
model.add(GRU(10))
model.add(Dense(1, activation='softmax'))

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['acc'])
model.fit(np.asarray(data), np.asarray(output), epochs=300)
rnn lstm keras machine-learning python
2 Answers

In fact, it keep guessing 1.

That's because you have used softmax as the activation of last layer. Since the last layer have only one unit and the softmax function normalizes its input such that the sum of elements equals to one, it would always outputs 1. Instead, you need to use sigmoid as the activation function of last layer to have an output between zero and one.

user_20996070

Change activation='softmax' to activation='sigmoid'

user_103058190
Related