Let's say I want to predict the consumption of electricity of my bulding.
The input is 1 week of consumption and I want the output to be the predicted consumption for the day after, hour by hour i.e. consumption at midnight, consumption at 1 hour etc.
For simulation and simplicity, I use a very simple consumption function : c(h)=h where h is the hour of the day and c(h) is the consumption for this hour, here identity. Hence, the consumption for 1 day is ```[0, 1, 2,..., 23, 24]
I first learned 1 example of 7 vectors of 24 hours of consumption (i.e. 1 week) and got quite sensible results:
from keras.models import Sequential
from keras import layers
import matplotlib.pyplot as plt
import numpy as np
from keras.layers import RepeatVector
Inputs = np.tile(np.arange(24), (7)).reshape(1, 7, 24)
Outputs = np.arange(24).reshape(1, 24)
model = Sequential()
model.add(layers.LSTM(32, # 32 is choosen at random
input_shape=(None, Inputs.shape[-1]),
activation= 'relu'))
model.add(layers.Dense(24, activation='linear'))
model.compile(loss='mean_squared_error', optimizer='adam')
history = model.fit(x=Inputs, y=Outputs, epochs=150,
batch_size= 1,
validation_data = (Inputs, Outputs))
plt.figure()
plt.plot(model(Inputs, training=False)[0])
plt.show()
But now, I want to learn scalar values (no more vectors) since I guess the results should be different. And that's where I get lost. If I only change the shapes, I've got something that works:
Inputs = np.tile(np.arange(24), (7)).reshape(1, 7*24, 1)
Outputs = np.arange(24).reshape(1, 24, 1)
model2 = Sequential()
model2.add(layers.LSTM(32, # 32 is choosen at random
input_shape=(7*24, Inputs.shape[-1]),
# activation= 'relu'
))
model2.add(layers.Dense(24, activation='linear'))
model2.compile(loss='mean_squared_error', optimizer='adam')
history = model2.fit(x=Inputs, y=Outputs, epochs=200,
batch_size= 1,
validation_data = (Inputs, Outputs))
plt.figure()
y = model2(Inputs, training=False)
plt.plot(y[0])
plt.show()
It works but I'm really not sure this is the good way. I've tried to use more sophisticated approach (suitable with many-to-many problems) but it never worked:
Inputs = np.tile(np.arange(24), (7)).reshape(1, 7 * 24, 1)
Outputs = np.arange(24).reshape(1, 24, 1)
model2 = Sequential()
model2.add(layers.LSTM(32, input_shape=(7*24, 1))) # encoder layer
model2.add(RepeatVector(7*24)) # repeat vector
model2.add(layers.LSTM(32, return_sequences=True)) # decoder layer
model2.add(layers.TimeDistributed(layers.Dense(1)))
model2.compile(optimizer='adam', loss='mse')
print(model2.summary())
history = model2.fit(Inputs, Outputs, epochs=500, verbose=1, batch_size=1, validation_data = (Inputs, Outputs))
plt.figure()
res = model2(Inputs, training=False)
plt.plot(res[0])
plt.show()
In the version above, Keras complains about the shapes for the loss evaluation, but I've tried many things that did not work :( What's wrong with this code ? Optionally is this encoder/decoder approach the good approach ? ;)
Thanks in advance.