Overfitting in RNN for S&P prediction

i'm doing a RNN based on the model from the Deep Learning A-Z course on Udemy. For the example for Google Stocks, we used 5 years of daily stock price. In the end of the lecture is said to test with more data or change the parameters or the structure of the RNN. My thought was that if i can get more data the RNN can get better results. I downloaded the data from S&P from 01/01/2006 to today, separated the train test except the last 23 days and the 23 days are my test for prediction. So im excited to see if i can get a kind of useful insigths...let it run in 100 epochs.

Epoch 1/100
3599/3599 [==============================] - 235s 65ms/step - loss: 0.0090
Epoch 2/100
3599/3599 [==============================] - 210s 58ms/step - loss: 0.0024
Epoch 3/100
3599/3599 [==============================] - 208s 58ms/step - loss: 0.0022
Epoch 4/100
3599/3599 [==============================] - 557s 155ms/step - loss: 0.0024
Epoch 5/100
3599/3599 [==============================] - 211s 59ms/step - loss: 0.0022
Epoch 6/100
3599/3599 [==============================] - 207s 58ms/step - loss: 0.0018
Epoch 7/100
3599/3599 [==============================] - 216s 60ms/step - loss: 0.0018
Epoch 8/100
3599/3599 [==============================] - 265s 74ms/step - loss: 0.0016
Epoch 9/100
3599/3599 [==============================] - 215s 60ms/step - loss: 0.0016
Epoch 10/100
3599/3599 [==============================] - 209s 58ms/step - loss: 0.0014
Epoch 11/100
3599/3599 [==============================] - 217s 60ms/step - loss: 0.0014
Epoch 12/100
3599/3599 [==============================] - 216s 60ms/step - loss: 0.0013
Epoch 13/100
3599/3599 [==============================] - 218s 60ms/step - loss: 0.0012
Epoch 14/100
3599/3599 [==============================] - 217s 60ms/step - loss: 0.0012
Epoch 15/100
3599/3599 [==============================] - 210s 58ms/step - loss: 0.0012
Epoch 16/100
3599/3599 [==============================] - 292s 81ms/step - loss: 0.0012
Epoch 17/100
3599/3599 [==============================] - 328s 91ms/step - loss: 0.0011
Epoch 18/100
3599/3599 [==============================] - 199s 55ms/step - loss: 9.8658e-04
Epoch 19/100
3599/3599 [==============================] - 199s 55ms/step - loss: 0.0010
Epoch 20/100
3599/3599 [==============================] - 286s 79ms/step - loss: 9.9106e-04

WOW 0,0010 was pretty good...but from here it's way too low. i stopped in the 39 epoch...because it's taking too long and the loss is too small.

Epoch 39/100
2560/3599 [====================>.........] - ETA: 1:00 - **loss: 6.3598e-04**

This is the results

Did i overfit the data? Or stopping too soon is the cause of the large errors? What can i do to optimize the time required to run the 100 epochs?

The code is the following:

  # Recurrent Neural Network



# Part 1 - Data Preprocessing

# Importing the libraries
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import LSTM
from keras.layers import Dropout

# Importing the training set
dataset_train = pd.read_csv('S&P_Train.csv')
training_set = dataset_train.iloc[:, 1:2].values

# Feature Scaling

sc = MinMaxScaler(feature_range =  [0, 1])
training_set_sc = sc.fit_transform(training_set)

# Creating a data structure with 60 timesteps and 1 output
X_train = []
y_train = []
for i in range(60, 3659):
    X_train.append(training_set_sc[i-60:i, 0])
    y_train.append(training_set_sc[i, 0])
X_train, y_train = np.array(X_train), np.array(y_train)


# Reshaping
X_train = np.reshape(X_train, (X_train.shape[0], X_train.shape[1], 1))



# Part 2 - Building the RNN

# Importing the Keras libraries and packages


# Initialising the RNN
regressor = Sequential()


# Adding the first LSTM layer and some Dropout regularisation
rnn = regressor.add(LSTM(units = 50, return_sequences = True, input_shape = (X_train.shape[1], 1)))
rnn = regressor.add(Dropout(0.2))

# Adding a second LSTM layer and some Dropout regularisation
rnn = regressor.add(LSTM(units = 50, return_sequences = True))
rnn = regressor.add(Dropout(0.2))

# Adding a third LSTM layer and some Dropout regularisation
rnn = regressor.add(LSTM(units = 50, return_sequences = True))
rnn = regressor.add(Dropout(0.2))

# Adding a fourth LSTM layer and some Dropout regularisation
rnn = regressor.add(LSTM(units = 50))
rnn = regressor.add(Dropout(0.2))

# Adding the output layer
rnn = regressor.add(Dense(units = 1))

# Compiling the RNN

rnn = regressor.compile(optimizer = 'Adam', loss = 'mean_squared_error')
# Fitting the RNN to the Training set
regressor.fit(X_train, y_train, epochs = 100, batch_size = 32)



# Part 3 - Making the predictions and visualising the results
print('ok')

# Getting the real stock price of 2017
dataset_test = pd.read_csv('S&P_Test.csv')
real_stock_price = dataset_test.iloc[:, 1:2].values

# Getting the predicted stock price of 2017
dataset_total = pd.concat((dataset_train['Open'], dataset_test['Open']), axis = 0)
inputs = dataset_total[len(dataset_total) - len(dataset_test) - 60:].values
inputs = inputs.reshape(-1, 1)
inputs = sc.transform(inputs)
X_test = []
for i in range(60, 83):
    X_test.append(inputs[i-60:i, 0])
X_test = np.array(X_test)
X_test = np.reshape(X_test, (X_test.shape[0], X_test.shape[1], 1))
predicted_stock_price = regressor.predict(X_test)
predicted_stock_price = sc.inverse_transform(predicted_stock_price)

# Visualising the results
plt.plot(real_stock_price, color = 'red', label = 'Real Stock Price')
plt.plot(predicted_stock_price, color = 'blue', label = 'Predicted Stock Price')
plt.title('Prediction of Stocks Values')
plt.xlabel('time')
plt.ylabel('Stock Price')
plt.legend()
plt.show()