Keras predict a number, pass if within a range

user_5398105 0
Viewed 931

I'm having issues with my epochs and the accuracy of my runs. The accuracy is all over the place and it has to do with the fact that I want to estimate a number. I want the test to pass if the estimated amount is say +/- 2% or something.

Code:

seed = 7
basepath = '.'

# find the right path for batch ai vs local
outpath = os.path.join (basepath, "out")
if not os.path.exists(outpath):
    os.makedirs(outpath)

# Importing the dataset
dataset = pd.read_csv(os.path.join (basepath, 'data.csv'))

# fix random seed for reproducibility
np.random.seed(seed)

#Encode columns using label encoding 
#use a new label encoder everytime is important!
vixpercentencoder = LabelEncoder()
dataset['VIX Open Percent'] = responsetimeencoder.fit_transform(dataset['VIX Open Percent'])

fiftydayaverageencoder = LabelEncoder()
dataset['50 day average'] = suppliesgroupencoder.fit_transform(dataset['50 day average'])

twohundreddayaverageencoder = LabelEncoder()
dataset['200 day average'] = suppliessubgroupencoder.fit_transform(dataset['200 day average'])

openingencoder = LabelEncoder()
dataset['opening'] = regionencoder.fit_transform(dataset['opening'])

#routetomarketencoder = LabelEncoder()
#dataset['Route To Market'] = routetomarketencoder.fit_transform(dataset['Route To Market'])

#What are the correlations between columns and target
correlations = dataset.corr()['closing'].sort_values()

#Throw out unneeded columns 
dataset = dataset.drop('Date', axis=1)
dataset = dataset.drop('VIX Open', axis=1)
dataset = dataset.drop('VIX Close', axis=1)
dataset = dataset.drop('Ticker', axis=1)
#dataset = dataset.drop('VIX Open Percent', axis=1)

#One Hot Encode columns that are more than binary
# avoid the dummy variable trap
#dataset = pd.concat([pd.get_dummies(dataset['Route To Market'], prefix='Route To Market', drop_first=True),dataset], axis=1)
#dataset = dataset.drop('Route To Market', axis=1)

#Create the input data set (X) and the outcome (y)
X = dataset.drop('closing', axis=1).iloc[:, 0:dataset.shape[1] - 1].values
y = dataset.iloc[:, dataset.columns.get_loc('closing')].values

# Feature Scaling
sc = StandardScaler()
X = sc.fit_transform(X)

# Initilzing the ANN
model = Sequential()

#Adding the input layer
model.add(Dense(units = 8, activation = 'relu', input_dim=X.shape[1], name= 'Input_Layer'))

#Add hidden layer
model.add(Dense(units = 8, activation = 'relu', name= 'Hidden_Layer_1'))

#Add the output layer
model.add(Dense(units = 1, activation = 'sigmoid', name= 'Output_Layer'))

# compiling the ANN
model.compile(optimizer= 'nadam', loss = 'binary_crossentropy', metrics=['accuracy'])

# summary to console
print (model.summary())

#Fit the ANN to the training set
history = model.fit(X, y, validation_split = .20, batch_size = 64, epochs = 25)

# summarize history for accuracy
plt.plot(history.history['acc'])
plt.plot(history.history['val_acc'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()

# summarize history for loss
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()
keras machine-learning neural-network python
1 Answers

It seems that you are trying to predict a continuous value (i.e. regression problem) rather than a discrete value (i.e. classification problem). Therefore, I would suggest the followings:

  1. Using sigmoid as the activation function of last layer is not appropriate here, unless the target values are strictly in the range [0,1]. Instead, don't use any activation for the last layer (i.e. linear activation) if the values are unbounded.

  2. Use an appropriate regression loss function, such as Mean Squared Error, i.e. 'mse'.

  3. Using 'accuracy' as the metric does not make sense in a regression task (i.e. it is only used in classification problems). Instead, if you want to have a metric to monitor training, use another metric such as Mean Absolute Error, i.e. 'mae'.

Following the above would be needed for a proper setup of your model. Then, the cycle of experimenting and tuning the model begins. You may experiment with different layers, different number of layers or units in layers, adding regularization, etc., until you find a model with good performance. Of course, you may also need to have a validation set so that you can compare the performance of different configurations with each other on a fixed unseen set of samples.

As a final note, don't expect anyone here gives you a complete "wining solution". You need to experiment yourself with the data you have, as in machine learning designing the suitable model for some specific data and a specific task is a combination of art, science and experience. At the end, all others can give you would be just some pointers or ideas (of course, apart from mentioning your mistakes).

user_20996070
Related