Training a Neural Network with Multiple Datasets (Keras)

Admin 1
Viewed 4183

The datasets I am working with correspond to individual time series signals. Each signal is unique, with differing total number of data points, though each signal represents the same semantic data (speed in mph).

I am working with Keras, and trying to fit a basic neural network to the data just to evaluate it. Below is the Python code for that:

model = Sequential()
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(64, activation='relu'))
model.add(Dense(1, activation='sigmoid'))

model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

Essentially, I am fitting the model to each dataset as follows:

for file in directory:
    data = pd.read_csv(file)
    # get x_train and y_train ...
    model.fit(X_train, y_train, epochs=10)

Is this a valid way to train a model on multiple datasets of the same semantic data?

keras deep-learning training-data neural-network python
1 Answers

Yes, either you can create a model and then call data in loop for training or you can stack data in a single matrix using loop and then call fit function. In first approach you will call fit() n times but in smaller data chunks while in latter approach you will call fit() only once but use a big data matrix.

However, the first one is better as it might be a problem assign all the data in one matrix. So go ahead with your current implementation.

user_143377750
Related