Force keras mode.fit() to use Multiprocessing

Question

I am using Keras with theano backend and I want to train my Network on a gpu. That actually works pretty good. But when I want to train on a huge amount of data, I recognized, that there is a bottleneck in the model.fit() function (I am using the functional API).

Actually in the model.fit() function Keras starts to use the GPU for the training. But before it starts on the GPU, it needs much CPU-effort to prepare the training (I don't know exactly what fit() is doing before the actual training). The problem is, that this part only uses one thread, so that this part takes pretty long.

Is it possible to force Keras to use multiprocessing at this step?

Edit: Added additional data to my function call:

My function call looks like this:

optimizer = SGD(lr=0.00001)
early_stopping = EarlyStopping(monitor='val_loss', patience=30, verbose=1, mode='auto')
outname = join(outdir, save_base_name+".model")
checkpoint = ModelCheckpoint(outname, monitor='val_loss', verbose=1, save_best_only=True)
model.compile(loss='hinge', optimizer=optimizer, metrics=['accuracy'])
model.fit(
    train_instances.x,
    train_instances.y,
    batch_size=60,
    epochs=50,
    verbose=1,
    callbacks=[checkpoint, early_stopping],
    validation_data=(valid_instances.x, valid_instances.y),
    shuffle=True
)

The model I use (you can find the implementation here: https://github.com/pexmar/DSCNN_document) has 90 inputs (shared Layers) of dimension 100 x 300 (word2vec embedding layer: 100 words, each has 300 dimensions). I give 12500 training instances and 1000 validation instances to the network.

Force keras mode.fit() to use Multiprocessing

0 Answers