Recommended approach for project-specific keras config?

Question

My goal is to maintain keras config on a per-project basis, e.g. one project prefers the theano backend, and another project prefers the tensorflow backend. As a bonus, I would like to share this config with other developers relatively seamlessly.

Here are a few ideas:

Can keras config be managed by/within a virtual environment?
Should I use something like dotenv or autoenv to manage some shared environment configuration (via the KERAS_BACKEND environment variable)?
Should keras be updated to look for a .keras/keras.json file in the working tree before using the version in $HOME?

user_6000165 · Answer

Can keras config be managed by/within a virtual environment?

The basic config parameters (like backend, floating point precision) are managed in the $KERAS_HOME/keras.json file. You could create a keras.json per Anaconda/virtual environment and set KERAS_HOME to point to a specific file as you load it.

Alternatively, these variables can be set during runtime through Keras backend, which would override the value in the config file:

from keras import backend as K
K.set_floatx('float16')

Depending on the Keras backend, there are other parameters one can configure. With tensorflow backend, for instance, one might want to configure tf.ConfigProto. One practical way to do it is during the runtime like:

import os 
if os.environ['KERAS_BACKEND'] == 'tensorflow':
    import tensorflow as tf
    from keras.backend.tensorflow_backend import set_session
    gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.95, allow_growth=True)
    config = tf.ConfigProto(gpu_options=gpu_options)
    set_session(tf.Session(config=config))

See config.proto for what can be configured.

Should I use something like dotenv or autoenv to manage some shared environment configuration (via the KERAS_BACKEND environment variable)?

It is definitely not a must, one could live with os.environ and get/set methods available in Keras to modify the variables.

Should keras be updated to look for a .keras/keras.json file in the working tree before using the version in $HOME?

It is possible to point to a custom location of the keras.json config file by changing the KERAS_HOME env variable or launching your application like:

env KERAS_HOME= python keras_app.py

Recommended approach for project-specific keras config?

1 Answers