Initializing variables in joined tensorflow graphs

Question

I am trying to join several Tensorflow models on separate graphs into a single model on a single graph (I imagine typical scenarios to be Transfer Learning Models, Ensembles etc. that needs to be frozen).

I am able to connect the two graphs by using tf.import_graph_def. However, when one of the original graphs contains a variable, I am unable to inject values into it on the new graph. The assign-operations are copied over as part of the graph, and if I run these manually I get the expected results. (Uncommenting the sess.run-call in the fourth with graph...-block)

import tensorflow as tf

graph1 = tf.Graph()
with graph1.as_default():
    with tf.Session(graph=graph1) as sess:
        var1 = tf.Variable([1, 1, 1], name='var1')
        tensor1 = tf.multiply(var1, 2, name='tensor1')

graph2 = tf.Graph()
with graph2.as_default():
    with tf.Session(graph=graph2) as sess:
        placeholder = tf.placeholder(tf.int32, [3], name='placeholder')
        tensor2 = tf.multiply(placeholder, 2, name='tensor2')

graph3 = tf.Graph()
with graph3.as_default():
    with tf.Session(graph=graph3) as sess:
        graph1out, = tf.import_graph_def(graph1.as_graph_def(), name='graph1', return_elements=['tensor1:0'])
        tf.import_graph_def(graph2.as_graph_def(), name='graph2', input_map={'placeholder:0': graph1out})

with graph3.as_default():
    with tf.Session(graph=graph3) as sess:
        sess.run(tf.global_variables_initializer())
        #sess.run(graph3.get_tensor_by_name('graph1/var1/Assign:0'))
        tensor = graph3.get_tensor_by_name('graph2/tensor2:0')
        result = sess.run(tensor)

Running this snippet as-is yields:

FailedPreconditionError (see above for traceback): Attempting to use uninitialized value graph1/var1

Manually running the assign-ops is a very little generic approach which will be messy to write as part of a general function. It also requires these assign-calls to be made every time i reinitialize a session with the graph.

Is there a more generic way of doing this? For instance by creating new variables based off the old ones and latching them on to the existing structure?

If not, is there a way of wrapping the assign-calls? So at least I can run a single general init-operation instead of an unknown number of specific ones for each case?

Initializing variables in joined tensorflow graphs

1 Answers