Working with tensorflow, I am trying to share identical weights from identical networks in different variable scopes to save memory. However there seems to be no simple way to do this. I have prepared a small code sample to illustrate on a smaller scale what I would like to do with a larger subnetwork:
import tensorflow as tf
graph = tf.Graph()
with graph.as_default():
with tf.variable_scope("super_scope_one"):
scope1 = tf.variable_scope("sub_scope_one")
with scope1:
number_one = tf.get_variable("number_one", shape=[1],
initializer=tf.ones_initializer)
with tf.variable_scope("super_scope_two"):
with tf.variable_scope("sub_scope_one", reuse=True) as scope2:
# Here is the problem.
# scope1.reuse_variables() # this crashes too if reuse=None.
number_one = tf.get_variable("number_one", shape=[1])
with tf.variable_scope("sub_scope_two"):
number_two = tf.get_variable("number_two", shape=[1],
initializer=tf.ones_initializer)
number_three = number_one + number_two
init_op = tf.global_variables_initializer()
with tf.Session(graph=graph):
init_op.run()
print(number_three.eval())
Is there a way to share the variables in the two sub scopes, without removing the scopes above? And if not is there a good reason why this would be a bad Idea?