Tensorflow variable reuse

Question

I've built my LSTM model. Ideally I want to use reuse the variables to define a test LSTM model later.

with tf.variable_scope('lstm_model') as scope:
    # Define LSTM Model
    lstm_model = LSTM_Model(rnn_size, batch_size, learning_rate,
                     training_seq_len, vocab_size)
    scope.reuse_variables()
    test_lstm_model = LSTM_Model(rnn_size, batch_size, learning_rate,
                     training_seq_len, vocab_size, infer=True)

The code above gives me an error

Variable lstm_model/lstm_vars/W already exists, disallowed. Did you mean to set reuse=True in VarScope?

If I set the reuse=True as shown in the below code block

with tf.variable_scope('lstm_model', reuse=True) as scope:

I get a different error

Variable lstm_model/lstm_model/lstm_vars/W/Adam/ does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=None in VarScope?

As a reference I've appended the relevant model code below. The corresponding section in the LSTM model where I have the weights

with tf.variable_scope('lstm_vars'):
    # Softmax Output Weights
    W = tf.get_variable('W', [self.rnn_size, self.vocab_size], tf.float32, tf.random_normal_initializer())

The corresponding section where I have my Adam optimizer:

optimizer = tf.train.AdamOptimizer(self.learning_rate)

Tensorflow variable reuse

2 Answers