validation during training of Estimator

EDIT: As pointed out in the comments, this feels like the right thing to do, but will reinitialize the weights every time it's evaluated, which makes it pretty much useless...

I ended up being able to monitor my validation error (which is what I understand you are trying to do) using the train_and_evaluate function. The EvalSpec object you have to use has parameters start_delay_secs and throttle_secs to define the frequency at which the error (or whatever you have defined in your estimator's EVAL mode) will be computed.

My code looks somewhat like

classifier = tf.estimator.Estimator(
    model_fn=model_fn,
    model_dir=model_dir,
    params=params)

train_spec = tf.estimator.TrainSpec(
    input_fn = input_fn,
)

eval_spec = tf.estimator.EvalSpec(
    input_fn = valid_input_fn,
    throttle_secs=120,
    start_delay_secs=120,
)

tf.estimator.train_and_evaluate(
    classifier,
    train_spec,
    eval_spec
)