MInimum working example tensorflow serving client

Question

I am working on the basic Tensorflow Serving example. I am following the MNIST example, except instead of classification I want to use a numpy array to predict another numpy array.

To do this I first trained my neural network

x = tf.placeholder("float", [None, n_input],name ="input_values")

weights = {
    'encoder_h1': tf.Variable(tf.random_normal([n_input, n_hidden_1])),
    'encoder_h2': tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2])),
    'encoder_h3': tf.Variable(tf.random_normal([n_hidden_2, n_hidden_3])),
    'decoder_h1': tf.Variable(tf.random_normal([n_hidden_3, n_hidden_2])),
    'decoder_h2': tf.Variable(tf.random_normal([n_hidden_2, n_hidden_1])),
    'decoder_h3': tf.Variable(tf.random_normal([n_hidden_1, n_input])),
}
biases = {
    'encoder_b1': tf.Variable(tf.random_normal([n_hidden_1])),
    'encoder_b2': tf.Variable(tf.random_normal([n_hidden_2])),
    'encoder_b3': tf.Variable(tf.random_normal([n_hidden_3])),
    'decoder_b1': tf.Variable(tf.random_normal([n_hidden_2])),
    'decoder_b2': tf.Variable(tf.random_normal([n_hidden_1])),
    'decoder_b3': tf.Variable(tf.random_normal([n_input])),
}

# Building the encoder
def encoder(x):
    # Encoder Hidden layer with sigmoid activation #1
    layer_1 = tf.nn.tanh(tf.matmul(x, weights['encoder_h1'])+biases['encoder_b1'])
    print(layer_1.shape)
    # Decoder Hidden layer with sigmoid activation #2
    layer_2 = tf.nn.tanh(tf.matmul(layer_1, weights['encoder_h2'])+biases['encoder_b2'])
    print(layer_2.shape)
    # Layer 3
    layer_3 = tf.nn.tanh(tf.matmul(layer_2, weights['encoder_h3'])+biases['encoder_b3'])
    print(layer_3.shape)
    return layer_3


# Building the decoder
def decoder(x):
    # Encoder Hidden layer with sigmoid activation #1
    layer_1 = tf.nn.tanh(tf.matmul(x, weights['decoder_h1'])+biases['decoder_b1'])
    print(layer_1.shape)
    # Decoder Hidden layer with sigmoid activation #2
    layer_2 = tf.nn.tanh(tf.matmul(layer_1, weights['decoder_h2'])+biases['decoder_b2'])
    # Layer 3
    layer_3 = tf.nn.tanh(tf.matmul(layer_2, weights['decoder_h3'])+biases['decoder_b3'])
    return layer_3

# Construct model
encoder_op = encoder(x)
decoder_op = decoder(encoder_op)

# Prediction
y = decoder_op



# Objective functions
y_ = tf.placeholder("float", [None,n_input],name="predict")

Next as someone suggested here I saved have my network like so..

import os
import sys

from tensorflow.python.saved_model import builder as saved_model_builder
from tensorflow.python.saved_model import utils
from tensorflow.python.saved_model import tag_constants, signature_constants
from tensorflow.python.saved_model.signature_def_utils_impl import     build_signature_def, predict_signature_def
from tensorflow.contrib.session_bundle import exporter

with tf.Session() as sess:
# Initialize variables
    sess.run(init)

    # Restore model weights from previously saved model
    saver.restore(sess, model_path)
    print("Model restored from file: %s" % save_path)

    export_path = '/tmp/AE_model/6'
    print('Exporting trained model to', export_path)
    builder = tf.saved_model.builder.SavedModelBuilder(export_path)


    signature = predict_signature_def(inputs={'inputs': x},
                                  outputs={'outputs': y})

    builder.add_meta_graph_and_variables(sess=sess,
                                         tags=[tag_constants.SERVING],
                                         signature_def_map={'predict': signature})

    builder.save()


    print 'Done exporting!'

Next I follow the instructions to run my server on localhost:9000

bazel build //tensorflow_serving/model_servers:tensorflow_model_server

I set up the server

bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server --port=9000 --model_base_path=/tmp/AE_model/

THE PROBLEM

Now I want to write a program so I can pass Mat vectors from a C++ program in eclipse(I use a LOT of libraries) to my server so I can make some sort of predictions.

I first thought about using the inception_client.cc as reference. However, it seems that I need Bazel to compile it because I cannot find prediction_service.grpc.pb.h anywhere :(

so it seems that my only other option is to call a script using python I get the following output:

Any help with this problem would be appreciated.

Thank you.

EDIT:

I reinstalled protobuf and grpc and ran the commands as suggested:

My command was a little different and I had to use it outside my serving folder(In Ubuntu 14.04).

sudo protoc -I=serving -I serving/tensorflow --grpc_out=. --plugin=protoc-gen-grpc=`which grpc_cpp_plugin` serving/tensorflow_serving/apis/*.proto

This generated the .gprc.pb.h files and I dragged them into the /apis/ folder and the errors went down. Now I get the error

/tensorflow/third_party/eigen3/unsupported/Eigen/CXX11/Tensor:1:42: fatal error: unsupported/Eigen/CXX11/Tensor: No such file or directory

Even though this file does exist. Any suggestions are appreciated.

Thank you @subzero!

EDIT 2

I was able to solve the problem with Eigen by updating to the newest Eigen version and building from source. Next I pointed to the /usr/local/include/eigen3/

Afterwards I had issues with the tensorflow libraries. These I resolved by generating the libtensorflow_cc.so library using lababidi's suggestion. https://github.com/tensorflow/tensorflow/issues/2412

I have one last issue. Everything seems to be fine except I get the error:

undefined reference to`tensorflow::serving::PredictRequest::~PredictRequest()'

It seems that I am missing either a linker or a library. Does anyone know what I am missing???

MInimum working example tensorflow serving client

THE PROBLEM

EDIT:

EDIT 2

3 Answers