How to use the Universal Sentence Encoder Tensorflow model with ML.NET?

user_1276129 0
Viewed 212

I have C# code which invokes Python to compute compute Universal Sentence Encoder (USE) embeddings. However, I figured that it would probably be much cleaner to do this using ML.NET (and thus without involving Python). Thing is that I haven't worked with Tensorflow despite doing simple stuff like the above (and I don't have time currently to dive into the details, although I'd like to at some point in time), and the ML.NET abstraction is not as high as I had hoped for. If somebody could jump in and fill in the missing pieces, I would highly appreciate that (and probably package and share a C# USE wrapper on GitHub).

Here's the prerequisites:

  • I'm using this model
  • Loading the model works fine. I looked into the model's schema and input schema as provided by the C# API, but this is beyond me for now. Here's the input schema (as printed from C#):
serving_default_inputs: Vector<String> {TensorflowOperatorType}
saver_filename: Vector<String> {TensorflowOperatorType}
  • And here's an excerpt of the (rather large) complete schema - I think that I have skipped the model internals (I guess that I only need to figure out for now how to feed data into the model and get the result back, so I hope that I have caught the right parts):
save_counter/Read/ReadVariableOp: Vector<Int64> {TensorflowOperatorType, TensorflowUpstreamOperators}
Embeddings/sharded_0/Read/ReadVariableOp: Vector<Single, 13334, 320> {TensorflowOperatorType, TensorflowUpstreamOperators}
... (28 of these)
Embeddings/sharded_29/Read/ReadVariableOp: Vector<Single, 13334, 320> {TensorflowOperatorType, TensorflowUpstreamOperators}
EncoderDNN/DNN/ResidualHidden_0/dense/kernel/part_0/Read/ReadVariableOp: Vector<Single, 11, 320> {TensorflowOperatorType, TensorflowUpstreamOperators}
... (lot's of apparently internal stuff)
EncoderTransformer/hidden_layers/tanh_layer_0/dense/bias/part_29/Read/ReadVariableOp: Vector<Single, 17> {TensorflowOperatorType, TensorflowUpstreamOperators}
StatefulPartitionedCall: Vector<Single> {TensorflowOperatorType, TensorflowUpstreamOperators}
Const: Vector<String> {TensorflowOperatorType}
serving_default_inputs: Vector<String> {TensorflowOperatorType}
StatefulPartitionedCall_1: Vector<Single, 512> {TensorflowOperatorType, TensorflowUpstreamOperators}
saver_filename: Vector<String> {TensorflowOperatorType}
StatefulPartitionedCall_2: Vector<String> {TensorflowOperatorType, TensorflowUpstreamOperators}
StatefulPartitionedCall_3: Vector<String> {TensorflowOperatorType, TensorflowUpstreamOperators}

Finally, here's my rather naive C# code. Note that from the "guessing" part on, I have no clue what I'm doing at all - I went for Const since USE produces a constant-size array of floats, and for serving_default_inputs since it has inputs in its name - the rest is merely for demonstrating what I want to do. Excuse my ignorance :-)

using Microsoft.ML;
using Microsoft.ML.Transforms;
using System.Collections.Generic;
using System.IO;
using System.Reflection;

namespace ML.UniversalSentenceEncoder
{
    public class UniversalSentenceEncoder
    {
        public double[] ComputeEmbedding(string text)
        {
            string modelDir = Path.Combine(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location), "ML", "UniversalSentenceEncoder", "Model");
            MLContext mlContext = new MLContext();
            TensorFlowModel model = mlContext.Model.LoadTensorFlowModel(modelDir);

            // here's where my guessing started :-)
            IEstimator<ITransformer> estimator = model.ScoreTensorFlowModel("Const", "serving_default_inputs");
            IDataView dataView = mlContext.Data.LoadFromEnumerable(new List<string>());
            ITransformer transformer = estimator.Fit(dataView);
            PredictionEngine<string, List<double>> engine = mlContext.Model.CreatePredictionEngine<string, List<double>>(transformer);
            return engine.Predict(text).ToArray();
        }

    }
}

In case this is useful (which I kind of doubt): The above code throws an exception at the ITransformer transformer... line: System.ArgumentOutOfRangeException: 'Could not find source column 'serving_default_inputs' Arg_ParamName_Name'

ml.net tensorflow c#
1 Answers

I think your issue is that the schema of your IDataView is of type List<string>. As a result, it can't find the column serving_default_inputs. Try creating a class with containing the serving_default_inputs column. You'll also have to do the same for the output when working with PredictionEngine. While not the same model, here's a tutorial that uses a TF model for text classification that might help. https://docs.microsoft.com/dotnet/machine-learning/tutorials/text-classification-tf

using Microsoft.ML;
using Microsoft.ML.Transforms;
using System.Collections.Generic;
using System.IO;
using System.Reflection;

namespace ML.UniversalSentenceEncoder
{
    public class USEInput
    {
        public string serving_default_inputs {get;set;}
    }

    public class USEOutput
    {
        public float[] Const {get;set;}
    }

    public class UniversalSentenceEncoder
    {
        public double[] ComputeEmbedding(string text)
        {
            string modelDir = Path.Combine(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location), "ML", "UniversalSentenceEncoder", "Model");
            MLContext mlContext = new MLContext();
            TensorFlowModel model = mlContext.Model.LoadTensorFlowModel(modelDir);

            // here's where my guessing started :-)
            IEstimator<ITransformer> estimator = model.ScoreTensorFlowModel("Const", "serving_default_inputs");
            IDataView dataView = mlContext.Data.LoadFromEnumerable(new List<USEInput>());
            ITransformer transformer = estimator.Fit(dataView);
            PredictionEngine<UseInput,  USEOutput> engine = mlContext.Model.CreatePredictionEngine<USEInput, USEOutput>(transformer);
            return engine.Predict(new UseInput {serving_default_inputs 
 = text }).ToArray();
        }

    }
}
user_151841840
Related