Adding metdata to tensorflow lite file

Viewed 3413

I have retrained a MobileNet model with tweaks in the model and custom outputs in TensorFlow. I have to run the model on Android using Google ML KIT but the problem is it requires metadata. But whenever I go through he process it is giving me an error:

ValueError: File, '/content/drive/My Drive/labels.txt', is recorded in the metadata, but has not been loaded into the populator.

Here is my code for adding metadata:

model_metadata=_metadata_fb.ModelMetadataT()
model_metadata.name="MobileNet_with_Metadata"
model_metadata.description="This model is trained on plant village leaf disease dataset so that it can be used for detectiong crop diseases"
model_metadata.version="v1.0.0.0"
model_metadata.author="open-source"
model_metadata.license=("Apache License. Version 2.0 "
                      "http://www.apache.org/licenses/LICENSE-2.0.")
input_metadata=_metadata_fb.TensorMetadataT()
output_metadata=_metadata_fb.TensorMetadataT()
input_metadata.name="image"
input_metadata.description="input_meta.description = (
    "Input image to be classified. The expected image is {0} x {1}, with "
    "three channels (red, blue, and green) per pixel. Each value in the "
    "tensor is a single byte between 0 and 1.".format(256, 256))"
input_normalization = _metadata_fb.ProcessUnitT()
input_normalization.optionsType = (
    _metadata_fb.ProcessUnitOptions.NormalizationOptions)
input_normalization.options = _metadata_fb.NormalizationOptionsT()
input_normalization.options.mean = [127.5]
input_normalization.options.std = [127.5]

input_metadata.processUnits = [input_normalization]
input_stats = _metadata_fb.StatsT()
input_stats.max = [255]
input_stats.min = [0]
input_metadata.stats = input_stats
output_metadata.name="Probability"
output_metadata.description="Probabbility of 50 classes"
output_stats = _metadata_fb.StatsT()
output_stats.max = [1.0]
output_stats.min = [0.0]
output_metadata.stats = output_stats
label_file = _metadata_fb.AssociatedFileT()
label_file.name = "/content/drive/My Drive/labels.txt"
label_file.description = "Labels for objects that the model can recognize."
label_file.type = _metadata_fb.AssociatedFileType.TENSOR_AXIS_LABELS
output_metadata.associatedFiles = [label_file]
subgraph = _metadata_fb.SubGraphMetadataT()
subgraph.inputTensorMetadata = [input_metadata]
subgraph.outputTensorMetadata = [output_metadata]
model_metadata.subgraphMetadata = [subgraph]

b = flatbuffers.Builder(0)
b.Finish(
    model_metadata.Pack(b),
    _metadata.MetadataPopulator.METADATA_FILE_IDENTIFIER)
metadata_buf = b.Output()
populator = _metadata.MetadataPopulator.with_model_file("/content/drive/My Drive/MobileNet_Model_latest.tflite")
populator.load_metadata_buffer(metadata_buf)
populator.load_associated_files(["/content/drive/My Drive/labels.txt"])
populator.populate()

I am doing this for first time and I couldn't get any proper help or documentation. Also, can anyone tell me properly on how to add meta data to the tflite model?

I have referred this link: Add Metadata to TensorFlow

2 Answers

I have faced this issue before while trying to deploy TF2 object detection models to mlkit vision sample. If I remember correctly it was trying to read labels from tflite metadata instead of the labelmap in assets folder.

There are existing issues on github for this metadata problem. Seems like it is not yet fixed as others pointed out. There are a few ways to solve it. Have a look at these issues,

https://github.com/tensorflow/models/issues/9341

https://github.com/tensorflow/tensorflow/issues/43583

Edit: The way shown below is not correct for adding metadata outputs. Correct way should in links above.

For adding metadata to object detection model have a look at these,

https://nbviewer.jupyter.org/github/quickgrid/CodeLab/blob/master/tensorflow/TFlite_Object_Detection_Custom_Model_Export_With_Metadata_TF1.ipynb

https://nbviewer.jupyter.org/github/quickgrid/CodeLab/blob/master/tensorflow/TFlite_Object_Detection_Custom_Model_Export_With_Metadata_TF2.ipynb

Related