why does model.predict in tensorflowjs keeps returning the same incorrect output regardless of the tensor given?

Viewed 488

I'm trying to get a keras model i converted to tensorflow js, to work in react native but the model keeps giving bad responses. Did some digging and found realized that the tensor i passed into model.predict is some how being changed causing it to give the same incorrect prediction. Any suggestions would be appreciated. I'm pretty much hard stuck. Code below:

import React, {useState, useEffect} from 'react';
import {View, Text, Button} from 'react-native';
import * as tf from '@tensorflow/tfjs';
import {
  bundleResourceIO
} from '@tensorflow/tfjs-react-native';
import * as mobilenet from '@tensorflow-models/mobilenet';

function thing() {
  const [model, setModel] = useState(null);
  const [tensor, setTensor] = useState(null); 
  
  async function loadModel() {
    const modelJson = require('./assets/model.json');
    const weight = require('./assets/group1-shard1of1.bin');
    const backend = await tf.ready();
    const item = await tf.loadLayersModel(
        bundleResourceIO(modelJson, weight)
    );
    const tfTensor = tf.tensor([[0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]);
    
    setModel(item);
    setTensor(tfTensor);
  }

  useEffect(() => {
    loadModel();
  }, []);

  async function test() {
    if(tensor !== null && model !== null) {
      const result = await model.predict(tensor);
      console.log(result.dataSync())
    }
  }

  return (
    <View>
      <Button
        onPress={test}
        title="click"
        color="#841584"
        accessibilityLabel="Learn more about this purple button"
      />

    </View>
  );
}

export default thing;


1 Answers

Just like changing a single pixel in an image doesn't change the image, changing one bit in an array doesn't significantly adjust the prediction.

I ran mobilenet on a black 224x224 image and it predicted class 819 (whatever that is). Then I changed the top-left pixel to white and re-ran mobilenet and it still classifies as class 819.

See example code here

Changing a single bit does not have a cascading effect like a hash function. Mobilenet, by its nature is resilient to noise.

Related