Is there a way to create a tensor from a File object using the fromPixels method?

Viewed 275

I am currently creating a webapplication using React.js on the frontend that does image classification. One problem I've run into is being able to make an image tensor using the fromPixels method of Tensorflow.js. The error I recieve is:

Unhandled Rejection (Error): pixels passed to tf.browser.fromPixels() must be either an HTMLVideoElement, HTMLImageElement, HTMLCanvasElement, ImageData in browser, or OffscreenCanvas, ImageData in webworker or {data: Uint32Array, width: number, height: number}, but was String

Also, here is my code:

async onSubmit(e){
e.preventDefault()

const model = await tf.loadLayersModel('http://localhost:81/model/model.json')

const reader = new FileReader()


reader.readAsDataURL(this.state.image)

 
const tensor =  tf.browser.fromPixels(reader.result)
      .resizeNearestNeighbor([224,224])
      .toFloat()
      .expandDims()

const predictions = await model.predict(tensor).data()

console.log(predictions)



const formdata = new FormData()
formdata.append('caption',this.state.caption)//key-value pair
formdata.append('description',this.state.description)
formdata.append('date', this.state.date)
formdata.append('image',this.state.image)


//creating  a formdata object

console.log('Image uploaded !')

axios.post('http://localhost:3002/images',formdata)
  .then(res=>console.log(res.data))


//window.location = '/results'
//this will go to the 'results' page--where the image is displayed

}

Also, for context, here is my render function:

render(){
return(
  
  <div>
    <form onSubmit = {this.onSubmit}>
      <div className="form-group">
        <label>Caption:</label>
        <input type="text"
            required
            className="form-control"
            value={this.state.caption}
            onChange={this.onChangeCaption}
            />

      </div>

      .....{//removed some code not relevant to the question here}

      <div className="form-group">
        <label>Choose a File</label>
        <input type="file" 
          className="form-control"
          onChange={this.onChangeImage}
          />

      </div>

      <div className="form-group">{//submit button}
        <input type="submit" value = "Upload Data" className = "btn btn-primary"/>
      </div>



      </form>
   </div>

  
    )
  }

Basically, the onSubmit function is called when the form is submitted and when I am about to make a post request to my REST API.

I have no idea how to preprocess the file object stored in this.state.image. Any ideas?

1 Answers

You can use FileReader to create an img element and load it from there:

const image = document.createElement("img");

reader.onload = async (event) => {
  image.src = event.target.result
  document.body.append(image);

  image.onload = async () => {
    tf.browser.fromPixels(image);
  }
}
Related