I am currently creating a webapplication using React.js on the frontend that does image classification. One problem I've run into is being able to make an image tensor using the fromPixels method of Tensorflow.js. The error I recieve is:
Unhandled Rejection (Error): pixels passed to tf.browser.fromPixels() must be either an HTMLVideoElement, HTMLImageElement, HTMLCanvasElement, ImageData in browser, or OffscreenCanvas, ImageData in webworker or {data: Uint32Array, width: number, height: number}, but was String
Also, here is my code:
async onSubmit(e){
e.preventDefault()
const model = await tf.loadLayersModel('http://localhost:81/model/model.json')
const reader = new FileReader()
reader.readAsDataURL(this.state.image)
const tensor = tf.browser.fromPixels(reader.result)
.resizeNearestNeighbor([224,224])
.toFloat()
.expandDims()
const predictions = await model.predict(tensor).data()
console.log(predictions)
const formdata = new FormData()
formdata.append('caption',this.state.caption)//key-value pair
formdata.append('description',this.state.description)
formdata.append('date', this.state.date)
formdata.append('image',this.state.image)
//creating a formdata object
console.log('Image uploaded !')
axios.post('http://localhost:3002/images',formdata)
.then(res=>console.log(res.data))
//window.location = '/results'
//this will go to the 'results' page--where the image is displayed
}
Also, for context, here is my render function:
render(){
return(
<div>
<form onSubmit = {this.onSubmit}>
<div className="form-group">
<label>Caption:</label>
<input type="text"
required
className="form-control"
value={this.state.caption}
onChange={this.onChangeCaption}
/>
</div>
.....{//removed some code not relevant to the question here}
<div className="form-group">
<label>Choose a File</label>
<input type="file"
className="form-control"
onChange={this.onChangeImage}
/>
</div>
<div className="form-group">{//submit button}
<input type="submit" value = "Upload Data" className = "btn btn-primary"/>
</div>
</form>
</div>
)
}
Basically, the onSubmit function is called when the form is submitted and when I am about to make a post request to my REST API.
I have no idea how to preprocess the file object stored in this.state.image. Any ideas?