How to load an onnx model using ONNX.js

user_18282629 0
Viewed 434

I am trying to import an ONNX model using onnxjs, but I get the below error:

Uncaught (in promise) TypeError: cannot resolve operator 'Cast' with opsets: ai.onnx v11

Below shows a code snippet from my html file.

<html>
  <head> </head>
  <body>
    <!-- Load ONNX.js -->
    <script src="https://cdn.jsdelivr.net/npm/onnxjs/dist/onnx.min.js"></script>
    <!-- Code that consume ONNX.js -->
    <script>
        async function test(){
            const sess = new onnx.InferenceSession()
            await sess.loadModel('./onnx_model.onnx')
            }
        test()
    </script>
  </body>
</html>

How to solve this?

onnx node.js machine-learning artificial-intelligence javascript
1 Answers

This will load Resnet 50 model

const sess = new onnx.InferenceSession()
async function test(){
  console.time("loading model")
  await sess.loadModel('https://microsoft.github.io/onnxjs-demo/resnet50v2.onnx')
  console.timeEnd("loading model")

  console.log("model loaded");
}
document.querySelector('#load').addEventListener('click', test);
<script src="https://cdn.jsdelivr.net/npm/onnxjs/dist/onnx.min.js"></script>

<button id="load">load model</button>

The message suggest something to do with the Cast operator not being supported by opset 11, maybe you want to use Cast-9. Maybe you have to generate a new model.

Edit

Your model loads using onnxruntime python

sess = onnxruntime.InferenceSession('../../Downloads/onnx_model.onnx');
{i.name: i.shape for i in sess.get_inputs()}
{o.name: o.shape for o in sess.get_outputs()}

{'input_ids': ['batch', 'sequence'],
 'attention_mask': ['batch', 'sequence'],
 'token_type_ids': ['batch', 'sequence']}

{'output_0': ['batch', 2]}

An Idea

You will probably have to debug it your self, hopefully the only problem is the cast operator.

You start looking here the operator support for onnxjs, and rewrite parts of the model where the operator appears.

For instance, the Cast operator appears only once, you can locate it as follows

import onnx
model = onnx.load('../../Downloads/onnx_model.onnx')
for node in model.graph.node:
    if 'cast' in node.op_type.lower():
        print(node.name, node.op_type)

That will print

Cast_2 Cast

Using https://netron.app/ (Or the desktop version) you can see that it is netron-view

So you should simply rewrite how your attention mask is processed in the model, a possible solution would be to let unsqueeze and cast operations outside the model.

user_127503530
Related