How to reconstruct the PNG returned by canvas.toBlob() in NodeJs

Viewed 269

Let's say I have a canvas in an HTML file (nothing special). And I'm using Puppeteer to draw something on the canvas in the browser context and then pass it to NodeJs context where I save it as a PNG file using OpenCV.

Since binary data cannot be transferred between the two contexts (the browser and Node), I have to convert the binary data to a string and reconstruct the binary data in Node. The following code does exactly that:

const puppeteer = require('puppeteer');
const cv = require('opencv4nodejs');

const pup = puppeteer.launch({
    headless: false,
    args: [
        '--use-gl=swiftshader',
        '--no-sandbox',
        '--enable-surface-synchronization',
        '--disable-web-security',
    ]
}).then(async browser => {
    const page = (await browser.pages())[0];
    page.on('console', msg => console.log(msg.text()));
    page.on("pageerror", function(err) {
        const theTempValue = err.toString();
        console.log("Page error: " + theTempValue);
    });
    page.on("error", function (err) {
        const theTempValue = err.toString();
        console.log("Error: " + theTempValue);
    });

    await page.exposeFunction('_setupCompleted', async () => {
        await page.evaluate(async () => {
            const c = document.getElementById("myCanvas");

            const dataUrl = c.toDataURL("image/png");
            const txt = await new Promise(resolve => c.toBlob((blb) => blb.text().then(resolve), "image/png"));
            await window._saveMat(dataUrl, txt);
        });

        await browser.close();
    });

    await page.exposeFunction('_saveMat', async (dataUrl, txt) => {
        dataUrl = dataUrl.replace(/^data:image\/png;base64,/, "");
        const dataUrlBuffer = Buffer.from(dataUrl, 'base64');
        const dataUrlMat = cv.imdecode(dataUrlBuffer, -1);
        cv.imwrite(`./dataUrl.png`, dataUrlMat);


        const txtBuffer = Buffer.from(txt, 'utf8');
        const txtMat = cv.imdecode(txtBuffer, -1);
        cv.imwrite(`./txtMat.png`, txtMat);
    });

    await page.goto('http://127.0.0.1:8887/index.html', {
        waitUntil: 'networkidle2',
    });
});

I the given code, I transfer the PNG in two ways. First, I use the toDataURL method of the canvas. This method encodes the binary data of the canvas using base64 and prepends some signature to it as well. Once received, the code in Node will strip the signature and decodes the base64. This part of the code works perfectly and it saves the PNG just fine.

The other approach uses the toBlob method of the canvas. This method returns a Blob which I'll call its text method to convert it to a string. On the Node side, I'll try to reconstruct a buffer out of the transferred string. This part is failing. Apparently, the reconstructed buffer is not a valid PNG and the imdecode method fails to create a mat object based on it.

My question is, how can I reconstruct the same buffer as I get from toDataURL using the text I get from the toBlob method in NodeJs?

0 Answers
Related