How do I pass a file/blob from JavaScript to emscripten/WebAssembly (C++)?

user_11942268 0
Viewed 2048

I'm writing a WebExtension that uses C++ code compiled with emscripten. The WebExtension downloads files which I want to process inside the C++ code. I'm aware of the File System API and I think I read most of it, but I don't get it to work - making a downloaded file accessible in emscripten.

This is the relevant JavaScript part of my WebExtension:

// Download a file
let blob = await fetch('https://stackoverflow.com/favicon.ico').then(response => {
  if (!response.ok) {
    return null;
  }     
  return response.blob();
});

// Setup FS API
FS.mkdir('/working');
FS.mount(IDBFS, {}, '/working');
FS.syncfs(true, function(err) {
  if (err) {
    console.error('Error: ' + err);
  }
});

// Store the file "somehow"
let filename = 'favicon.ico';
// ???

// Call WebAssembly/C++ to process the file
Module.processFile(filename);

The directory is created, what can be seen, when inspecting the Web Storage of the browser. If I understand the File System API correctly, I have to "somehow" write my data to a file inside /working. Then, I should be able to call a function of my C++ code (from JavaScript) and open that file as if there was a directory called 'working' at the root, containing the file. The call of the C++ function works (I can print the provided filename).

But how do I add the file (currently a blob) to that directory?

C++ code:

#include "emscripten/bind.h"

using namespace emscripten;

std::string processFile(std::string filename)
{
    // open and process the file
}

EMSCRIPTEN_BINDINGS(my_module)
{
    function("processFile", &processFile);
}
emscripten webassembly javascript
1 Answers

It turned out, that I was mixing some things up while trying different methods, and I was also misinterpreting my debugging tools. So the easiest way to accomplish this task (without using IDBFS) is:

JS:

// Download a file
let blob = await fetch('https://stackoverflow.com/favicon.ico').then(response => {
  if (!response.ok) {
    return null;
  }     
  return response.blob();
});

// Convert blob to Uint8Array (more abstract: ArrayBufferView)
let data = new Uint8Array(await blob.arrayBuffer());

// Store the file
let filename = 'favicon.ico';
let stream = FS.open(filename, 'w+');
FS.write(stream, data, 0, data.length, 0);
FS.close(stream);

// Call WebAssembly/C++ to process the file
console.log(Module.processFile(filename));

C++:

#include "emscripten/bind.h"
#include <fstream>

using namespace emscripten;

std::string processFile(std::string filename)
{
    std::fstream fs;
    fs.open (filename, std::fstream::in | std::fstream::binary);
    if (fs) {
        fs.close();
        return "File '" + filename + "' exists!";
    } else {
        return "File '" + filename + "' does NOT exist!";
    }
}

EMSCRIPTEN_BINDINGS(my_module)
{
    function("processFile", &processFile);
}

If you want to do it with IDBFS, you can do it like this:

// Download a file
let blob = await fetch('https://stackoverflow.com/favicon.ico').then(response => {
  if (!response.ok) {
    return null;
  }     
  return response.blob();
});

// Convert blob to Uint8Array (more abstract: ArrayBufferView)
let data = new Uint8Array(await blob.arrayBuffer());

// Setup FS API
FS.mkdir('/persist');
FS.mount(IDBFS, {}, '/persist');
// Load persistant files (sync from IDBFS to MEMFS), will do nothing on first run
FS.syncfs(true, function(err) {
  if (err) {
    console.error('Error: ' + err);
  }
});
FS.chdir('/persist');

// Store the file
let filename = 'favicon.ico';
let stream = FS.open(filename, 'w+');
FS.write(stream, data, 0, data.length, 0);
FS.close(stream);

// Persist the changes (sync from MEMFS to IDBFS)
FS.syncfs(false, function(err) {
  if (err) {
    console.error('Error: ' + err);
  }
});
// NOW you will be able to see the file in your browser's IndexedDB section of the web storage inspector!

// Call WebAssembly/C++ to process the file
console.log(Module.processFile(filename));

Notes:

  • When using FS.chdir() in the JS world to change the directory, this also changes the working directory in the C++ world. So respect that, when working with relative paths.

  • When working with IDBFS instead of MEMFS, you are actually still working with MEMFS and just have the opportunity to sync data from or to IDBFS on demand. But all your work is still done with MEMFS. I would consider IDBFS as an add-on to MEMFS. Didn't read that directly from the docs.

user_119422680
Related