Converting between strings and ArrayBuffers

Viewed 554350

Is there a commonly accepted technique for efficiently converting JavaScript strings to ArrayBuffers and vice-versa? Specifically, I'd like to be able to write the contents of an ArrayBuffer to localStorage and to read it back.

28 Answers

Based on the answer of gengkev, I created functions for both ways, because BlobBuilder can handle String and ArrayBuffer:

function string2ArrayBuffer(string, callback) {
    var bb = new BlobBuilder();
    bb.append(string);
    var f = new FileReader();
    f.onload = function(e) {
        callback(e.target.result);
    }
    f.readAsArrayBuffer(bb.getBlob());
}

and

function arrayBuffer2String(buf, callback) {
    var bb = new BlobBuilder();
    bb.append(buf);
    var f = new FileReader();
    f.onload = function(e) {
        callback(e.target.result)
    }
    f.readAsText(bb.getBlob());
}

A simple test:

string2ArrayBuffer("abc",
    function (buf) {
        var uInt8 = new Uint8Array(buf);
        console.log(uInt8); // Returns `Uint8Array { 0=97, 1=98, 2=99}`

        arrayBuffer2String(buf, 
            function (string) {
                console.log(string); // returns "abc"
            }
        )
    }
)

Just

const buffer = thisReturnsBuffers();

const blob = new Blob([buffer], {type: 'text/plain; charset=utf-8'});

blob.text().then(text => console.log(text));

Or

const stringVal = "string here";

const blob = new Blob([stringVal], {type: 'text/plain; charset=utf-8'});

blob.arrayBuffer().then(buffer => console.log(buffer));

Why are you all making this so complicated?

For node.js and also for browsers using https://github.com/feross/buffer

function ab2str(buf: Uint8Array) {
  return Buffer.from(buf).toString('base64');
}
function str2ab(str: string) {
  return new Uint8Array(Buffer.from(str, 'base64'))
}

Note: Solutions here didn't work for me. I need to support node.js and browsers and just serialize UInt8Array to a string. I could serialize it as a number[] but that occupies unnecessary space. With that solution I don't need to worry about encodings since it's base64. Just in case other people struggle with the same problem... My two cents

The following is a working Typescript implementation:

bufferToString(buffer: ArrayBuffer): string {
    return String.fromCharCode.apply(null, Array.from(new Uint16Array(buffer)));
}

stringToBuffer(value: string): ArrayBuffer {
    let buffer = new ArrayBuffer(value.length * 2); // 2 bytes per char
    let view = new Uint16Array(buffer);
    for (let i = 0, length = value.length; i < length; i++) {
        view[i] = value.charCodeAt(i);
    }
    return buffer;
}

I've used this for numerous operations while working with crypto.subtle.

  stringToArrayBuffer(byteString) {
    var byteArray = new Uint8Array(byteString.length);
    for (var i = 0; i < byteString.length; i++) {
      byteArray[i] = byteString.codePointAt(i);
    }
    return byteArray;
  }
  arrayBufferToString(buffer) {
    var byteArray = new Uint8Array(buffer);
    var byteString = '';
    for (var i = 0; i < byteArray.byteLength; i++) {
      byteString += String.fromCodePoint(byteArray[i]);
    }
    return byteString;
  }

Let's say you have an arrayBuffer binaryStr:

let text = String.fromCharCode.apply(null, new Uint8Array(binaryStr));

and then you assign the text to the state.

Recently I also need to do this for one of my project so did a well research and got a result from Google's Developer community which states this in a simple manner:

For ArrayBuffer to String

function ab2str(buf) {
  return String.fromCharCode.apply(null, new Uint16Array(buf));
}
// Here Uint16 can be different like Uinit8/Uint32 depending upon your buffer value type.

For String to ArrayBuffer

function str2ab(str) {
  var buf = new ArrayBuffer(str.length*2); // 2 bytes for each char
  var bufView = new Uint16Array(buf);
  for (var i=0, strLen=str.length; i < strLen; i++) {
    bufView[i] = str.charCodeAt(i);
  }
  return buf;
}
//Same here also for the Uint16Array.

For more in detail reference you can refer this blog by Google.

Yes:

const encstr = (`TextEncoder` in window) ? new TextEncoder().encode(str) : Uint8Array.from(str, c => c.codePointAt(0));

ArrayBuffer -> Buffer -> String(Base64)

Change ArrayBuffer to Buffer and then to String.

Buffer.from(arrBuffer).toString("base64");

Use splat unpacking instead of loops:

arrbuf = new Uint8Array([104, 101, 108, 108, 111])
text = String.fromCharCode(...arrbuf)
console.log(text)

For substrings arrbuf.slice() can be employed.

From emscripten:

function stringToUTF8Array(str, outU8Array, outIdx, maxBytesToWrite) {
  if (!(maxBytesToWrite > 0)) return 0;
  var startIdx = outIdx;
  var endIdx = outIdx + maxBytesToWrite - 1;
  for (var i = 0; i < str.length; ++i) {
    var u = str.charCodeAt(i);
    if (u >= 55296 && u <= 57343) {
      var u1 = str.charCodeAt(++i);
      u = 65536 + ((u & 1023) << 10) | u1 & 1023
    }
    if (u <= 127) {
      if (outIdx >= endIdx) break;
      outU8Array[outIdx++] = u
    } else if (u <= 2047) {
      if (outIdx + 1 >= endIdx) break;
      outU8Array[outIdx++] = 192 | u >> 6;
      outU8Array[outIdx++] = 128 | u & 63
    } else if (u <= 65535) {
      if (outIdx + 2 >= endIdx) break;
      outU8Array[outIdx++] = 224 | u >> 12;
      outU8Array[outIdx++] = 128 | u >> 6 & 63;
      outU8Array[outIdx++] = 128 | u & 63
    } else {
      if (outIdx + 3 >= endIdx) break;
      outU8Array[outIdx++] = 240 | u >> 18;
      outU8Array[outIdx++] = 128 | u >> 12 & 63;
      outU8Array[outIdx++] = 128 | u >> 6 & 63;
      outU8Array[outIdx++] = 128 | u & 63
    }
  }
  outU8Array[outIdx] = 0;
  return outIdx - startIdx
}

Use like:

stringToUTF8Array('abs', new Uint8Array(3), 0, 4);
Related