JavaScript strings outside of the BMP

user_330644 0
Viewed 12904

BMP being Basic Multilingual Plane

According to JavaScript: the Good Parts:

JavaScript was built at a time when Unicode was a 16-bit character set, so all characters in JavaScript are 16 bits wide.

This leads me to believe that JavaScript uses UCS-2 (not UTF-16!) and can only handle characters up to U+FFFF.

Further investigation confirms this:

> String.fromCharCode(0x20001);

The fromCharCode method seems to only use the lowest 16 bits when returning the Unicode character. Trying to get U+20001 (CJK unified ideograph 20001) instead returns U+0001.

Question: is it at all possible to handle post-BMP characters in JavaScript?

2011-07-31: slide twelve from Unicode Support Shootout: The Good, The Bad, & the (mostly) Ugly covers issues related to this quite well:

surrogate-pairs astral-plane utf-16 unicode javascript
6 Answers

Using for (c of this) instruction, one can make various computations on a string that contains non-BMP characters. For instance, to compute the string length, and to get the nth character of the string:

String.prototype.magicLength = function()
{
    var c, k;
    k = 0;
    for (c of this) // iterate each char of this
    {
        k++;
    }
    return k;
}

String.prototype.magicCharAt = function(n)
{
    var c, k;
    k = 0;
    for (c of this) // iterate each char of this
    {
        if (k == n) return c + "";
        k++;
    }
    return "";
}
user_24582020

This old topic has now a simple solution in ES6:

Split characters into an array

simple version

[..."⛔"] // ["", "", "", "⛔", "", "", ""]

Then having each one separated you can handle them easily for most common cases.

Credit: DownGoat

Full solution

To overcome special emojis as the one in the comment, one can search for the connection charecter (char code 8205 in UTF-16) and make some modifications. Here is how:

let myStr = "‍‍‍"
let arr = [...myStr]

for (i = arr.length-1; i--; i>= 0) {
    if (arr[i].charCodeAt(0) == 8205) { // special combination character
        arr[i-1] += arr[i] + arr[i+1]; // combine them back to a single emoji 
        arr.splice(i, 2)
    }
}
console.log(arr.length) //3

Haven't found a case where this doesn't work. Comment if you do.

To conclude

it seems that JS uses the 8205 char code to represent UCS-2 characters as a UTF-16 combinations.

user_187626280
Related