How do I remove / replace Unicode Characters in Node16

user_3165626 0
Viewed 716

I've got a file that I read into a JSON Object

{
    "city": "Delicias",
    "address": "FRANCISCO DOMÍN\u0002GUEZ 9"
}

We use this address to pass it to Google maps API to get latitude & longitude but Google maps api isn't happy about non utf-8 characters like or \u0002

I've searched all over and can't find a solution that works in Node16

I can get rid of with JSON.stringify(jsonObject).replace(/[^\x00-\x7F]/g, "")

But nothing I've tried seems to get rid of \u0002

Things I've tried

const escapeUnicode = (str) => {
        return str.replace(/[\u00A0-\uffff]/gu, function (c) {
            return "\\u" + ("000" + c.charCodeAt().toString(16)).slice(-4);
        });
    };

JSON.parse(JSON.stringify(jsonObject));
JSON.parse(escapeUnicode(JSON.stringify(jsonObject)));
JSON.parse(JSON.stringify(jsonObject).replace(/[^ -~]+/g, ""));
JSON.parse(decodeURIComponent(JSON.stringify(jsonObject)));

I've tried all of the above, and a bunch of different regex .replace() none seem to do the trick,

node.js json javascript
1 Answers

After a lot of searching and trying I figured this one possible way to get rid of unicodes and replace them with the proper utf-8 character.

let jsonObj = {
    "city": "Delicias",
    "address": "FRANCISCO DOMÍN\u0002GUEZ 9"
};

myString = JSON.stringify(jsonObj);

const decodeString = (str) => {
    return str.replace(/\\u[\dA-F]{4}/gi, (unicode) => {
            return String.fromCharCode(parseInt(unicode.replace(/\\u/g, ""), 16));
        });
}


console.log("Decoding a unicode: ", decodeString("\\u00C4")); //double \\ to stop auto conversion by JS

console.log("Decoded jsonString: ", decodeString(myString));

I assume the \u0002 is simply nothing, right? Anyways, this function is supposed to convert any unicode to utf-8

user_142591650
Related