Convert unicode codepoint to utf-16

Scott_Langham_11898 0
Viewed 402

In C++ on Windows how do you convert an xml character reference of the form &#xhhhh; to a utf-16 little endian string?

I'm thinking if the hhhh part is 4 characters or less, then it's 2 bytes, which fit into one utf-16 character. But, this wiki page has a table of character references and some near the bottom are 5 digit hex numbers which won't fit into two bytes. How can they be converted to utf-16?

I'm wondering if the MultiByteToWideChar function is capable of doing the job.

My understanding of how a code point that's bigger than 2 bytes gets converted to utf-16 is lacking! (Or for that matter I'm not too sure how a code point that's bigger than 1 byte get's converted to utf-8, but that's another question).

Thanks.

unicode-escapes unicode windows c++
1 Answers

Unicode code-points (UTF-32) are 4 bytes wide and can be converted into a UTF-16character (and possible surrogate) using the following code (that I happen to have lying around).

It is not heavily tested so bug reports gratefully accepted:

/**
 * Converts U-32 code point to UTF-16 (and optional surrogate)
 * @param utf32 - UTF-32 code point
 * @param utf16 - returned UTF-16 character
 * @return - The number code units in the UTF-16 char (1 or 2).
 */
unsigned utf32_to_utf16(char32_t utf32, std::array<char16_t, 2>& utf16)
{
    if(utf32 < 0xD800 || (utf32 > 0xDFFF && utf32 < 0x10000))
    {
        utf16[0] = char16_t(utf32);
        utf16[1] = 0;
        return 1;
    }

    utf32 -= 0x010000;

    utf16[0] = char16_t(((0b1111'1111'1100'0000'0000 & utf32) >> 10) + 0xD800);
    utf16[1] = char16_t(((0b0000'0000'0011'1111'1111 & utf32) >> 00) + 0xDC00);

    return 2;
}
user_38077290
Related