How to test a string for containing emoji characters?

screwnut 0
Viewed 45

I'm using ICU4C and trying to find the clusters in a UTF-8 string that are emojis. This is the closest I've gotten so far but it incorrectly qualifies the simple character '#' as an emoji (because '#️⃣' begins with '#' and is "potentially" an emoji so '#' does carry the property UCHAR_EMOJI).

I think the best would be trying to get the property RGI_Emoji as indicated here but that's a "string" property and not a "codepoint" property and I don't know how to do that. If I could I would analyze every single character as a "string" and test for that string property. The documentation states that, currently, using a regular expression is not possible to get "string" properties.

const std::string s8 = "#asd‍dds‍️️‍♂️ds#️⃣ds‍‍‍ds‍❤️‍‍ds";
const icu::UnicodeString us = icu::UnicodeString::fromUTF8(s8);
UErrorCode status = U_ZERO_ERROR;
icu::BreakIterator* bi = icu::BreakIterator::createCharacterInstance(icu::Locale::getUS(), status);
bi->setText(us);
bool is_emoji = false;
for(int32_t e = bi->first(), b = e; e != icu::BreakIterator::DONE; b = e, e = bi->next())
{
    // Analyze character for emoji-ness.
    for(int32_t i = b; i != e; ++i)
    {
        std::cout << us.char32At(i) << ' ';
        is_emoji = u_hasBinaryProperty(us.char32At(i), UProperty::UCHAR_EMOJI) || u_hasBinaryProperty(us.char32At(i), UProperty::UCHAR_EMOJI_COMPONENT);
    }
    if(is_emoji)
    {
        std::cout << "<- is emoji\n";
        ++emojis;
        is_emoji = false;
    }
    else
    {
        std::cout << "<- is not emoji\n";
    }
    ++characters;

}
delete bi;
icu c++
1 Answers

Looks like u_stringHasBinaryProperty will give you access to UCHAR_RGI_EMOJI. Note that this method is not available in ICU versions < 70.

I think that you need to distinguish between basic emojis that consist of a single code point (e.g. U+1F600 ), and emoji sequences (e.g. U+0023 U+FE0F U+20E3 #️⃣). The basic emoji will have both UCHAR_EMOJI and UCHAR_BASIC_EMOJI properties set and UCHAR_EMOJI_COMPONENT unset; the first code point in a sequence will have UCHAR_EMOJI and UCHAR_EMOJI_COMPONENT set but not UCHAR_BASIC. Subsequent code points will have UCHAR_EMOJI_COMPONENT set, but not UCHAR_EMOJI (nor UCHAR_BASIC_EMOJI).

If you encounter '#' (basically any code point with UCHAR_EMOJI and UCHAR_EMOJI_COMPONENT set) you need to check the next code point. The '#' is part of an emoji only if the next code point doesn't have UCHAR_EMOJI but does have UCHAR_EMOJI_COMPONENT.

decocijo0
Related