I am interested in the word iterator of the ICU63 library in a JavaScript project (in a browser). So after reading the docs, I believe that ICU uses UTF-16 by default which is the same than JS and it would avoid me to encode JS strings into something else.
First step was to build a wrapper with the only function that I need (I don't know yet if it is working):
#include "emscripten.h"
#include <string.h>
#include <unicode/brkiter.h>
#include <unicode/unistr.h>
#include <unicode/errorcode.h>
using namespace icu_63;
EMSCRIPTEN_KEEPALIVE
int splitWords(const char *locale, const uint16_t *text, uint16_t *splitted) {
//Note that Javascript is working in UTF-16
//icu::
UnicodeString result = UnicodeString();
UnicodeString visibleSpace = UnicodeString(" ");
int32_t previousIdx = 0;
int32_t idx = -1;
//Create a Unicode String from input
UnicodeString uTextArg = UnicodeString(text);
if (uTextArg.isBogus()) {
return -1; // input string is bogus
}
//Create and init the iterator
UErrorCode err = U_ZERO_ERROR;
BreakIterator *iter = BreakIterator::createWordInstance(locale, err);
if (U_FAILURE(err)) {
return -2; // cannot build iterator
}
iter->setText(uTextArg);
//Iterate and store results
while ((idx = iter->next()) != -1) {
UnicodeString word = UnicodeString(uTextArg, idx, idx - previousIdx);
result += word;
result += visibleSpace;
previousIdx = idx;
}
result.trim();
//The buffer contains UTF-16 characters, so it takes 2 bytes per point
memcpy(splitted, result.getBuffer(), result.getCapacity() * 2);
return 0;
}
It compiles and looks good except that symbols are missing when trying to link because I have no clue about how to proceed.
LibICU looks to need a lot of builtin data. For my case, the frequency tables are mandatory for using the word iterator.
Should I try to copy my wrapper into the source folder and try to figure out how to use emconfigure. Or is it possible to link the libicu when I try to compile my wrapper? Second option looks like a waste of data as I am not interested by the larger portion of the lib.