How to check the charset of string in Java?

user_219433 0
Viewed 116962

In my application I'm getting the user info from LDAP and sometimes the full username comes in a wrong charset. For example: 

ТеÑÑ61 ТеÑÑовиÑ61

It can also be in English or in Russian and displayed correctly. If the username changes it's updated in database. Even if I change the value in the db it wont solve the problem.

I can fix it before saving by doing this 

new String(incorrect.getBytes("ISO-8859-1"), "UTF-8");

However, if I will use it for the string including characters in Russian (for ex., "Тест61 Тестович61") I get something like this "????61 ????????61".

Can you please suggest something that can determine the charset of string?

character-encoding java string
6 Answers

I recommend Apache.tika CharsetDetector, very friendly and strong.

CharsetDetector detector = new CharsetDetector();
detector.setText(yourStr.getBytes());
detector.detect();  // <- return the result, you can check by .getName() method

Further, you can convert any encoded string to your desired one, take utf-8 as example:

detector.getString(yourStr.getBytes(), "utf-8");
user_48099380

I highly appreciate Lluís Turró Cutiller's answer (+1), but want to add a variant based on that.

private String convert(String value, Charset fromEncoding, Charset toEncoding) throws UnsupportedEncodingException {
    return new String(value.getBytes(fromEncoding), toEncoding);
}

private boolean probe(String value, Charset charset) throws UnsupportedEncodingException {
    Charset probe = StandardCharsets.UTF_8;
    return value.equals(convert(convert(value, charset, probe), probe, charset));
}

public String convert(String value, Charset charsetWanted, List<Charset> charsetsOther) throws UnsupportedEncodingException {
    if (probe(value, charsetWanted)) {
        return value;
    }
    for (Charset other: charsetsOther) {
        if (probe(value, other)) {
            return convert(value, other, charsetWanted);
        }
    }
    System.err.println("WARNING: Could not convert string: " + value);
    return value;
}
user_20812790
Related