Bizarre encoding of umlaute (äöü)

user_6067562 0
Viewed 118

I'm currently making a school project, where I need to download data from a website (by crawling) and process/display it in an app for Android. The downloaded data often contains letters like ä, which can be displayed perfectly in a TextView without any doubt.

Sometimes, the data that I receive has gaps, which my parser fills after downloading it. Instead of displaying textThatContainsÄ, it displays  . Easy fix:

String data = downloadFromServer();

// Fill the gap
data = data.replace(" ", textThatContainsÄ);

The output as expected:

Output: [..]ä[..]

The logic I wrote above was packaged into a .jar and imported into my Android app. The data I downloaded is not affected in any way in the Android app because of encapsulation.

Now comes the twist. If the downloaded data from the server does NOT originally contain textThatContainsÄ (so that my parser has to fill the gap) Android interprets it as textThatContainsä, which is clearly a UTF-8 encoding problem, which is caused by treating the UTF-8 bytes as Windows-1252 (or ISO 8859-1) bytes (Android interprets the text as 21 instead of 18 bytes (Text has size of 9)). This problem does not occur if the data originally contains the ä.

I don't know why the ä generated by java code creates encoding problems in Android.

Edit: Was caused by Gradle.

gradle android java
1 Answers

I shot myself in the leg by using Gradle. After some research, I found out that Gradle normally does NOT compile .java files using UTF-8 by default. So when I compiled the .jar it accidentally translated ä to ä as I could see in the decompiled .class files.

By adding the code below, the whole problem was solved.

tasks.withType(JavaCompile) {
    options.encoding = 'UTF-8'
}
user_60675620
Related