I have got a file with special characters. And to load these files in hadoop hive the file should be in UTF-8.
When I type file -bi $filename command to get the encoding type, it gives me text/plain; charset=us-ascii
I then used icon to convert to utf-8 by:
iconv -f iso-8859-1 -t utf8 $filename > ${filename}_conv
Now this utf-8 file is successfully loading into hive.
But when I run the file -bi $${filename}_conv command it gives:
text/plain; charset=us-ascii
I was expecting the charset to be utf8 but it is returning ascii.
Is there any other way to find the encoding. Because I may re-use the file and do not want to encode again.
There is no isutf8 or chardet and I tried file --mime. All these doen't seem to work.
Is there any other command that can be used to determine the encoding type, would be really helpfull.
Thank you.