How can I remove the BOM from a UTF-8 file?

user_2249798 0
Viewed 34701

I have a file in UTF-8 encoding with BOM and want to remove the BOM. Are there any linux command-line tools to remove the BOM from the file?

$ file test.xml
test.xml:  XML 1.0 document, UTF-8 Unicode (with BOM) text, with very long lines
utf-8 byte-order-mark linux command-line file
6 Answers

Well, just dealt with this today and my preferred way was dos2unix:

dos2unix will remove BOM and also take care of other idiosyncrasies from other SOs:

$ sudo apt install dos2unix
$ dos2unix test.xml

It's also possible to remove BOM only (-r, --remove-bom):

$ dos2unix -r test.xml

Note: tested with dos2unix 7.3.4

user_45340780

Joshua Pinter's answer works correctly on mac so I wrote a script that removes the BOM from all files in a given folder, see here.

It can be used like follows:

Remove BOM from all files in current directory: rmbom .

Print all files with a BOM in the current directory: rmbom . -a

Only remove BOM from all files in current directory with extension txt or cs: rmbom . -e txt -e cs

user_49247370

If you want to work on a bulk of files, by improving Reginaldo Santos's answers, there is a quick way:

find . -name "*.java" | grep java$ | xargs -n 1 dos2unix
user_67996990
Related