How do I remove  from the beginning of a file?

user_366133 0
Viewed 231273

I have a CSS file that looks fine when I open it using gedit, but when it's read by PHP (to merge all the CSS files into one), this CSS has the following characters prepended to it: 

PHP removes all whitespace, so a random  in the middle of the code messes up the entire thing. As I mentioned, I can't actually see these characters when I open the file in gedit, so I can't remove them very easily.

I googled the problem, and there is clearly something wrong with the file encoding, which makes sense being as I've been shifting the files around to different Linux/Windows servers via ftp and rsync, with a range of text editors. I don't really know much about character encoding though, so help would be appreciated.

If it helps, the file is being saved in UTF-8 format, and gedit won't let me save it in ISO-8859-15 format (the document contains one or more characters that cannot be encoded using the specified character encoding). I tried saving it with Windows and Linux line endings, but neither helped.

mojibake utf-8 character-encoding byte-order-mark php
23 Answers

You can use

vim -e -c 'argdo set fileencoding=utf-8|set encoding=utf-8| set nobomb| wq'

Replacing with awk seems to work, but it is not in place.

user_1483370

grep -rl $'\xEF\xBB\xBF' * | xargs vim -e -c 'argdo set fileencoding=utf-8|set encoding=utf-8| set nobomb| wq'

user_4589470

This works for me!

def removeBOMs(fileName):
     BOMs = ['',#Bytes as CP1252 characters
    'þÿ',
    'ÿþ',
    '^@^@þÿ',
    'ÿþ^@^@',
    '+/v',
    '÷dL',
    'Ýsfs',
    'Ýsfs',
    '^Nþÿ',
    'ûî(',
    '„1•3']
     inputFile = open(fileName, 'r')
     contents = inputFile.read()
     for BOM in BOMs:
         if not BOM in contents:#no BOM in the file...
             pass
         else:
             newContents = contents.replace(BOM,'', 1)
             newFile = open(fileName, 'w')
             newFile.write(newContents)
             return None
user_88966530
Related