How to skip invalid characters in XML file using PHP

user_315396 0
Viewed 51516

I'm trying to parse an XML file using PHP, but I get an error message:

parser error : Char 0x0 out of allowed range in

I think it's because of the content of the XML, I think there is a speical symbol "☆", any ideas what I can do to fix it?

I also get:

parser error : Premature end of data in tag item line

What might be causing that error?

I'm using simplexml_load_file.

Update:

I try to find the error line and paste its content as single xml file and it can work!! so I still cannot figure out what makes xml file parse fails. PS it's a huge xml file over 100M, will it makes parse error?

utf-8 xml php
9 Answers

Certain Unicode characters must not appear in XML 1.0:

  • C0 control codes (U+0000 - U+001F) expect tab, CR and LF.
  • UTF-16 surrogates (U+D800 - U+DFFF). These are invalid in UTF-8 as well and indicate more serious problems when encountered.
  • U+FFFE and U+FFFF.

But in practice, you often have to handle XML which was carelessly produced from other sources containing such characters. If you want to handle this special case of invalid XML in an UTF-8 encoded string, I'd suggest:

$str = preg_replace(
    '/[\x00-\x08\x0B\x0C\x0E-\x1F]|\xED[\xA0-\xBF].|\xEF\xBF[\xBE\xBF]/',
    "\xEF\xBF\xBD",
    $str
);

This doesn't use the u Unicode regex modifier but works directly on UTF-8 encoded bytes for extra performance. The parts of the pattern are:

  • Invalid control chars: [\x00-\x08\x0B\x0C\x0E-\x1F]
  • UTF-16 surrogates: \xED[\xA0-\xBF].
  • Non-characters U+FFFE and U+FFFF: \xEF\xBF[\xBE\xBF]

Invalid characters are replaced with the replacement character U+FFFD (�) instead of simply stripping them. This makes it easier to diagnose invalid chars and can even prevent security issues.

user_19560100

I used this to clean the string:

public static function Clean($inputName)
    {
        $strName=trim($inputName);
        
        if($strName!="")
        {
            $strName = iconv("UTF-8", "UTF-8//IGNORE", $strName); // drop all non utf-8 characters
            
            $strName=str_replace(array('\\','/',':','*','?','"','<','>','|'),'@',$strName); 
            $string = preg_replace('/[\x00-\x1F\x7F\xA0]/u', '', $string);
            
            // [\x00-\x1F]  control characters http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247%28v=vs.85%29.aspx   
            
            // Invalid control chars: [\x00-\x08\x0B\x0C\x0E-\x1F]
            // UTF-16 surrogates: \xED[\xA0-\xBF].
            // Non-characters U+FFFE and U+FFFF: \xEF\xBF[\xBE\xBF]
            // Invalid characters are replaced with the replacement character U+FFFD 

            $strName = preg_replace(
            '/[\x00-\x08\x0B\x0C\x0E-\x1F]|\xED[\xA0-\xBF].|\xEF\xBF[\xBE\xBF]/',
            "\xEF\xBF\xBD",
            $strName);
            
            // Reduce all multiple whitespace to a single space
            // $strName = preg_replace('/\s+/', ' ', $strName); 
            
            if(trim($strName)=="")
            {
                $strName="@" . "empty-name";
            }
        }
        else
        {
            $strName=" ";
        }       
        
        return $strName;
    }
user_31249950
Related