Remove control characters from PHP string

user_130376 0
Viewed 87742

How can I remove control characters like STX from a PHP string? I played around with

preg_replace("/[^a-zA-Z0-9 .\-_;!:?äÄöÖüÜß<>='\"]/","",$pString)

but found that it removed way to much. Is there a way to remove only control chars?

php regex string
6 Answers

TLDR Answer

Use this Regex...

/[^\PCc^\PCn^\PCs]/u

Like this...

$text = preg_replace('/[^\PCc^\PCn^\PCs]/u', '', $text);

TLDR Explanation

  • ^\PCc : Do not match control characters.
  • ^\PCn : Do not match unassigned characters.
  • ^\PCs : Do not match UTF-8-invalid characters.

Working Demo

Simple demo to demonstrate: IDEOne Demo

$text = "\u{0019}hello";
print($text . "\n\n");
$text = preg_replace('/[^\PCc^\PCn^\PCs]/u', '', $text);
print($text);

Output:

(-Broken-Character)hello
hello

Alternatives

  • ^\PC : Match only visible characters. Do not match any invisible characters.
  • ^\PCc : Match only non-control characters. Do not match any control characters.
  • ^\PCc^\PCn : Match only non-control characters that have been assigned. Do not match any control or unassigned characters.
  • ^\PCc^\PCn^\PCs : Match only non-control characters that have been assigned and are UTF-8 valid. Do not match any control, unassigned, or UTF-8-invalid characters.
  • ^\PCc^\PCn^\PCs^\PCf : Match only non-control, non-formatting characters that have been assigned and are UTF-8 valid. Do not match any control, unassigned, formatting, or UTF-8-invalid characters.

Source and Explanation

Take a look at the Unicode Character Properties available that can be used to test within a regex. You should be able to use these regexes in Microsoft .NET, JavaScript, Python, Java, PHP, Ruby, Perl, Golang, and even Adobe. Knowing Unicode character classes is very transferable knowledge, so I recommend using it!

This regex will match anything visible, given in both its short-hand and long-hand form...

\PL\PM\PN\PP\PS\PZ
\PLetter\PMark\PNumber\PPunctuation\PSymbol\PSeparator

Normally, \p indicates that it's something we want to match and we use \P (capitalized) to indicate something that does not match. But PHP doesn't have this functionality, so we need to use ^ in the regex to do a manual negation.

A simpler regex then would be ^\PC, but this might be too restrictive in deleting invisible formatting. You may want to look closely and see what's best, but one of the alternatives should fit your needs.

All Matchable Unicode Character Sets

If you want to know any other character sets available, check out regular-expressions.info...

  • \PL or \PLetter: any kind of letter from any language.
    • \PLl or \PLowercase_Letter: a lowercase letter that has an uppercase variant.
    • \PLu or \PUppercase_Letter: an uppercase letter that has a lowercase variant.
    • \PLt or \PTitlecase_Letter: a letter that appears at the start of a word when only the first letter of the word is capitalized.
    • \PL& or \PCased_Letter: a letter that exists in lowercase and uppercase variants (combination of Ll, Lu and Lt).
    • \PLm or \PModifier_Letter: a special character that is used like a letter.
    • \PLo or \POther_Letter: a letter or ideograph that does not have lowercase and uppercase
  • \PM or \PMark: a character intended to be combined with another character (e.g. accents, umlauts, enclosing boxes, etc.).
    • \PMn or \PNon_Spacing_Mark: a character intended to be combined with another character without taking up extra space (e.g. accents, umlauts, etc.).
    • \PMc or \PSpacing_Combining_Mark: a character intended to be combined with another character that takes up extra space (vowel signs in many Eastern languages).
    • \PMe or \PEnclosing_Mark: a character that encloses the character it is combined with (circle, square, keycap, etc.).
  • \PZ or \PSeparator: any kind of whitespace or invisible separator.
    • \PZs or \PSpace_Separator: a whitespace character that is invisible, but does take up space.
    • \PZl or \PLine_Separator: line separator character U+2028.
    • \PZp or \PParagraph_Separator: paragraph separator character U+2029.
  • \PS or \PSymbol: math symbols, currency signs, dingbats, box-drawing characters, etc.
    • \PSm or \PMath_Symbol: any mathematical symbol.
    • \PSc or \PCurrency_Symbol: any currency sign.
    • \PSk or \PModifier_Symbol: a combining character (mark) as a full character on its own.
    • \PSo or \POther_Symbol: various symbols that are not math symbols, currency signs, or combining characters.
  • \PN or \PNumber: any kind of numeric character in any script.
    • \PNd or \PDecimal_Digit_Number: a digit zero through nine in any script except ideographic scripts.
    • \PNl or \PLetter_Number: a number that looks like a letter, such as a Roman numeral.
    • \PNo or \POther_Number: a superscript or subscript digit, or a number that is not a digit 0–9 (excluding numbers from ideographic scripts).
  • \PP or \PPunctuation: any kind of punctuation character.
    • \PPd or \PDash_Punctuation: any kind of hyphen or dash.
    • \PPs or \POpen_Punctuation: any kind of opening bracket.
    • \PPe or \PClose_Punctuation: any kind of closing bracket.
    • \PPi or \PInitial_Punctuation: any kind of opening quote.
    • \PPf or \PFinal_Punctuation: any kind of closing quote.
    • \PPc or \PConnector_Punctuation: a punctuation character such as an underscore that connects words.
    • \PPo or \POther_Punctuation: any kind of punctuation character that is not a dash, bracket, quote or connector.
  • \PC or \POther: invisible control characters and unused code points.
    • \PCc or \PControl: an ASCII or Latin-1 control character: 0x00–0x1F and 0x7F–0x9F.
    • \PCf or \PFormat: invisible formatting indicator.
    • \PCo or \PPrivate_Use: any code point reserved for private use.
    • \PCs or \PSurrogate: one half of a surrogate pair in UTF-16 encoding.
    • \PCn or \PUnassigned: any code point to which no character has been assigned.
user_24305490
Related