How do I remove accents from characters in a PHP string?

Viewed 190841

I'm attempting to remove accents from characters in PHP string as the first step to making the string usable in a URL.

I'm using the following code:

$input = "Fóø Bår";

setlocale(LC_ALL, "en_US.utf8");
$output = iconv("utf-8", "ascii//TRANSLIT", $input);

print($output);

The output I would expect would be something like this:

F'oo Bar

However, instead of the accented characters being transliterated they are replaced with question marks:

F?? B?r

Everything I can find online indicates that setting the locale will fix this problem, however I'm already doing this. I've already checked the following details:

  1. The locale I am setting is supported by the server (included in the list produced by locale -a)
  2. The source and target encodings (UTF-8 and ASCII) are supported by the server's version of iconv (included in the list produced by iconv -l)
  3. The input string is UTF-8 encoded (verified using PHP's mb_check_encoding function, as suggested in the answer by mercator)
  4. The call to setlocale is successful (it returns 'en_US.utf8' rather than FALSE)

The cause of the problem:

The server is using the wrong implementation of iconv. It has the glibc version instead of the required libiconv version.

Note that the iconv function on some systems may not work as you expect. In such case, it'd be a good idea to install the GNU libiconv library. It will most likely end up with more consistent results.
PHP manual's introduction to iconv

Details about the iconv implementation that is used by PHP are included in the output of the phpinfo function.

(I'm not able to re-compile PHP with the correct iconv library on the server I'm working with for this project so the answer I've accepted below is the one that was most useful for removing accents without iconv support.)

28 Answers

I think the problem here is that your encodings consider ä and å different symbols to 'a'. In fact, the PHP documentation for strtr offers a sample for removing accents the ugly way :(

http://ie2.php.net/strtr

You could use urlencode. Does not quite do what you want (remove accents), but will give you a url usable string

$output = urlencode ($input);

In Perl I could use a translate regex, but I cannot think of the PHP equivalent

$input =~ tr/áâàå/aaaa/;

etc...

you could do this using preg_replace

$patterns[0] = '/[á|â|à|å|ä]/';
$patterns[1] = '/[ð|é|ê|è|ë]/';
$patterns[2] = '/[í|î|ì|ï]/';
$patterns[3] = '/[ó|ô|ò|ø|õ|ö]/';
$patterns[4] = '/[ú|û|ù|ü]/';
$patterns[5] = '/æ/';
$patterns[6] = '/ç/';
$patterns[7] = '/ß/';
$replacements[0] = 'a';
$replacements[1] = 'e';
$replacements[2] = 'i';
$replacements[3] = 'o';
$replacements[4] = 'u';
$replacements[5] = 'ae';
$replacements[6] = 'c';
$replacements[7] = 'ss';

$output = preg_replace($patterns, $replacements, $input);

(Please note this was typed from a foggy beer ridden Friday after noon memory, so may not be 100% correct)

or you could make a hash table and do a replacement based off of that.

What's wrong with this one? Works with UTF8

function strip_accents($s){
  return str_replace(
    explode(' ', preg_replace('/ +/', ' ', 'č ć ž š đ  Č Ć Ž Š Đ  à á â ã ä ç è é ê ë ì í î ï ñ ò ó ô õ ö ù ú û ü ý ÿ À Á Â Ã Ä Ç È É Ê Ë Ì Í Î Ï Ñ Ò Ó Ô Õ Ö Ù Ú Û Ü Ý')),
    explode(' ', preg_replace('/ +/', ' ', 'c c z s dj C C Z S DJ a a a a a c e e e e i i i i n o o o o o u u u u y y A A A A A C E E E E I I I I N O O O O O U U U U Y')),
    $s);
}

It can be faster by not using preg_replace, but speed was not my goal here.

I agree with georgebrock's comment.

If you find a way to get //TRANSLIT to work, you can build friendly URLs:

  1. use iconv with //TRANSLIT ñ => n~
    • remove non-alphanumeric non-whitespace chars inside words: $url = preg_replace( '/(\w)[^\w\s](\w)/', '$1$2', $url );
    • replace remaining separations: $url = preg_replace( '/[^a-z0-9]+/', '-', $url );
    • remove double/leading/traling: $url = preg_replace( '-', e.g. '/(?:(^|\-)\-+|\-$)/', '', $url );

If you can't get it to work, replace setp 1 with strtr/character-based replacement, like Xetius' solution.

I can't reproduce your problem. I get the expected result.

How exactly are you using mb_detect_encoding() to verify your string is in fact UTF-8?

If I simply call mb_detect_encoding($input) on both a UTF-8 and ISO-8859-1 encoded version of your string, both of them return "UTF-8", so that function isn't particularly reliable.

iconv() gives me a PHP "notice" when it gets the wrongly encoded string and only echoes "F", but that might just be because of different PHP/iconv settings/versions (?).

I suggest to you try calling mb_check_encoding($input, "utf-8") first to verify that your string really is UTF-8. I think it probably isn't.

Merged Cazuma Nii Cavalcanti's implementation with Junior Mayhé's char list, hoping to save some time for some of you.

function stripAccents($str) {
    return strtr(utf8_decode($str), utf8_decode('ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖØÙÚÛÜÝßàáâãäåæçèéêëìíîïñòóôõöøùúûüýÿĀāĂ㥹ĆćĈĉĊċČčĎďĐđĒēĔĕĖėĘęĚěĜĝĞğĠġĢģĤĥĦħĨĩĪīĬĭĮįİıIJijĴĵĶķĹĺĻļĽľĿŀŁłŃńŅņŇňʼnŌōŎŏŐőŒœŔŕŖŗŘřŚśŜŝŞşŠšŢţŤťŦŧŨũŪūŬŭŮůŰűŲųŴŵŶŷŸŹźŻżŽžſƒƠơƯưǍǎǏǐǑǒǓǔǕǖǗǘǙǚǛǜǺǻǼǽǾǿ'), 'AAAAAAAECEEEEIIIIDNOOOOOOUUUUYsaaaaaaaeceeeeiiiinoooooouuuuyyAaAaAaCcCcCcCcDdDdEeEeEeEeEeGgGgGgGgHhHhIiIiIiIiIiIJijJjKkLlLlLlLlllNnNnNnnOoOoOoOEoeRrRrRrSsSsSsSsTtTtTtUuUuUuUuUuUuWwYyYZzZzZzsfOoUuAaIiOoUuUuUuUuUuAaAEaeOo');
}

In laravel you can simply use str_slug($accentedPhrase) and if you care about dash (-) that this method substitute with space you can use str_replace('-', ' ', str_slug($accentedPhrase))

Something like this?

$arrSearch  = explode(","," ,ç,æ, œ, á,é,í,ó,ú,à,è,ì,ò,ù,ä,ë,ï,ö,ü,ÿ,â,ê,î,ô,û,å,e,i,ø,u");

$arrReplace = explode(",","_,c,ae,oe,a,e,i,o,u,a,e,i,o,u,a,e,i,o,u,y,a,e,i,o,u,a,e,i,o,u");

$output = str_replace($arrSearch, $arrReplace, $input);

If the main task is just to use the string in a URL, why not to use slugyfier?

composer require cocur/slugify

then

use Cocur\Slugify\Slugify;

$slugify = new Slugify();
echo $slugify->slugify('Fóø Bår');

It also has many bridges for popular frameworks. E.g. you can use Doctrine Extensions Sluggable behaviour to generate automatically unique slug for each entity in DB and use it in URL.

If you want just to wipe out all accents you can play around with rulesets to satisfy the requirements.

function Unaccent($string) { return preg_replace('~&([a-z]{1,2}) (acute|cedil|circ|grave|lig|orn|ring|slash|th|tilde|uml);~i', '$1', htmlentities($string, ENT_QUOTES, 'UTF-8'));

}

This answer I've got following tips here, so it is not really mine. It works for me using LATIN1 or UTF-8. If you use other charsets, you probably should add them to mb_detect_encoding function. Correct environment set is probably needed also.

function NoAccents($s){
        return iconv(mb_detect_encoding($s,'UTF-8, ASCII, ISO-8859-1'),'ASCII//TRANSLIT//INGORE',$s);
}

Based on @Mimouni answer I made this function to transliterate Accented strings to Non Accented strings.

/**
 * @param $str Convert string to lowercase and replace special chars to equivalents ou remove its
 * @return string
 */
function _slugify(string $string): string
{
    $str = $string; // for comparisons
    $str = _toUtf8($str); // Force to work with string in UTF-8
    $str = iconv('UTF-8', 'ASCII//TRANSLIT', $str);

    if ($str != htmlentities($string, ENT_QUOTES, 'UTF-8')) { // iconv fails
        $str = _toUtf8($string);
        $str = htmlentities($str, ENT_QUOTES, 'UTF-8');
        $str = preg_replace('#&([a-z]{1,2})(acute|cedil|circ|grave|lig|orn|ring|slash|th|tilde|uml);#i', '$1', $str);
        // Need to strip non ASCII chars or any other than a-z, A-Z, 0-9...
        $str = html_entity_decode($str, ENT_QUOTES, 'UTF-8');
        $str = preg_replace(array('#[^0-9a-z]#i', '#[ -]+#'), ' ', $str);
        $str = trim($str, ' -');
    }

    // lowercase
    $string = strtolower($str);

    return $string;
}

To convert strings to UTF-8, here I use the Multi Byte String extension. Note that I break string in pieces to avoid trouble with mixed content (I have such situation) and convert word by word.

/**
 * @param $str string String in any encoding
 * @return string
 */
function _toUtf8(string $str_in): ?string
{
    if (!function_exists('mb_detect_encoding')) {
        throw new \Exception('The Multi Byte String extension is absent!');
    }
    $str_out = [];
    $words = explode(" ", $str_in);
    foreach ($words as $word) {
        $current_encoding = mb_detect_encoding($word, 'UTF-8, ASCII, ISO-8859-1');
        $str_out[] = mb_convert_encoding($word, 'UTF-8', $current_encoding);
    }
    return implode(" ", $str_out);
}

Footer Notes: Was the only solution that pass in PHPUnit UnitTests in Windows command Line (locale issues) The @gabo solution should work but unfortunately not for me

Combining some of the answers:

/**
 * Given an utf8 string, returns an ascii string.
 * Only supports ISO-8859-1 chars, not chars like č, œ, Œ, ř, Š, š, ů, Ÿ or ž
 * @param string $utf8
 * @return string ascii
 */
function stripAccents(string $utf8): string
{
    $src = 'àáâãäåçèéêëìíîïñòóôõöùúûüýÿÀÁÂÃÄÇÈÉÊËÌÍÎÏÑÒÓÔÕÖÙÚÛÜÝ';
    $dst = 'aaaaaaceeeeiiiinooooouuuuyyAAAAACEEEEIIIINOOOOOUUUUY';
    $iso8859_1 = strtr(
        utf8_decode($utf8),
        utf8_decode($src),
        $dst
    );
    // iconv below not used for UTF-8 directly because it adds chars like ^'"`
    // which are nice to not lose info, but not very readable and not compatible with filenames
    // iconv() additionally converts chars like
    // ¡ ¢ £  ¥   ¦ §  ¨  ©  ª «  ¬   SHY ®    °  ±   ²  ³  ´ µ ¶ · ¸ ¹  º »  ¼   ½   ¾   ¿ Å Æ  Ð Ø Þ  ß  æ  ÷   ø þ
    // ! c lb yen | SS " (c) a << not SHY (R)  ^0 +/- ^2 ^3 ' u P . , ^1 o >> 1/4 1/2 3/4 ? A AE D O Th ss ae d : o th
    return iconv('ISO-8859-1', 'ASCII//TRANSLIT//IGNORE', $iso8859_1);
}

One of the tricks I stumbled upon on the web was using htmlentities then stripping the encoded character :

$stripped = preg_replace('`&[^;]+;`','',htmlentities($string));

Not perfect but it does work well in some case.

But, you're writing about creating an URL string, so urlencode and its counterpart urldecode may be better. Or, if you are creating a query string, use this last function : http_build_query.

Related