How does Stack Overflow generate its SEO-friendly URLs?

Here's how we do it. Note that there are probably more edge conditions than you realize at first glance.

This is the second version, unrolled for 5x more performance (and yes, I benchmarked it). I figured I'd optimize it because this function can be called hundreds of times per page.

/// <summary>
/// Produces optional, URL-friendly version of a title, "like-this-one". 
/// hand-tuned for speed, reflects performance refactoring contributed
/// by John Gietzen (user otac0n) 
/// </summary>
public static string URLFriendly(string title)
{
    if (title == null) return "";

    const int maxlen = 80;
    int len = title.Length;
    bool prevdash = false;
    var sb = new StringBuilder(len);
    char c;

    for (int i = 0; i < len; i++)
    {
        c = title[i];
        if ((c >= 'a' && c <= 'z') || (c >= '0' && c <= '9'))
        {
            sb.Append(c);
            prevdash = false;
        }
        else if (c >= 'A' && c <= 'Z')
        {
            // tricky way to convert to lowercase
            sb.Append((char)(c | 32));
            prevdash = false;
        }
        else if (c == ' ' || c == ',' || c == '.' || c == '/' || 
            c == '\\' || c == '-' || c == '_' || c == '=')
        {
            if (!prevdash && sb.Length > 0)
            {
                sb.Append('-');
                prevdash = true;
            }
        }
        else if ((int)c >= 128)
        {
            int prevlen = sb.Length;
            sb.Append(RemapInternationalCharToAscii(c));
            if (prevlen != sb.Length) prevdash = false;
        }
        if (i == maxlen) break;
    }

    if (prevdash)
        return sb.ToString().Substring(0, sb.Length - 1);
    else
        return sb.ToString();
}

To see the previous version of the code this replaced (but is functionally equivalent to, and 5x faster), view revision history of this post (click the date link).

Also, the RemapInternationalCharToAscii method source code can be found here.

You will want to setup a custom route to point the URL to the controller that will handle it. Since you are using Ruby on Rails, here is an introduction in using their routing engine.

In Ruby, you will need a regular expression like you already know and here is the regular expression to use:

def permalink_for(str)
    str.gsub(/[^\w\/]|[!\(\)\.]+/, ' ').strip.downcase.gsub(/\ +/, '-')
end

You can also use this JavaScript function for in-form generation of the slug's (this one is based on/copied from Django):

function makeSlug(urlString, filter) {
    // Changes, e.g., "Petty theft" to "petty_theft".
    // Remove all these words from the string before URLifying

    if(filter) {
        removelist = ["a", "an", "as", "at", "before", "but", "by", "for", "from",
        "is", "in", "into", "like", "of", "off", "on", "onto", "per",
        "since", "than", "the", "this", "that", "to", "up", "via", "het", "de", "een", "en",
        "with"];
    }
    else {
        removelist = [];
    }
    s = urlString;
    r = new RegExp('\\b(' + removelist.join('|') + ')\\b', 'gi');
    s = s.replace(r, '');
    s = s.replace(/[^-\w\s]/g, ''); // Remove unneeded characters
    s = s.replace(/^\s+|\s+$/g, ''); // Trim leading/trailing spaces
    s = s.replace(/[-\s]+/g, '-'); // Convert spaces to hyphens
    s = s.toLowerCase(); // Convert to lowercase
    return s; // Trim to first num_chars characters
}

For good measure, here's the PHP function in WordPress that does it... I'd think that WordPress is one of the more popular platforms that uses fancy links.

    function sanitize_title_with_dashes($title) {
            $title = strip_tags($title);
            // Preserve escaped octets.
            $title = preg_replace('|%([a-fA-F0-9][a-fA-F0-9])|', '---$1---', $title);
            // Remove percent signs that are not part of an octet.
            $title = str_replace('%', '', $title);
            // Restore octets.
            $title = preg_replace('|---([a-fA-F0-9][a-fA-F0-9])---|', '%$1', $title);
            $title = remove_accents($title);
            if (seems_utf8($title)) {
                    if (function_exists('mb_strtolower')) {
                            $title = mb_strtolower($title, 'UTF-8');
                    }
                    $title = utf8_uri_encode($title, 200);
            }
            $title = strtolower($title);
            $title = preg_replace('/&.+?;/', '', $title); // kill entities
            $title = preg_replace('/[^%a-z0-9 _-]/', '', $title);
            $title = preg_replace('/\s+/', '-', $title);
            $title = preg_replace('|-+|', '-', $title);
            $title = trim($title, '-');
            return $title;
    }

This function as well as some of the supporting functions can be found in wp-includes/formatting.php.

I am not familiar with Ruby on Rails, but the following is (untested) PHP code. You can probably translate this very quickly to Ruby on Rails if you find it useful.

$sURL = "This is a title to convert to URL-format. It has 1 number in it!";
// To lower-case
$sURL = strtolower($sURL);

// Replace all non-word characters with spaces
$sURL = preg_replace("/\W+/", " ", $sURL);

// Remove trailing spaces (so we won't end with a separator)
$sURL = trim($sURL);

// Replace spaces with separators (hyphens)
$sURL = str_replace(" ", "-", $sURL);

echo $sURL;
// outputs: this-is-a-title-to-convert-to-url-format-it-has-1-number-in-it

I hope this helps.

If you are using Rails edge, you can rely on Inflector.parametrize - here's the example from the documentation:

  class Person
    def to_param
      "#{id}-#{name.parameterize}"
    end
  end

  @person = Person.find(1)
  # => #<Person id: 1, name: "Donald E. Knuth">

  <%= link_to(@person.name, person_path(@person)) %>
  # => <a href="/person/1-donald-e-knuth">Donald E. Knuth</a>

Also if you need to handle more exotic characters such as accents (éphémère) in previous version of Rails, you can use a mixture of PermalinkFu and DiacriticsFu:

DiacriticsFu::escape("éphémère")
=> "ephemere"

DiacriticsFu::escape("räksmörgås")
=> "raksmorgas"

I don't much about Ruby or Rails, but in Perl, this is what I would do:

my $title = "How do you change a title to be part of the url like Stackoverflow?";

my $url = lc $title;   # Change to lower case and copy to URL.
$url =~ s/^\s+//g;     # Remove leading spaces.
$url =~ s/\s+$//g;     # Remove trailing spaces.
$url =~ s/\s+/\-/g;    # Change one or more spaces to single hyphen.
$url =~ s/[^\w\-]//g;  # Remove any non-word characters.

print "$title\n$url\n";

I just did a quick test and it seems to work. Hopefully this is relatively easy to translate to Ruby.

T-SQL implementation, adapted from dbo.UrlEncode:

CREATE FUNCTION dbo.Slug(@string varchar(1024))
RETURNS varchar(3072)
AS
BEGIN
    DECLARE @count int, @c char(1), @i int, @slug varchar(3072)

    SET @string = replace(lower(ltrim(rtrim(@string))),' ','-')

    SET @count = Len(@string)
    SET @i = 1
    SET @slug = ''

    WHILE (@i <= @count)
    BEGIN
        SET @c = substring(@string, @i, 1)

        IF @c LIKE '[a-z0-9--]'
            SET @slug = @slug + @c

        SET @i = @i +1
    END

    RETURN @slug
END

Assuming that your model class has a title attribute, you can simply override the to_param method within the model, like this:

def to_param
  title.downcase.gsub(/ /, '-')
end

This Railscast episode has all the details. You can also ensure that the title only contains valid characters using this:

validates_format_of :title, :with => /^[a-z0-9-]+$/,
                    :message => 'can only contain letters, numbers and hyphens'

Brian's code, in Ruby:

title.downcase.strip.gsub(/\ /, '-').gsub(/[^\w\-]/, '')

downcase turns the string to lowercase, strip removes leading and trailing whitespace, the first gsub call globally substitutes spaces with dashes, and the second removes everything that isn't a letter or a dash.

There is a small Ruby on Rails plugin called PermalinkFu, that does this. The escape method does the transformation into a string that is suitable for a URL. Have a look at the code; that method is quite simple.

To remove non-ASCII characters it uses the iconv lib to translate to 'ascii//ignore//translit' from 'utf-8'. Spaces are then turned into dashes, everything is downcased, etc.

You can use the following helper method. It can convert the Unicode characters.

public static string ConvertTextToSlug(string s)
{
    StringBuilder sb = new StringBuilder();

    bool wasHyphen = true;

    foreach (char c in s)
    {
        if (char.IsLetterOrDigit(c))
        {
            sb.Append(char.ToLower(c));
            wasHyphen = false;
        }
        else
            if (char.IsWhiteSpace(c) && !wasHyphen)
            {
                sb.Append('-');
                wasHyphen = true;
            }
    }

    // Avoid trailing hyphens
    if (wasHyphen && sb.Length > 0)
        sb.Length--;

    return sb.ToString().Replace("--","-");
}

No, no, no. You are all so very wrong. Except for the diacritics-fu stuff, you're getting there, but what about Asian characters (shame on Ruby developers for not considering their nihonjin brethren).

Firefox and Safari both display non-ASCII characters in the URL, and frankly they look great. It is nice to support links like 'http://somewhere.com/news/read/お前たちはアホじゃないかい'.

So here's some PHP code that'll do it, but I just wrote it and haven't stress tested it.

<?php
    function slug($str)
    {
        $args = func_get_args();
        array_filter($args);  //remove blanks
        $slug = mb_strtolower(implode('-', $args));

        $real_slug = '';
        $hyphen = '';
        foreach(SU::mb_str_split($slug) as $c)
        {
            if (strlen($c) > 1 && mb_strlen($c)===1)
            {
                $real_slug .= $hyphen . $c;
                $hyphen = '';
            }
            else
            {
                switch($c)
                {
                    case '&':
                        $hyphen = $real_slug ? '-and-' : '';
                        break;
                    case 'a':
                    case 'b':
                    case 'c':
                    case 'd':
                    case 'e':
                    case 'f':
                    case 'g':
                    case 'h':
                    case 'i':
                    case 'j':
                    case 'k':
                    case 'l':
                    case 'm':
                    case 'n':
                    case 'o':
                    case 'p':
                    case 'q':
                    case 'r':
                    case 's':
                    case 't':
                    case 'u':
                    case 'v':
                    case 'w':
                    case 'x':
                    case 'y':
                    case 'z':

                    case 'A':
                    case 'B':
                    case 'C':
                    case 'D':
                    case 'E':
                    case 'F':
                    case 'G':
                    case 'H':
                    case 'I':
                    case 'J':
                    case 'K':
                    case 'L':
                    case 'M':
                    case 'N':
                    case 'O':
                    case 'P':
                    case 'Q':
                    case 'R':
                    case 'S':
                    case 'T':
                    case 'U':
                    case 'V':
                    case 'W':
                    case 'X':
                    case 'Y':
                    case 'Z':

                    case '0':
                    case '1':
                    case '2':
                    case '3':
                    case '4':
                    case '5':
                    case '6':
                    case '7':
                    case '8':
                    case '9':
                        $real_slug .= $hyphen . $c;
                        $hyphen = '';
                        break;

                    default:
                       $hyphen = $hyphen ? $hyphen : ($real_slug ? '-' : '');
                }
            }
        }
        return $real_slug;
    }

Example:

$str = "~!@#$%^&*()_+-=[]\{}|;':\",./<>?\n\r\t\x07\x00\x04 コリン ~!@#$%^&*()_+-=[]\{}|;':\",./<>?\n\r\t\x07\x00\x04 トーマス ~!@#$%^&*()_+-=[]\{}|;':\",./<>?\n\r\t\x07\x00\x04 アーノルド ~!@#$%^&*()_+-=[]\{}|;':\",./<>?\n\r\t\x07\x00\x04";
echo slug($str);

Outputs: コリン-and-トーマス-and-アーノルド

The '-and-' is because &'s get changed to '-and-'.

Rewrite of Jeff's code to be more concise

    public static string RemapInternationalCharToAscii(char c)
    {
        var s = c.ToString().ToLowerInvariant();

        var mappings = new Dictionary<string, string>
        {
            { "a", "àåáâäãåą" },
            { "c", "çćčĉ" },
            { "d", "đ" },
            { "e", "èéêëę" },
            { "g", "ğĝ" },
            { "h", "ĥ" },
            { "i", "ìíîïı" },
            { "j", "ĵ" },
            { "l", "ł" },
            { "n", "ñń" },
            { "o", "òóôõöøőð" },
            { "r", "ř" },
            { "s", "śşšŝ" },
            { "ss", "ß" },
            { "th", "Þ" },
            { "u", "ùúûüŭů" },
            { "y", "ýÿ" },
            { "z", "żźž" }
        };

        foreach(var mapping in mappings)
        {
            if (mapping.Value.Contains(s))
                return mapping.Key;
        }

        return string.Empty;
    }