shorter php cipher than md5?

Viewed 27116

For a variety of stupid reasons, the maximum length of a given form variable that we are posting to an external server is 12 characters.

I wanted to obscure that value with md5, but obviously with 12 characters that isn't going to work. Is there a cipher with an already-made PHP function which will result in something 12 characters or less?

The security and integrity of the cipher isn't super important here. My last resort is to just write a function which moves each letter up or down an ascii value by x. So the goal isn't to obscure it from a cryptography expert, but just to not post it in plain text so a non-technical worker looking at it won't know what it is.

Thanks for any advice.

10 Answers

You can make use of a larger alphabet and make hash shorter but still reversible to original value.

I implemented it here - for example, hash ee45187ab28b4814cf03b2b4224eb974 becomes 7fBKxltZiQd7TFsUkOp26w - it goes from 32 to 22 characters. And it can become even less if you use a larger alpahabet. If you use unicode, you can even encode hash with emoji...

This probably won't be of use to the OP since they were looking for 2 way function but may help someone looking for a shorter hash than md5. Here is what I came up with for my needs (thanks to https://rolandeckert.com/notes/md5 for highlighting the base64_encode function). Encode the md5 hash as base(64) and remove any undesirable base(64) characters. I'm removing vowels + and / so reducing the effective base from 64 to 52.

Note if you truncate a base(b) encoded hash after c characters it will allow for b ^ c unique hashes. Is this robust enough to avoid collisions? It depends on how many items (k) you are hashing. The probability of collision is roughly (k * k) / (b ^ c) / 2, so if you used the function below to hash k = 1 million items with base b = 52 encoding truncated after c = 12 characters the probability of collision is < 1 in 750 million. Compare to truncating the hex encoded (b = 16) hash after c = 12 characters. The probability of collision is roughly 1 in 500! Just say no to truncating hex encoded hashes. :)

I'll go out on a limb and say the function below (with length 12) is reasonably safe for 10 million items (< 1 in 7.5 million probability of collision), but if you want to be extra safe use base(64) encoding (comment out the $remove array) and/or truncate fewer characters.

// convert md5 to base64, remove undesirable characters and truncate to $length
function tinymd5($str, $length) { // $length 20-22 not advised unless $remove = '';
    // remove vowels to prevent undesirable words and + / which may be problematic
    $remove = array('a', 'e', 'i', 'o', 'u', 'A', 'E', 'I', 'O', 'U', '+', '/');
    $salt = $str;
    do { // re-salt and loop if rebase removes too many characters
        $salt = $base64 = base64_encode(md5($salt, TRUE));
        $rebase = substr(str_replace($remove, '', $base64), 0, $length);
    } while ($length < 20 && substr($rebase, -1) == '=');
    return str_pad($rebase, min($length, 22), '='); // 22 is max possible length
}

$str = 'Lorem ipsum dolor sit amet 557726776';
echo '<br />' . md5($str);         // 565a0bf7e0ba474fdaaec57b82e6504a
$x = md5($str, TRUE);
echo '<br />' . base64_encode($x); // VloL9+C6R0/arsV7guZQSg==
echo '<br />' . tinymd5($str, 12); // VlL9C6R0rsV7
echo '<br />' . tinymd5($str, 17); // VlL9C6R0rsV7gZQSg
$x = md5(base64_encode($x), TRUE); // re-salt triggered < 20
echo '<br />' . base64_encode($x); // fmkPW/OQLqp7PTex0nK3NQ==
echo '<br />' . tinymd5($str, 18); // fmkPWQLqp7PTx0nK3N
echo '<br />' . tinymd5($str, 19); // fmkPWQLqp7PTx0nK3NQ
echo '<br />' . tinymd5($str, 20); // fmkPWQLqp7PTx0nK3NQ=
echo '<br />' . tinymd5($str, 22); // fmkPWQLqp7PTx0nK3NQ===

$hashlen = 4;
$cxtrong = TRUE;
$sslk = openssl_random_pseudo_bytes($hashlen, $cxtrong);
$rand = bin2hex($sslk);

echo $rand;

You can change the hash length (in multiples of two) by changing the value of the variable $hashlen

I came up with base 90 for reducing md5 to 20 multi-byte characters (that I tested to fit properly in a mysql's varchar(20) column). Unfortunately this actually makes the string potentially larger than even the 32 bytes from php's md5, with the only advantage that they can be stored in varchar(20) columns. Of course you could just replace the alphabet with single-byte ones if your worries are about storage...

There are a couple of rules that are important to have in mind if your idea is to use this reduced hash as a lookup key in something like mysql and for other kinds of processing:

  1. By default MySQL does not differentiate Upper Case from Lower Case in a typical where clause which takes out a lot of characters right out of the possible target alphabets. This include not only english character but also almost all characters in other languages.

  2. It's important that your hash can be upper-cased and lower-cased transparently since many systems uppercase these keys, so to keep it consistent with md5 in that sense you should use only lowercase when using case-able characters.

This is the alphabet I used (I handpicked each character to make the hashes as nice as possible):

define('NICIESTCHARS', [
    "0","1","2","3","4","5","6","7","8","9",
    "a","b","c","d","e","f","g","h","i","j","k","l","m","n","o","p","q","r","s","t","u","v","w","x","y","z",
    "¢","£","¥","§","¶","ø","œ","ƒ","α","δ","ε","η","θ","ι","λ","μ","ν","π","σ","τ","φ","ψ","ω","ћ","џ","ѓ","ѝ","й","ќ","ў","ф","э","ѣ","ѷ","ѻ","ѿ","ҁ","∂","∆","∑","√","∫",
    "!","#","$","%","&","*","+","=","@","~","¤","±"
]);

Here is the code in PHP (I suppose it's not the best code but does the job). And keep in mind that it only works for strings in hexa (0-F) that are a multiple of 8 in length like md5 in php which is 32 0-f bytes:

function mbStringToArray ($string) {
    $strlen = mb_strlen($string);
    while ($strlen) {
        $array[] = mb_substr($string,0,1,"UTF-8");
        $string = mb_substr($string,1,$strlen,"UTF-8");
        $strlen = mb_strlen($string);
    }
    return $array;
} 

class Base90{ 
    static function toHex5($s){
        // Converts a base 90 number with a multiple of 5 digits to hex (compatible with "hexdec").
        $chars = preg_split('//u', $s, null, PREG_SPLIT_NO_EMPTY);
        $map = array_flip(NICIESTCHARS);
        $rt = '';
        $part = [];
        $b90part = '';
        foreach($chars as $c){
            $b90part .= $c;
            $part[] = $map[$c];
            if(count($part) == 5){
                $int = base90toInt($part);
                $rt .= str_pad(dechex($int), 8, "0", STR_PAD_LEFT);
                $part = [];
                $b90part = '';
            }
        }
        return $rt;
    }
    
    static function fromHex8($m){
        // Converts an hexadecimal number compatible with "hexdec" to base 90
        $parts = [];
        $part = '';
        foreach(str_split($m) as $i => $c){
            $part.= $c;
            if(strlen($part) === 8){
                $parts[] = intToBase90(hexdec($part));
                $part = '';
            }
        }
        return implode('', $parts);
    }
}



function intToBase90($int){
    $residue = $int;
    $result = [];
    while($residue){
        $digit = $residue % 90;
        $residue -= $digit;
        $residue = $residue / 90;
        array_unshift($result,  NICIESTCHARS[$digit]);
    }
    $result = implode('', $result);
    return $result;
}

function base90toInt($digits){
    $weight = 1;
    $rt = 0;
    while(count($digits)){
        $rt += array_pop($digits)*$weight;
        $weight *= 90;
    }
    return $rt;
}
Related