I am adding memoization to several functions. These functions take 2-3 string parameters (object names), an optional int parameter (record ID), and a boolean parameter (include deleted records). Each combination of parameters is guaranteed to produce a unique result (thus worth caching).
I am wondering if it would be faster to concatenate the given parameters ($param1 . $param2 . $param3 etc) and use that as the array key, or to take the same concatenated string and use the md5 hash as the key. The length of the concatenated parameter string is between 20-32 characters in 99% of cases (averaging around 27), whereas an md5 hash is always 32 characters.
Edit: an md5 hash is only 16 bytes, not 32. Thanks Mjh.
I am leaning toward the first option, as it:
- saves me the cost of performing the md5 hash
- it will usually save a few bytes of memory (27 average vs 32 hashed) (Mjh pointed out this is not true: md5 is only 16 bytes), and
- since an md5 hash is just another string it will usually be faster to compare the shorter string
The only reason I'm doubting this is that the vast majority of memoization functions seem to use (md5) hashes, so I am wondering if I'm missing something.
Thanks in advance.
P.S. I forgot to mention: I separate the individual parameters with a # character, which can never naturally occur in any of the parameters.
P.P.S. So far ankhzet's comment seems to be the best solution given my strings are practically unique to begin with: crc32($paramString). Small memory footprint, and a very quick checksum calculation function.
Testing crc32() performance
Below is a testing script which fills 4 arrays with 1 million key => value pairs each. The values of all 4 arrays are identical. The keys are also identical, except that for the first 2 arrays the concatenated string-keys first have crc32() run on them.
$test1Array = [];
$start1 = microtime(true);
for ($i = 0; $i < 1000000; $i++)
{
$test1Array[crc32("pagemanagement" . "#" . "staticblocktype" . "#" . $i . "#" . 1)] = "test " . $i;
}
$end1 = microtime(true);
$test2Array = [];
$start2 = microtime(true);
for ($j = 0; $j < 1000000; $j++)
{
$test2Array[crc32("pagemanagement" . "#" . "staticblocktype" . "#" . $i . "#" . 1)] = "test " . $j;
}
$end2 = microtime(true);
$test3Array = [];
$start3 = microtime(true);
for ($x = 0; $x < 1000000; $x++)
{
$test3Array["pagemanagement" . "#" . "staticblocktype" . "#" . $i . "#" . 1] = "test " . $x;
}
$end3 = microtime(true);
$test4Array = [];
$start4 = microtime(true);
for ($y = 0; $y < 1000000; $y++)
{
$test4Array["pagemanagement" . "#" . "staticblocktype" . "#" . $i . "#" . 1] = "test " . $y;
}
$end4 = microtime(true);
Results of 3 test runs:
Test 1: 3.9902291297913
Test 2: 3.6312079429626
Test 3: 0.91605305671692
Test 4: 0.91405177116394
Test 1: 3.9842278957367
Test 2: 3.6172070503235
Test 3: 0.91405200958252
Test 4: 0.918053150177
Test 1: 3.9842278957367
Test 2: 3.6282079219818
Test 3: 0.91205215454102
Test 4: 0.91605186462402
If I take the average of all "Test 2" and "Test 4" values (since "Test 1" seems to have initialisation overhead), I am left with 3.6255409717560 for "Test 2" and 0.9160522619883 for "Test 4". That is a difference of 2.7094887097677, and (2.7094887097677 / 1000000) = 0.0000027094887 or 2.72 microseconds per function call.
Unfortunately I can't easily calculate memory usage at the moment, but storing the 4 byte crc32() value is guaranteed to take significantly less memory than the average 27-character length strings. Assuming best-case scenario 1 byte characters, that is a difference of 23 bytes per cached result.
For completeness I ran a quick test with md5() as well:
Test 1: 4.2855787277221
Test 2: 3.8108838399251
I am actually surprised by how little performance difference there is between md5() and crc32(). Of course, crc32() still has the advantage of using only 4 bytes to md5()'s 16.
Conclusion: since the main overhead of my functions is in the repeated database calls, and since these functions are called on the order of around 50-200 times per request, I personally think the ~135-540 microseconds of added computing time is worth saving the ~1150-4600 bytes of memory.
If anyone disagrees with my tests and/or conclusion, I'd love to know.