Cache hit ratio effect on performance

user_1556026 0
Viewed 58

I'm currently reading the second edition of Systems Performance by Brendan Gregg and had a question on the section about caching in Chapter 2. This section defines cache hit ratio as

hit ratio = hits / (hits + misses)

It goes on to say that the relationship between cache hit ratio and "performance" (for some hypothetical measure of system performance) is nonlinear. Specifically,

The performance difference between 98% and 99% is much greater than that between 10% and 11%. This is a nonlinear profile because of the difference in speed between cache hits and misses - the two storage tiers at play. The greater the difference, the steeper the slope becomes.

I don't quite understand where the nonlinearity in this relationship originates from. In order to explain this to myself, I concocted the following example. Consider the following, we model performance by some function f, where a lower value of f denotes better performance.

f(hit) = 10
f(miss) = 100

i.e. misses are 10x more expensive than hits. Assuming a hit ratio of 0, the "expected" performance of this system will be (0*10) + (1*100) = 100. A hit ratio of .01 (1% hits) yields (.01*10)+(.99*100) = 99.1. Finally a hit ratio of .02 (2% hits) yields (.02*10) + (.98*100) = 98.2. AFAICT, this is a linear relationship. What am I missing?

Thanks

math system caching performance
1 Answers

I had the same question after reading the same book, and now that I have answered my question, so I will answer it.

Performance is nonlinear with hit rate because execution time and performance are different things.

As you wrote, equations with f (eg. (0*10) + (1*100) = 100) is linear in the hit ratio. However, it does not actually represent performance, but average execution time. Performance is relative, and when comparing two performances, we use the ratio of the inverse of the execution time.

This inverse is the source of the seemingly linear but actually nonlinear behavior.

I'm a non-native English speaker, so sorry if it's hard to read. I hope my answer helps you.

user_197115810
Related