Why use the C# class System.Random at all instead of System.Security.Cryptography.RandomNumberGenerator?

Speed and intent. If you're generating a random number and have no need for security, why use a slow crypto function? You don't need security, so why make someone else think that the number may be used for something secure when it won't be?

Apart from the speed and the more useful interface (NextDouble() etc) it is also possible to make a repeatable random sequence by using a fixed seed value. That is quite useful, amongst others during Testing.

Random gen1 = new Random();     // auto seeded by the clock
Random gen2 = new Random(0);    // Next(10) always yields 7,8,7,5,2,....

System.Random is much more performant since it does not generate cryptographically secure random numbers.

A simple test on my machine filling a buffer of 4 bytes with random data 1,000,000 times takes 49 ms for Random, but 2845 ms for RNGCryptoServiceProvider. Note that if you increase the size of the buffer you are filling, the difference narrows as the overhead for RNGCryptoServiceProvider is less relevant.

The most obvious reasons have already been mentioned, so here's a more obscure one: cryptographic PRNGs typically need to be continually be reseeded with "real" entropy. Thus, if you use a CPRNG too often, you could deplete the system's entropy pool, which (depending on the implementation of the CPRNG) will either weaken it (thus allowing an attacker to predict it) or it will block while trying to fill up its entropy pool (thus becoming an attack vector for a DoS attack).

Either way, your application has now become an attack vector for other, totally unrelated applications which – unlike yours – actually vitally depend on the cryptographic properties of the CPRNG.

This is an actual real world problem, BTW, that has been observed on headless servers (which naturally have rather small entropy pools because they lack entropy sources such as mouse and keyboard input) running Linux, where applications incorrectly use the /dev/random kernel CPRNG for all sorts of random numbers, whereas the correct behavior would be to read a small seed value from /dev/urandom and use that to seed their own PRNG.

If you're programming an online card game or lotter then you would want to make sure the sequence is next to impossible to guess. However, if you are showing users, say, a quote of the day the performance is more important than security.

Not everyone needs cryptographically secure random numbers, and they might benefit more from a speedier plain prng. Perhaps more importantly is that you can control the sequence for System.Random numbers.

In a simulation utilizing random numbers you might want to recreate, you rerun the simulation with the same seed. It can be handy for tracking bugs when you want to regenerate a given faulty scenario as well - running your program with the exact same sequence of random numbers that crashed the program.

If I don't need the security, i.e., I just want a relatively indeterminate value not one that's cryptographically strong, Random has a much easier interface to use.