Surprising Substring behavior

I came across this behavior today while using the Substring method:

static void Main(string[] args) {
    string test = "123";
    for (int i = 0; true; i++) {
        try {
            Console.WriteLine("\"{0}\".Substring({1}) is \"{2}\"", test, i, test.Substring(i));
        } catch (ArgumentOutOfRangeException e) {
            Console.WriteLine("\"{0}\".Substring({1}) threw an exception.", test, i);
                break;
        }
    }
}

Output:

"123".Substring(0) is "123"
"123".Substring(1) is "23"
"123".Substring(2) is "3"
"123".Substring(3) is ""
"123".Substring(4) threw an exception.

"123".Substring(3) returns an empty string and "123".Substring(4) throws an exception. However, "123"[3] and "123"[4] are both out of bounds. This is documented on MSDN, but I'm having a hard time understanding why the Substring method is written this way. I'd expect any out-of-bounds index to either always result in an exception or always result in an empty string. Any insight?