Why can't we change values of a dictionary while enumerating its keys?

class Program
{
    static void Main(string[] args)
    {
        var dictionary = new Dictionary<string, int>()
        {
            {"1", 1}, {"2", 2}, {"3", 3}
        };

        foreach (var s in dictionary.Keys)
        {
            // Throws the "Collection was modified exception..." on the next iteration
            // What's up with that?

            dictionary[s] = 1;  
        }
    }
}

I completely understand why this exception is thrown when enumerating a list. It seems reasonable to expect that during enumeration the structure of the enumerated object does not change. However, does changing a value of a dictionary also changes its structure? Specifically, the structure of its keys?