First, note that = '\0' is the same as = 0. This is because'\0' has the numerical value of 0. Your enum has the default underlying type of int. The constant expression '\0' is converted to the int of 0 at compile time.
Second, as per the language spec, note that enum members can have duplicated associated values.
Third, from the the same section of the language spec,
The associated value of an enum member is assigned either implicitly
or explicitly. [...] If the declaration of the enum member has no
initializer, its associated value is set implicitly, as follows:
If the enum member is the first enum member declared in the enum type, its associated value is zero.
Otherwise, the associated value of the enum member is obtained by increasing the associated value of the textually preceding enum member
by one. This increased value must be within the range of values that
can be represented by the underlying type, otherwise a compile-time
error occurs.
So if I were to write out your enum's members' associated values explicitly, it would be:
public enum TokenType {
ILLEGAL = 0,
EOF = 0,
IDENT = 1,
INT = 2
};
ILLEGAL and EOF have the same associated value.
Fourth, Console.WriteLine calls ToString on your enum. Now look at what Enum.ToString does (in the Notes to Callers section):
If multiple enumeration members have the same underlying value and you
attempt to retrieve the string representation of an enumeration
member's name based on its underlying value, your code should not make
any assumptions about which name the method will return.
So, it outputting ILLEGAL is completely normal, as you "should not make any assumptions about which name the method will return".