I'm having trouble understanding when the empty string (epsilon) is a subset or element of an alphabet? My understanding was that epsilon was only part of a language, but my TA in the class said it was an element of all alphabets so now I am confused.
e.g. would {a,b,c} contain epsilon as an element?
e.g. would {} contain epsilon as an element?
e.g. is {eps} a subset of all alphabets or languages?
This question is probably better suited for cs.stackexchange but I will try to help you according to my understanding, please do correct me if necessary.
In general, your own intuition seems quite correct to me. ϵ is not automatically part of every alphabet. It is the empty string of characters.
However, this means that ϵ is a string over any alphabet, even your alphabet {a, b, c}.
So to answer your three examples:
No, it does not. If {a, b, c} is an alphabet, it is a set of symbols, and ϵ is a string. However, ϵ is definitely part of some languages defined over this alphabet.
No, {} is the empty set, and it contains nothing, not even ϵ.
{ϵ} is the set containing only ϵ. ϵ is a string, not a symbol, so it is not a subset of all alphabets (however, it seems there are cases where some alphabets are defined to contain ϵ but that is a different confusing story). It is also not a subset of all languages, because consider the language L = {aa, ab, ba, bb}. The empty string ∈ is not one of these elements.
The analogue to set theory might cause some additional confusion. Notice that the empty set is a subset of every set. The empty string is not a subset of every language, but rather it is a substring of any string.