Why does PyTorch's built-in loss function only work with the Long tensor type?

I wonder why exactly targets need to be a 64-bit integer and not 32-bit?

This is because PyTorch is precompiled. Some other frameworks that preceded it (not Tensorflow) would invoke the compiler on-the-fly, which can cause a delay and some other unpleasantness. PyTorch doesn't. But this means the developers have to be mindful of the size of the precompiled library. They have to balance the utility of supporting yet another data type vs the increase in size that compiling everything for that data type would cause, and the decision here went against int32, uint32, uint64, etc.