How to add fade effect to color selector

I understand that we can add fade effect to drawable selector,

<selector xmlns:android="http://schemas.android.com/apk/res/android" android:enterFadeDuration="300" android:exitFadeDuration="300">

I'm using this for a custom button, which gives the fade effect to the background. I also have a text color selector for the button, but I can't add the fade effect to it. Any idea how could I achieve the same fade effect on the text as well?