Is a good practice to keep Retrofit instances in memory?

Should I cache objects generated by Retrofit or is it recommended to create on the fly?

I am starting a gradual migration from an old networking framework to Retrofit which started with a very basic logic to cache Retrofit created instances. This is due to our architecture which will use different endpoints in different Activities/Fragments, this method will be called as the user visit different parts of the app:

static public <T> T getApi(Class<T> apiInterface) {
  validateInitialization("getApi()");

  if (!mApis.containsKey(apiInterface.getSimpleName())) {
    mApis.put(apiInterface.getSimpleName(), retrofit().create(apiInterface));
  }

  return (T) mApis.get(apiInterface.getSimpleName());
}

However as we continue migrating our schemas to use Retrofit, this map is increasing in size (currently around 20 different interfaces/apis can be created) and sometimes contains instances that are no longer used.