Annotated field is not accessible from prefetched model

I have two models as:

class Training(models.Model):
    """Training records Model."""


class PersonTraining(models.Model):
    """Saves the trainings, the employee has completed or is enrolled in."""

    employee = models.ForeignKey(Person, related_name='records', on_delete=models.CASCADE)
    training = models.ForeignKey(Training, related_name='person_training', on_delete=models.CASCADE)

in the get_queryset method of the PersonTrainingView(viewsets.ModelViewSet) i have annotated Training model's objects with 'average_rating' and 'is_top_rated' fields

and then i have prefetched the training model in PersonTraining queryset as:

person_training_queryset = person_training_queryset.prefetch_related(
                              Prefetch('training', queryset=annotated_trainings)
                           )

The issue is that i need to sort the PersonTraining queryset based on the annotated fields of the prefetched Training model as:

person_training_queryset = person_training_queryset.order_by('-training__is_top_rated', '-training__average_rating')

But it is not accessible during execution in the get_queryset method and throws,

django.core.exceptions.FieldError: Cannot resolve keyword

kindly suggest a solution.

note: I am not annotating PersonTraining queryset with 'average_rating' and 'is_top_rated' fields, as i need to apply distinct on the PersonTraining queryset and it will through

'Error: annotate() + distinct(fields) is not implemented.'