Your type annotations should express your intentended commitments, and no more.
Anything which you consider an implementation detail and reserve the right to change should be annotated to maintain that encapsulation. Parameters should be annotated as the most general thing you want to commit to accepting. Return values should be annotated as the most specific thing you want to commit to your return value allowing.
Suppose that you have some code doing brute force factorisation, and this includes a class which produces some candidate prime numbers to loop through. Initially, you store the first 100 candidates in a list and expose them like so:
def primes(self):
return self._internal_primes
You could annotate that as List or as Sequence, because self._internal_primes is both of those. But you shouldn't.
The foremost reason that you shouldn't is the most philosophical. The intended purpose of this method is to supply some candidate primes to loop over. Iterable[int] is the right thing to convey that purpose, irrespective of what happens under the hood.
The second reason is encapsulation and the associated flexibility. You didn't intend to commit to exposing a private list as your implementation. Maybe you anticipate rewriting primes in future as an on-demand generator implementing a sieve. Commiting to List now means you'd have to break your interface promise to do so.
The third reason is that even if you never expect to change the implementation, you may not want to advertise the full capabilities of that implementation. For example, lists can be mutated. If you promise someone it's a list, you also promise that they can mutate it. There is simply nothing good to gain by allowing someone to write p.primes()[2] = 72. By exposing an interface which doesn't even have the mutation methods, such a mistake becomes much less likely to slip through undetected.