What's the point of __future__?

__future__ allows introducing new behavior into the language that would be backwards-incompatible. For example, in Python 2.2 - 2.7 you could do:

>>> 3/2
1
>>> from __future__ import division
>>> 3/2
1.5

__future__ means "we added this feature into the language, but we can't make it the default because it would break other code".

Why was __future__ introduced originally?

Let's say you have a Python 2 project, and want to transition it to Python 3. Rather than converting it in one "big bang" and hoping it still behaves the same under Python 3, developers can upgrade their project piece-by-piece, by first importing division for example and testing for regressions, then importing print_function and so on.

What is the use of __future__ in Python 3?

A "Python 4" release is not on the horizon. But two features have already been added to Python 3 this way:

+----------------|---------------|----------------+
|    feature     | __future__ in | implemented in |
|----------------|---------------|----------------|
|----------------|---------------|----------------|
| generator_stop |      3.5      |      3.7       |
|----------------|---------------|----------------|
|  annotations   |      3.7      |      3.10      |
+----------------|---------------+----------------+

Both of these changes were backwards-incompatible. This means the language maintainers made a big change to how generators worked that was ready for release in 3.5. However they "hid" it in the __future__ module, because they were worried a bunch of packages would break upon 3.5's release if they put it out directly.

There are always smaller backwards-incompatible changes in Python, for example the introduction of the async and await keywords in 3.7, which makes code that used them as variable names (async = 1) break. It is really a judgment call whether to introduce a feature directly, or as a __future__, depending on how much effort a migration is.

Thus putting it a feature in a __future__ with a DeprecationWarning allows for a transition period for maintainers, which allows opting-in to the feature early.

Software changes

This is the fundamental problem this all deals with. Software is always changing. It's soft after all. When you write some code today and run it, it behaves a certain way. Next week you'll decide that you don't like what it does and that it needs some changes. You change it, and run it, and it behaves differently now. If you'd be following a proper release workflow, you'd tag your software with version numbers and have now produced two different versions of your software.

Python is also just software, written by people which are just programmers. The impact of changes in the Python language are obviously a bit different than you changing your little application which you only run yourself. If the Python developers decided tomorrow that the next version of Python required semicolons at the end of every line to be more like other languages, it would require thousands and thousands of software developers to rewrite millions upon millions of lines of code to make it compatible with that new version of Python. Obviously that would be insane, so the Python developers aren't going to make such drastic changes and will ensure a reasonable amount of backwards compatibility with existing code.

Why changes?

So, why change anything at all? Why can't we just keep the current version of Python and the current version of our code and keep running that forever as is? Well, you could. That's a perfectly fine strategy, within limits. There are three main categories of change, which are reflected in the Python versioning scheme:

• X.Y.Z version changes: when Python version 3.7.1 is updated to 3.7.2, that's because bugs have been found in 3.7.1 that required fixing. Often these are security patches. Existing code should keep working as is, unless it has been directly affected by the bug in question. You should always keep your Python version in production updated with the latest patch version, to ensure you're not falling pray to security vulnerabilities.
• X.Y version changes: when Python releases a new version like 3.8, this is to introduce new features. You'll want to upgrade to that version if you're interested in using those new features, because they make your work easier or your code nicer. Those versions also sometimes introduce breaking changes, e.g. removing deprecated functions or changing existing functions in minor ways. While you're consciously updating to that new version because of new features anyway, you can also double check whether you might be affected by breaking changes and fix your code accordingly.
• X version changes. This is typically almost akin to an entirely new language, which happens very rarely. Python 3 introduced many changes over Python 2 which made a lot of code simply incompatible and required sometimes substantial rewriting of existing code. As a benefit, Python 3 contained significant improvements which made the switch worthwhile.

There are entire pages in the documentation devoted to those changes, and you should peruse them occasionally, and especially before any major upgrades: https://docs.python.org/3/whatsnew/index.html

So there are various incentives for keeping up with changes in the Python language, though they're all optional. If you want to stay on one specific Python version forever and ever, you can. Until you need to replace the hardware it's running on and you find that that version isn't compatible with your new replacement hardware…

Compatibility

There are different compatibility concerns. When new features are introduced, that introduces a minimum required version for any code using that feature. E.g., this code:

print(f'foo {bar} baz')

This uses f-strings, a syntactic feature introduced in Python 3.6. If you write this code, you require at least Python 3.6 to run it. Python 3.5 refuses to run this code, as it has no idea what f-strings are. However, this kind of change has no backwards compatibility concerns. The f'' syntax is simply a syntax error in older Python versions and simply doesn't work.

Changes which work on older versions of Python, but behave differently, are more of a concern. The current issue is with annotations:

foo: bar = 'baz'

The behaviour of this will change subtly in Python 3.10. That is a change that has been decided upon and that's coming. Starting in Python 3.10, the above code will behave like:

foo: 'bar' = 'baz'

In other words, annotations will be evaluated as strings, not as objects. For most developers that will make little difference, and will in fact solve certain issues with annotations as they work today. Hence why this change is being made. However, depending on what you do with annotations, this change can also break your code.

Futures

And thus we finally come to futures: you can opt into the new behaviour of annotations today if you're running at least Python 3.7 by importing the appropriate future:

from __future__ import annotations

If you do this in your code, your code will already evaluate annotations as strings today. This behaviour will become on-by-default in Python 3.10. In other words, this change has been implemented in Python 3.7, but is optional and opt-in. It will become mandatory in 3.10. At that point, from __future__ import annotations will simply do nothing; whether or not you specify it, your code will behave one way and one way only.

This gives developers three major versions, or about three full years time, to update their code if they need to. That way nobody will be surprised by their code breaking if and when they try to run their existing code on Python 3.10 one day. (Unless they've been living under a rock those past years, and there will be people like that, unfortunately. But the impact will be greatly minimised.)

So, that's what futures are for. They implement breaking changes, but hide those changes behind an explicit opt-in import, in order to allow a sliding window within which developers can upgrade their code so as not require thousands of developers to have to rewrite their code in a panic when they're suddenly confronted with breaking code in production because their production Python version got upgraded for some reason or another.

And it's not just a direct relationship between the Python developers and coders like yourself; you'll often depend on 3rd party libraries, and those will need to be updated too. Every change to the Python language has a slow trickle-down effect from Python to library developers to repository maintainers to coders like you. This simply requires time, and futures buy this kind of time.

There have been a ton of those kinds of changes in the years when Python was transitioning from Python 2 to Python 3, and many of the breaking changes introduced in Python 3 could slowly be transitioned into with the use of futures. At the time of writing, there's only the aforementioned annotations change which is currently of active concern. But that doesn't preclude Python from introducing other, new, breaking changes which will be rolled out through futures in… the future.

Those points you cite from the documentation that you don’t understand aren’t very important and are just internal implementation details. In a nutshell, even though futures are opted into using from ... import ... statements, those aren’t actual module imports. They’re compiler flags that just look like they import an actual module. (An actual module import wouldn’t have the kind of power to make the changes those futures actually effect.) What you cite from the documentation is merely explaining that because those aren’t actual imports, some tools may get confused if they do try to treat those like actual module imports. For that purpose alone, there does exist a valid “__future__” module, and that documentation you cite is just about that and explains why it exists even when it has no actual effect.

Use __future__ when you want to be proactive in modernizing your code.

In many cases, you will find yourself using it when you deem a new feature worth it for your project, yet you still need to support older Python versions.

However, you can also use it to reduce your tech debt around future breaking changes as well as avoiding version splits in your project.