Reinstall python packages after homebrew python@3.8 update

user_3303232 0
Viewed 1028

I have recently updated python to python@3.8 via homebrew.

I then noticed that my python scripts were unable to link to the global python packages that I had previously installed with pip (packages installed globally and not within a virtual environment).

This has sent me down the rabbit hole of understanding how python is set up on macos and I have several questions.

  1. pip installed by homebrew will now install all packages in /usr/local/lib/python3.8/site-packages.

    • Does that mean I need to reinstall all my python packages that used to live (and still are) in /usr/local/lib/python3.7? Should I remove the latter directory after? (For that matter I can see I have also a /usr/local/lib/python3.6 directory)
    • What's the efficient way of managing this in the future when the next python update comes along? Virtual envs per project (I'm not very familiar with the use of virtual envs)?

  2. When I run brew list | grep python I get the following list

python
python3
python@3.8

I can see python@3.8 is correctly symlinked as /usr/local/bin/python3 but I don't know what the python and python3 in the above list are since looking at /usr/local/Cellar I can see that I have an empty /usr/local/Cellar/python directory and a /usr/local/Cellar/python3 directory that is symlinked to it. Can I juste remove these two folders?

  1. Finally, looking into all this, I was surprised to discover a python3 executable living in /usr/bin. Looking into the sys.path when I execute /usr/bin/python3, I can see it's some sort of python installed by Xcode since it's looking for libraries in various Xcode.app directories. Since my /usr/local/bin is at the top of my PATH envt variable, this python3 will never be called but I wonder why I'd need it at all and whether I can just get rid of it.

I suppose the answer to a lot of these questions is to use virtual environments, which I've miraculously avoided doing for so long... Would love some help understanding the above though.

EDIT: Running brew info python or brew info python3 returns the same thing

python@3.8: stable 3.8.3 (bottled)
Interpreted, interactive, object-oriented programming language
https://www.python.org/
/usr/local/Cellar/python@3.8/3.8.3_2 (4,268 files, 65.5MB) *
  Poured from bottle on 2020-07-12 at 23:19:08
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/python@3.8.rb
homebrew macos python-3.x
1 Answers

I had a similar issue with a system running Python 3.9

I had initially not run the final command (only the --dry-run), as everything seemed fine, but as suggested during the installation, using brew link to clobber and create new symlinks was needed.

% brew link --overwrite --dry-run python@3.9
Would remove:
% brew link --overwrite python@3.9
Linking /usr/local/Cellar/python@3.9/3.9.1_1... 23 symlinks created
%

As suggested by MisterMiyagi in the comments, this was uncovered via brew doctor

...
Warning: You have unlinked kegs in your Cellar.
Leaving kegs unlinked can lead to build-trouble and cause brews that depend on
those kegs to fail to run properly once built. Run `brew link` on these:
  python@3.9
user_45410450
Related