airflow initdb: undefined symbol: Py_GetArgcArgv

user_12527238 0
Viewed 8644

I followed the documentation to install Apache-airflow. https://airflow.apache.org/docs/stable/start.html

When I execute airflow initdb, an error occurs every time.

x-MacBook-Pro:~ x$ airflow initdb
 ......
import airflow.utils.dag_processing
  File "/Library/Python/3.7/site-packages/airflow/utils/dag_processing.py", line 40, in <module>
    from setproctitle import setproctitle
ImportError: dlopen(/Library/Python/3.7/site-packages/setproctitle.cpython-37m-darwin.so, 2): Symbol not found: _Py_GetArgcArgv
  Referenced from: /Library/Python/3.7/site-packages/setproctitle.cpython-37m-darwin.so
  Expected in: flat namespace
 in /Library/Python/3.7/site-packages/setproctitle.cpython-37m-darwin.so

One answer suggested that this is a problem with the binary package. But I still don't know how to solve that. This is the link https://github.com/psycopg/psycopg2/issues/807.

MacOSX 10.15.3

pip 20.0.2

Python 3.7.3

python-wheel macos python
10 Answers

Of course I had the same issue. macOS 10.15.2.

I had python 3.8 globally installed. I did some research and I found out that the reason was the way that c library m-darwin.so was compiled.

This steps help me fixed the issue:

  1. Installed pyenv
brew install pyenv
  1. Downgraded to python 3.7.0
pyenv install 3.7.0
  1. Set python 3.7.0 globally
pyenv global 3.7.0
  1. Running python3 -V should give you:
Python 3.7.0
  1. Recreated my project environment using virtualenv:
python3 -m virtualenv --python=python3.7 my awesome_env
  1. Activated my env and installed all dependencies and it woorked.
source awesome_env/bin/activate
pip install -r requirements.txt

I think this whole process trigger the recompilation of the libraries.

user_38013400

I had the same problem, when I was using the system (OS) python3 interpreter, i.e. /usr/bin/python3

you can simply install python 3.7 or 3.8 (it works for both) with homebrew:

brew install python@3.8

Make sure python3 is now pointing to /usr/local/bin/python3 by running which python3.

Then install apache-airflow:

python3 -m venv .venv
pip install apache-airflow
airflow initdb
user_65423040
user_10543220

I had the same problem in python 3.7 but Works great for Python 3.8 installed directly from python page, create a new environment and install again apache-airflow in your new environment, be sure you have all dependencies for this version of apache-airflow and you are not using the local environment to avoid errors.

user_77582050

If you're tied to 3.7.x and using pipenv like me, running the latest patch helps.

brew install pyenv
pyenv install 3.7.10
pyenv global 3.7.10
pipenv --rm
pipenv install --dev --python 3.7.10
user_30260510

I had the same problem as you (MAC OS Catalina). The problem was with two python versions on the system (Python2.7 and Python3.7).

$ python -V
Python 2.7
$ pip -V
pip 20.0.2 from /Library/Python/3.7/site-packages/pip (python 3.7)
$ which python
/usr/bin/python
$ which pip
/usr/local/bin/pip

I resolved the problem by adding softlink for python

$ ln -s /usr/local/bin/python3 /usr/local/bin/python
$ python -V
Python 3.7.7

After this, reinstall apache-airflow. Worked for me both in local and virtualenv.

user_46500480
user_99543940

I had the same problem with python 3. Try creating an virtualenv of python 2.7 and install airflow in it.

user_28514750

Python 3.7.3 is causing this issue. I upgraded to Python 3.7.7 and issue got resolved.

mkvirtualenv -p python3.7.7 airflow
workon airflow
pip install apache-airflow
airflow initdb
user_33088050

I solved the problem by following these steps:

Note: the order of the steps matters!

  • Install the latest version of python, it was 3.8 now it is 3.9.13.
  • Create a virtual env.
  • Update the pip command by executing python3 -m pip install --upgrade pip
  • Install apache airflow pip install apache-airflow
user_11706770
Related