celery worker throws AttributeError: module 'os' has no attribute 'register_at_fork'

user_5605353 0
Viewed 1405

I am running celery worker(version 4.4) on windows machine, when I run the worker with -P eventlet option it throws Attribute error. Error logs are as follows:-

pipenv run celery worker -A src.celery_app -l info -P eventlet --without-mingle --without-heartbeat --without-gossip -Q queue1 -n worker1
Traceback (most recent call last):
  File "d:\python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "d:\python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\G-US01.Test\.virtualenvs\celery-z5n-38Vt\Scripts\celery.exe\__main__.py", line 7, in <module>
  File "c:\users\g-us01.test\.virtualenvs\celery-z5n-38vt\lib\site-packages\celery\__main__.py", line 14, in main
    maybe_patch_concurrency()
  File "c:\users\g-us01.test\.virtualenvs\celery-z5n-38vt\lib\site-packages\celery\__init__.py", line 152, in maybe_patch_concurrency
    patcher()
  File "c:\users\g-us01.test\.virtualenvs\celery-z5n-38vt\lib\site-packages\celery\__init__.py", line 109, in _patch_eventlet
    eventlet.monkey_patch()
  File "c:\users\g-us01.test\.virtualenvs\celery-z5n-38vt\lib\site-packages\eventlet\patcher.py", line 334, in monkey_patch
    fix_threading_active()
  File "c:\users\g-us01.test\.virtualenvs\celery-z5n-38vt\lib\site-packages\eventlet\patcher.py", line 331, in fix_threading_active
    _os.register_at_fork(
AttributeError: module 'os' has no attribute 'register_at_fork'

I have installed eventlet in virtual environment, the pipfile contents are as follows:-

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]
rope = "*"
autopep8 = "*"

[packages]
eventlet = "*"
psutil = "*"
celery = "*"
pythonnet = "*"
redis = "*"
gevent = "*"

[requires]
python_version = "3.7"

Please let me know where I am going wrong.

celery flask python
4 Answers

os.register_at_fork is a new function available since Python 3.7, it is only available for Unix systems (Source from Python doc) and Eventlet use it to patch threading library.

There is an issue opened in Eventlet Github: https://github.com/eventlet/eventlet/issues/644

So I don't think Celery -A app worker -l info -P eventlet is any longer a good alternative for Windows.

Personally I ran:

Celery -A app worker -l info

And it worked with Windows 10, Python 3.7 and Celery 4.4.7.

user_121529370
user_124041010

As hinted at by @金奕峰, os.register_at_fork was introduced in eventlet v0.27.0 c.f. commit compare on github. Specifying version 0.26.0 in your virtual environment's package list might solve the problem (for now).

user_28872870

is the eventlet version 0.26; pip install eventlet==0.26

user_142612910
Related