When I run my flask app with worker-class=gevent on gunicorn, the server blocks.
gunicorn command gunicorn app:app --workers=5 --worker-class=gevent --threads=5 --timeout=1800 --log-level=DEBUG
Source code
query = '...'
query_job = bigquery_client.query(query)\
query_job.to_dataframe()# to_dataframe function where block occurs
- Source code where a block in the bigquery library occurs (lines 678 to 680 of the google/cloud/bigquery/_pandas_helpers.py file)
try:
frame = worker_queue.get(timeout=_PROGRESS_INTERVAL) # this point
yield frame
- Python library version
- python=3.7.10
- gunicorn=20.1.0
- gevnet=21.1.2
- eventlet=0.30.2
- google-cloud-bigquery=2.20.0
- google-cloud-bigquery-storage=2.4.0
- google-cloud-core=1.6.0
- pyarrow=4.0.0
The same happens when worker-class is an eventlet. It does not occur when worker-class is gthread or sync. The block in _pandas_helpers.py is executed in the following syntax, is it a thread-related problem?
with concurrent.futures.ThreadPoolExecutor(max_workers=total_streams) as pool:
Why do blocks happen?