I've written a Flask app that runs perfectly fine - exactly as I want it to, when using Flask's development server via flask run. It's a long web-scraping process using a lot of Google Cloud libraries.

After deploying to Google App Engine, I figured out that I had to wrap my Flask api with gunicorn. Okay, no problem, I installed it locally and ran it in the same way I did before. But suddenly, now, I am getting a completely new error which I have no idea how to debug - here is the stack trace:

[2020-07-20 05:26:45 -0400] [7354] [INFO] Starting gunicorn 20.0.4
[2020-07-20 05:26:45 -0400] [7354] [INFO] Listening at: http://127.0.0.1:8000 (7354)
[2020-07-20 05:26:45 -0400] [7354] [INFO] Using worker: eventlet
[2020-07-20 05:26:45 -0400] [7356] [INFO] Booting worker with pid: 7356
WARNING:root:course-collect manually triggered
ERROR:grpc._plugin_wrapping:AuthMetadataPluginCallback "<google.auth.transport.grpc.AuthMetadataPlugin object at 0x7fc3f7710970>" raised exception!
Traceback (most recent call last):
  File "/mnt/c/Users/*******/Projects/course_collect/venv/lib/python3.8/site-packages/grpc/_plugin_wrapping.py", line 77, in __call__
    self._metadata_plugin(
  File "/mnt/c/Users/*******/Projects/course_collect/venv/lib/python3.8/site-packages/google/auth/transport/grpc.py", line 84, in __call__
    callback(self._get_authorization_headers(context), None)
  File "/mnt/c/Users/*******/Projects/course_collect/venv/lib/python3.8/site-packages/google/auth/transport/grpc.py", line 70, in _get_authorization_headers
    self._credentials.before_request(
  File "/mnt/c/Users/*******/Projects/course_collect/venv/lib/python3.8/site-packages/google/auth/credentials.py", line 133, in before_request
    self.refresh(request)
  File "/mnt/c/Users/*******/Projects/course_collect/venv/lib/python3.8/site-packages/google/oauth2/service_account.py", line 359, in refresh
    access_token, expiry, _ = _client.jwt_grant(request, self._token_uri, assertion)
  File "/mnt/c/Users/*******/Projects/course_collect/venv/lib/python3.8/site-packages/google/oauth2/_client.py", line 153, in jwt_grant
    response_data = _token_endpoint_request(request, token_uri, body)
  File "/mnt/c/Users/*******/Projects/course_collect/venv/lib/python3.8/site-packages/google/oauth2/_client.py", line 105, in _token_endpoint_request
    response = request(method="POST", url=token_uri, headers=headers, body=body)
  File "/mnt/c/Users/*******/Projects/course_collect/venv/lib/python3.8/site-packages/google/auth/transport/requests.py", line 180, in __call__
    response = self.session.request(
  File "/mnt/c/Users/*******/Projects/course_collect/venv/lib/python3.8/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/mnt/c/Users/*******/Projects/course_collect/venv/lib/python3.8/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/mnt/c/Users/*******/Projects/course_collect/venv/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/mnt/c/Users/*******/Projects/course_collect/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "/mnt/c/Users/*******/Projects/course_collect/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "/mnt/c/Users/*******/Projects/course_collect/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 976, in _validate_conn
    conn.connect()
  File "/mnt/c/Users/*******/Projects/course_collect/venv/lib/python3.8/site-packages/urllib3/connection.py", line 342, in connect
    self.ssl_context = create_urllib3_context(
  File "/mnt/c/Users/*******/Projects/course_collect/venv/lib/python3.8/site-packages/urllib3/util/ssl_.py", line 276, in create_urllib3_context
    context.options |= options
  File "/usr/lib/python3.8/ssl.py", line 602, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  File "/usr/lib/python3.8/ssl.py", line 602, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  File "/usr/lib/python3.8/ssl.py", line 602, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  [Previous line repeated 476 more times]
RecursionError: maximum recursion depth exceeded while calling a Python object
malloc(): mismatching next->prev_size (unsorted)
[2020-07-20 05:27:29 -0400] [7361] [INFO] Booting worker with pid: 7361

I'm currently running my app with the command gunicorn --worker-class eventlet app:app (with app.py and app = Flask(__name__)).

When I switch to just gunicorn app:app gunicorn runs the same as Flask.

But the problem is, knowing that this api's endpoints take an "undefined amount of time", make "blocking calls", and otherwise requests'ing a lot of webpages, my app seems like the cookie cutter case for using async (eventlet/gevent) workers.

Choosing a Worker Type

The default synchronous workers assume that your application is resource-bound in terms of CPU and network bandwidth. Generally this means that your application shouldn’t do anything that takes an undefined amount of time. An example of something that takes an undefined amount of time is a request to the internet. At some point the external network will fail in such a way that clients will pile up on your servers. So, in this sense, any web application which makes outgoing requests to APIs will benefit from an asynchronous worker.

This resource bound assumption is why we require a buffering proxy in front of a default configuration Gunicorn. If you exposed synchronous workers to the internet, a DOS attack would be trivial by creating a load that trickles data to the servers. For the curious, Hey is an example of this type of load.

Some examples of behavior requiring asynchronous workers:

• Applications making long blocking calls (Ie, external web services)
• Serving requests directly to the internet
• Streaming requests and responses
• Long polling
• Web sockets
• Comet

Can someone point out the reason (if it's somewhat obvious) why async workers break my application (pasted below)

app.py

from flask import Flask
import logging
from firebase_admin import firestore, _apps, initialize_app, credentials
from google.cloud.storage import Client
from google.cloud.scheduler_v1 import CloudSchedulerClient
from google.api_core.exceptions import NotFound, GoogleAPICallError, PermissionDenied

app = Flask(__name__)

@app.route('/init')
def start_process():
    start_time = time()
    storage_client = Client()
    scheduler_client = CloudSchedulerClient()
    scheduler_path = scheduler_client.location_path(config.PROJECT_ID, config.REGION_ID)
    cred = credentials.ApplicationDefault()

    try:
        scheduler_client.delete_job(f"{scheduler_path}/jobs/{config.CRON_NAME}")
    except GoogleAPICallError or PermissionDenied:
        logging.warning("course-collect manually triggered")
    
    # I had more code here but even all commented out, this error still happened
    
    return "200 OK"