Python3.4+requests 2.26 can't verify SSL certificat when the HTTPS site use CA of "ISRG Root X1" - why?

user_15609336 0
Viewed 253

My environment is:
windows 10, Python 3.4.3, Requests 2.26.0

Code is like this:

requests.get("https://example.com/")

When CA of SSL certificate of example.com is not ISRG root X1 (letsencrypt), the above call is ok.
(e.g： https://www.baidu.com/ just normal certificate) However, when CA of example.com's certificate IS "ISRG root X1 ", there will be errors below: (e.g: https://www.farmersworld.io its certificate is issued by letsencrypt)

And I checked the certifi.where() and the cacert.pem file in which I see both CA content of above, in other words, I did see the CA certificate content of ISRG in cacert.pem. So I was confused why this happened. Both CA of normal website and letsencrypt website are in cacert.pem file, why they have different behaviors? Also, when I deleted the CA of normal website from cacert.pem, the normal site will not be access by HTTPS then which proved that the cacert.pem is used by requests library indeed.

Traceback (most recent call last):
  File "C:\Python34\lib\site-packages\urllib3\connectionpool.py", line 706, in urlopen
    chunked=chunked,
  File "C:\Python34\lib\site-packages\urllib3\connectionpool.py", line 382, in _make_request
    self._validate_conn(conn)
  File "C:\Python34\lib\site-packages\urllib3\connectionpool.py", line 1010, in _validate_conn
    conn.connect()
  File "C:\Python34\lib\site-packages\urllib3\connection.py", line 426, in connect
    tls_in_tls=tls_in_tls,
  File "C:\Python34\lib\site-packages\urllib3\util\ssl_.py", line 450, in ssl_wrap_socket
    sock, context, tls_in_tls, server_hostname=server_hostname
  File "C:\Python34\lib\site-packages\urllib3\util\ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "C:\Python34\lib\ssl.py", line 365, in wrap_socket
    _context=self)
  File "C:\Python34\lib\ssl.py", line 583, in __init__
    self.do_handshake()
  File "C:\Python34\lib\ssl.py", line 810, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python34\lib\site-packages\requests\adapters.py", line 449, in send
    timeout=timeout
  File "C:\Python34\lib\site-packages\urllib3\connectionpool.py", line 756, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "C:\Python34\lib\site-packages\urllib3\util\retry.py", line 574, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='farmersworld.io', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)'),))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python34\lib\site-packages\requests\api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "C:\Python34\lib\site-packages\requests\api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Python34\lib\site-packages\requests\sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Python34\lib\site-packages\requests\sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python34\lib\site-packages\requests\adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='farmersworld.io', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)'),))```
lets-encrypt python-requests certificate https ssl
1 Answers

Aside: your text says www.farmersworld.io but your stacktrace says farmersworld.io; those are different sites, with different certificates, but both from LE with the compatibility chain and thus both with the same problem.

It's not Python as such that matters, but Python 3.4.3 dates to 2015 when OpenSSL 1.0.2 was current, and OpenSSL 1.0.2 when attempting to verify the chain LE uses by default (which bridges from ISRG's own root to the expired DST X3 root to support old Android versions) incorrectly fails when both the new ISRG root and the old/obsolete DST root are in the truststore.

Yes python requests does use by default the cacert file supplied by certifi. It's not clear to me if Mozilla has actually removed the obsolete cert; curl.se thinks so, but certifi doesn't seem to.

You could delete DST X3 manually from your cacert file, although modifying package-managed files is usually a bad idea; more cleanly you could make a copy and delete from that and set envvar REQUESTS_CA_BUNDLE and/or modify your code to use the copy. Alternatively you could use a python built with OpenSSL 1.1.0 up, which I expect anything from 2017 on should. Or if you have influence (or control) of the server, have them configure LE (e.g. via certbot) to use the 'ISRG-only' chain instead.

For the official explanations see https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/ and https://www.openssl.org/blog/blog/2021/09/13/LetsEncryptRootCertExpire/ .

user_28688010
Related