I have a squid proxy set up in my lab replicating a production issue with the python requests library.
Squid Setup: - Client --> Squid over HTTPS - Squid is NOT doing any TLS Intercept/Bump - Works fine from cURL. - No password required - Valid certificate on squid server
Curl Output Works Fine:
curl -I -x https://squid-signed.hacksbrain.com:3128 https://www.google.com
HTTP/1.1 200 Connection established
HTTP/2 200
date: Sat, 04 Apr 2020 00:20:50 GMT
... Content from google.com ...
However, when I attempt to make the call from Python requests, I get the following error:
requests.exceptions.ProxyError: HTTPSConnectionPool(host='google.com', port=443): Max retries exceeded with url: / (Caused by ProxyError('Cannot connect to proxy.', ConnectionResetError(54, 'Connection reset by peer')))
I packet capture on the squid server and I see the 3-way handshake complete and a push come from the python client followed by a TCP RESET from the squid server.
I see the following entry in the squid log that corresponds to this connect attempt:
1585958924.312 0 192.168.1.100 NONE/000 0 NONE error:transaction-end-before-headers - HIER_NONE/- -
The CONNECT command to create the end-to-end tunnel is never sent from the python client.
def main():
proxies = {'https': 'https://squid-signed.hacksbrain.com:3128'}
headers = {'User-Agent': 'curl/7.64.1', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive'}
r = requests.get('https://google.com', headers=headers, proxies=proxies)
My suspicion is that the requests client does not know how to deal with the client --> squid over https vs over http. In my production environment this is required as it is an authenticated proxy sending credentials in the header as clear text, so the https connection to squid is a hard requirement.
UPDATE: I opened a capture from the working cURL session and from the python session in wireshark. I think I see the problem. Python requests is sending the CONNECT command to the proxy server in clear text and not inside of the https tunnel. I do not see this behavior from the working cURL capture.
