Using request in python for CURL

user_16098755 0
Viewed 247

When I use CURL, I am getting the below output, which says that proxy was able to connect to the endpoint via proxy. CURL output was 200 connection established and later showing the 401 Unauthorized. I am OK as long as proxy was able to connect (200 connection established) but when I execute the python code, the python output is showing only 301. I only care about if the connection can be made over the proxy or not. Can I please how can I check the (200 connection established) like in the CURL output using the python?

export HTTPS_PROXY=<proxy_details> && curl -vvv https://www.sap.com


* About to connect() to proxy proxy.abc.local.port 1256 (#0)
*   Trying 169.31.123.234...
* Connected to proxy.abc.local.port () port 1256 (#0)
* Establish HTTP proxy tunnel to www.sap.com:443
> CONNECT www.sap.com:443 HTTP/1.1
> Host: www.sap.com:443
> User-Agent: curl/7.29.0
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 Connection established
< 
* Proxy replied OK to CONNECT request
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate:
*   subject: CN=www.sap.com,O=SAP SE,L=Walldorf,ST=Baden-Württemberg,C=DE
*   start date: May 26 00:00:00 2021 GMT
*   expire date: May 31 23:59:59 2022 GMT
*   common name: www.sap.com
*   issuer: CN=GeoTrust RSA CA 2018,OU=www.digicert.com,O=DigiCert Inc,C=US
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: www.sap.com
> Accept: */*
> 
< HTTP/1.1 301 Moved Permanently
< Content-Type: text/html; charset=iso-8859-1
< Content-Length: 238
< Strict-Transport-Security: max-age=15724800; includeSubDomains
< Location: https://www.sap.com/index.html
< Date: Sat, 24 Jul 2021 03:44:35 GMT
< Connection: keep-alive
< 
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="https://www.sap.com/index.html">here</a>.</p>
</body></html>
* Connection #0 to host   proxy.abc.local.port left intact

python code

import requests 
proxies = { 'https':'proxy.config.pcp.local:3128'}
res = requests.get('https://www.sap.com', proxies=proxies)
print (res.status_code)

output:

301
python-requests python-2.7 curl python-3.x python
2 Answers

The connection is not made over proxy too. It shows 200 Connection established at first because the socket connection is established but then it hits the redirect and it throws 301 Moved Permanently. When you are using python's request module it only shows the final status code (which is 301) and not the intermediate socket connections.

user_98552450

You should probably try to directly send only the CONNECT request then, to have a chance at looking at thats response code.

But I don't think Python requests allows that. Lower-level tools like http.client do though:

import http.client
conn = http.client.HTTPConnection("proxy.config.pcp.local", 1256)
conn.request("CONNECT", "www.sap.com:443",'',{"Host":"www.sap.com:443"})
response = conn.getresponse()
print(response.status)

(use http.client.HTTPSConnection in case of an HTTPS proxy)

user_73623960
Related