GET request works with postman but not with python requests and curl

user_11555839 0
Viewed 5069

I have tried to create a specific GET request in python to the site : https://sgfm.elcorteingles.es/SGFM/dctm/MEDIA03/202006/24/00117731276964____5__210x210.jpg

But I dont get any response from the browser and its keep waiting forever. then I tried to create a curl request and same thing happens(keeps waiting forever). After all that, I tried to create a POSTMAN request and it works perfect! And I did not understood why it works with postman but not with python and curl as all these platforms are not browser like. I used the postman method to convert postman request to python and curl:

#python
import requests

url = "https://sgfm.elcorteingles.es/SGFM/dctm/MEDIA03/202006/24/00117731276964____5__210x210.jpg"

payload = {}
headers= {}

response = requests.request("GET", url, headers=headers, json = payload)

print(response.text.encode('utf8'))


#curl

curl --location --request GET 'https://sgfm.elcorteingles.es/SGFM/dctm/MEDIA03/202006/24/00117731276964____5__210x210.jpg'

but they both not getting any response. Does anyone know why this is happening and convert it to a python request? or even curl I can handle that too.

python-requests postman urllib curl python
2 Answers

Your headers do not get accepted by the server. I tried with these and it works for me.

headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.76 Safari/537.36', "Upgrade-Insecure-Requests": "1","DNT": "1","Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","Accept-Language": "en-US,en;q=0.5","Accept-Encoding": "gzip, deflate"}

Also, you need to change the request you make to that the requests.requests('GET') is not a proper way of making a GET request. The proper one is requests.get(url). They are the same methods, but for cleaner code you should stick to requests.get(url)

import requests

url = "https://sgfm.elcorteingles.es/SGFM/dctm/MEDIA03/202006/24/00117731276964____5__210x210.jpg"

payload = {}
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.76 Safari/537.36', "Upgrade-Insecure-Requests": "1","DNT": "1","Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","Accept-Language": "en-US,en;q=0.5","Accept-Encoding": "gzip, deflate"}

response = requests.get(url, headers=headers)

print(response.text.encode('utf8'))
user_131372200

Might be headers but might also be that Postman uses a system proxy. Set the same proxy in python using either

url = "https://example.com"
headers = {'User-Agent':...}
proxies = {"https":"<same postman proxy>"}
response = requests.get(url, headers=headers, proxies=proxies)

or

url = "https://example.com"
s = requests.Session()
s.proxies.update({"https":"<same postman proxy>"})
s.headers.update({"User-Agent":...})
s.get(url)
user_191076530
Related