How can I set proxy in scrapy?

Viewed 25

I try proxy by requests like below:

proxies = {
            "http": "http://127.0.0.1:1080",
            "https": "http://127.0.0.1:1080"
            }
resp=requests.get(url,headers=headers,proxies=proxies)
with open("resp.html","w", encoding='UTF-8') as f:
    f.write(resp.text)    

But I try to use the proxies in scrapy:

yield scrapy.Request(            
                url=link,
                method='GET',
                headers=self.headers,
                callback=self.parse_detail,
                meta={"MyItem":item,"proxy": "https://127.0.0.1:1080"})

Scrapy always timeout! How can I set proxy in scrapy? Thanks!

0 Answers
Related