python requests via session proxy

user_13897549 0
Viewed 147

I am having a use case to get data from a specific site which needs to have requests via session every time. I have created the session in python and also cookies are set which contain my logged in details.

I am currently hosting my script on a data center but the account is getting blocked. I am thinking of requesting the data via proxy but still feel that if my session is created from a different machine and proxy is used to get data via session then what are the chances that the proxy ip is going to be black-listed?

What are the possible solutions here to cater this kind of problem.

web-crawler session proxy python
1 Answers

Scenarios in Python Requests Session differs according to different regions, Like in some countries some headers are not permitted in the request due to country laws.

Since you are already making logged in session that means your user-agent and other headers are being set according to the response from login request.

One of the solution might be using a proxy of a country which doesnot have strict rules of data extraction from that platfrom.

user_91956130
Related