server sends 403 status code when using requests library in python, but works with browser

user_13775115 0
Viewed 57

I'm trying to automate a login using python's requests module, but whenever I use the POST or GET request the server sends 403 status code; the weird part is that I can access that same URL with any browser but it just won't work with curl and requests. here is the code:

import requests
import lxml
from bs4 import BeautifulSoup
import os

url = "https://ais.usvisa-info.com/en-am/niv/users/sign_in"
req = requests.get(url).text
soup = BeautifulSoup(req, 'lxml')
ready = soup.prettify()

FILE = open("usvisa.html", "w")
FILE.write(ready)
FILE.close()

I'd appreciate any help or idea!

python-requests get https post python
1 Answers

Its probably the /robots.txt, thats blocking you. try overriding the user-agent with a custom one.

import requests
import lxml
from bs4 import BeautifulSoup
import os

headers={"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36"}


url = "https://ais.usvisa-info.com/en-am/niv/users/sign_in"
req = requests.get(url, headers=headers).text
soup = BeautifulSoup(req, 'lxml')
ready = soup.prettify()

FILE = open("usvisa.html", "w", encoding="utf-8")
FILE.write(ready)
FILE.close()
  • you also didnt specify the file encoding when opening a file.
user_129547330
Related