Get response 403 when i'm trying to crawling, user agent doesn't work in Python 3

Viewed 83

I'm trying to crawling this website and get the message:

"You don't have permission to access"

there is a way to bypass this ? already used user agents and urlopen

Here is my code:

import requests
from bs4 import BeautifulSoup
import json 
import pandas as pd
from urllib.request import Request, urlopen


url = 'https://www.oref.org.il/12481-he/Pakar.aspx'

header = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.76 Safari/537.36'}

res = requests.get(url, headers=header)

soup = BeautifulSoup(res.content, 'html.parser')

print(res)

output:

<Response [403]>

also tried to do that:

req = Request(url, headers={'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36'})

webpage = urlopen(req).read()

output:

HTTP Error 403: Forbidden

still blocked and get response 403, Anyone who can help?

0 Answers
Related