I'm trying to web scrape a website but I got the following error :
<Response [403]>
When I display the output I got this :
<body style="margin:0"><p id="cmsg">Please enable JS and disable any ad blocker</p><script>var dd={\'cid\':\'AHrlqAAAAAMAFuvaKabaZaMAVsjd8A==\',\'hsh\':\'05B30BD9055986BD2EE8F5A199D973\',\'t\':\'fe\',\'s\':2089,\'host\':\'geo.captcha-delivery.com\'}</script><script src="https://ct.captcha-delivery.com/c.js"></script>
After checking on the net, I've find that we need to add a header otherwise the websit will know that is a bot. I've added it but still the error.
The code I'm using :
from IPython.display import Image
import requests
from bs4 import BeautifulSoup
import requests_cache
import pandas as pd
requests_cache.install_cache("bases_scraping", expire_after=10e5)
url = "https://www.leboncoin.fr/recherche?category=55&text=electric%20bike&locations=r_12"
headers = {"user-agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:81.0) Gecko/20100101 Firefox/81.0"}
response = requests.get(url, headers=headers)
print(response)
print(response.content, "html.parser")
If someone can help me.
Thanks