I am trying to scrape some data from OLX, there are multiple pages, so I have added loops. My code works for 1st 4 iterations but after i = 3 I get error 404, even though the sites exist beyond i=3 they give 404 btw I can open them on the browser. I tried changing user agent but didn't work. Site for page 4: https://www.olx.com.pk/mobiles_c1411/q-samsung-galaxy-s10?page=4&sorting=asc-price Please help
import csv
from urllib.request import urlopen as ureq
from bs4 import BeautifulSoup as soup
import time
import requests
headers = {'USER-AGENT': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36'}
for i in range(0, 8, 1):
page_html = requests.get('https://www.olx.com.pk/mobiles_c1411/q-samsung-galaxy-s10?page='+ str(i) + '&sorting=asc-price', headers=headers, timeout=10)
print(page_html.status_code)