I made a script to scrape data from a webpage which is cloudflare protected. I was scraping around 25k links from this website and the script was working fine. I have been able to extract all the links from this website and now want to scrape information from these links. Earlier the script was working well but because of recent security update in website, I am getting error 503 by requests library and "checking your browser" webpage by selenium. Is there any way to bypass it?
I also have scraper api subscription to make requests using proxies and using "scraper_api" library for the same.
I am sharing some of the links that needs to be scraped but getting these errors:
https://coinatmradar.com/bitcoin_atm/31285/bitcoin-atm-general-bytes-birmingham-altadena-spirits/
https://coinatmradar.com/bitcoin_atm/23676/bitcoin-atm-general-bytes-birmingham-marathon-gas/
Already tried other approaches like cfscraper, cloud scraper, undetected chromedriver, but no luck. Please try to scrape any other link and share any solution. Thanks