Can't bypass cloudflare with python cloudscraper

user_13204943 0
Viewed 23980

I faced with cloudflare issue when I tried to parse the website.

I got this code

import cloudscraper

url = "https://author.today"
scraper = cloudscraper.create_scraper()
print(scraper.post(url).status_code)

This code prints me

cloudscraper.exceptions.CloudflareChallengeError: Detected a Cloudflare version 2 challenge, This feature is not available in the opensource (free) version.

I searched for workaround, but couldn't find any solution. If visit the website via a browser you could see

Checking your browser before accessing author.today.

Is there any solution to bypass cloudflare in my case?

cloudflare web-scraping python
4 Answers

Although for this site is does not seem to work, sometimes adding some parameters when initializing the scraper helps:

import cloudscraper

url = "https://author.today"
scraper = cloudscraper.create_scraper(
    browser={
        'browser': 'chrome',
        'platform': 'android',
        'desktop': False
    }
)
print(scraper.post(url).status_code)
user_62720610

Install httpx

pip3 install httpx[http2]

Define http2 client

client = httpx.Client(http2=True)

Make request

response = await client.get("https://author.today")

Cheers!

user_165784830 
import cfscrape
from fake_useragent import UserAgent
ua = UserAgent()

s = cfscrape.create_scraper()

k = s.post("https://author.today", headers = {"useragent": f"{ua.random}"})
print(k)
user_180101620

I used this line: scraper = cloudscraper.create_scraper(browser={'browser': 'chrome','platform': 'windows','mobile': False})

and then used httpx package after that with httpx.Client() as s: //Remaining Code

And I was able to bypass the issue cloudscraper.exceptions.CloudflareChallengeError: Detected a Cloudflare version 2 challenge, This feature is not available in the opensource (free) version.

user_22841440
Related