How to bypass Cloudflare with Python on GET requests?

user_18376864 0
Viewed 2995

I want to bypass Cloudflare on a GET request I have tried using Cloudscraper which worked for me in the past but now seems decreped.

I tried:

import cloudscraper

ses = requests.Session()
ses.headers = {
    'referer': 'https://magiceden.io/',
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.84 Safari/537.36',
    'accept': 'application/json'
}
scraper = cloudscraper.create_scraper(sess=ses)
hookLink = f"https://magiceden.io/launchpad/planetarians"
meG = scraper.get("https://api-mainnet.magiceden.io/launchpads/planetarians")
print(meG.status_code)
print(meG.text)

The issue seems to be that Im getting a captcha on the request

python-requests cloudflare python
1 Answers

The python library works well (I never knew about it), the issue is your user agent. Cloudflare uses some sort of extra checks to determine whether you're faking it.

For me, any of the following works:

ses.headers = {
    'referer': 'https://magiceden.io/',
    'accept': 'application/json'
}

ses.headers = {
    'accept': 'application/json'
}

And also just:

scraper = cloudscraper.create_scraper()
meG = scraper.get("https://api-mainnet.magiceden.io/launchpads/planetarians")

EDIT:

You can use this dict syntax instead to fake the user agent (as per the manual)

scraper = cloudscraper.create_scraper(
    browser={
        'browser': 'chrome',
        'platform': 'windows',
        'desktop': True
    }
)
user_104993980
Related