403 Forbidden when trying to download a video from a link - Python

user_14524684 0
Viewed 1587

Overview

The website I'm using is Gogoanime, which allows users to watch and download anime. I want to automate the download process. It's also worth noting that I'm only making this for personal use.

! PLEASE HAVE AN AD BLOCKER BEFORE CLICKING ANY OF THE LINKS !

Here's the website --> https://www2.gogoanime.cm/ Here's a an episode example -- > https://www2.gogoanime.cm/death-note-dub-episode-1

Each episode has a download button. This is the download page for the above episode. -- > https://gogoplay1.com/download?id=OTA3OTk=&typesub=Gogoanime-DUB&title=Death+Note+%28Dub%29+Episode+1

If you click on any of the link downloads, the episode will start downloading. The download will also start if you select "save link as". However, there will be a 403 error if you paste the link into a new tab. I've already been able to scrape these links, but I can't download them through Python. It seems like these link downloads are from "vidstreaming", which was later renamed to gogoplay1 --> https://gogoplay1.com/.

I've tried many things, such as fake user agents, modifying cookies, and using cloudflare, but every time I request the download link, I'm hit with a 403 forbidden error.

It's also worth noting that this package -- > https://pythonrepo.com/repo/BaraniARR-anikimiapi-python-third-party-apis-wrappers, while is currently broken, mentions something regarding "tokens". I spent a very long time going over the code, but I can't get the tokens to help me.

I read somewhere that I needed to be authenticated into some servers, but I couldn't find the authentication type for gogoanime or gogoplay1.

Minimum reproducible example:

Here is one of the described download links -- > https://gogo-cdn.com/download.php?url=aHR0cHM6LyAawehyfcghysfdsDGDYdgdsfsdfwstdgdsgtert9AdrefsdsdfwerFrefdsfrersfdsrfer36343534jZG41LmFuaWNkbi5zdHJlYW0vdXNlcjEzNDIvN2ZjZmYzYzBkYjgxNWQ5MTIzNzI1MzA3MWI3ZTc0NzIvRVAuMS52MC4xNjM5MTc0MzgyLjcyMHAubXA0P3Rva2VuPVdIRVVaOUVjd0lJLU9iUXAwcGhXTXcmZXhwaXJlcz0xNjQxMjcyNDA4JmlkPTkwNzk5

You can run this code for more (of different quality)

import requests_html 

url = "https://gogoplay1.com/download?id=OTA3OTk=&typesub=Gogoanime-DUB&title=Death+Note+%28Dub%29+Episode+1"
session = requests_html.HTMLSession()
response = session.get(url)

links = response.html.absolute_links
for link in links:
    if "gogo-cdn" in link:
        print(link)

Here's the code that fails to download from the link.

from fake_useragent import UserAgent
import cloudscraper
import requests

url = "https://gogo-cdn.com/download.php?url=aHR0cHM6LyAawehyfcghysfdsDGDYdgdsfsdfwstdgdsgtert9AdrefsdsdfwerFrefdsfrersfdsrfer36343534jZG41LmFuaWNkbi5zdHJlYW0vdXNlcjEzNDIvN2ZjZmYzYzBkYjgxNWQ5MTIzNzI1MzA3MWI3ZTc0NzIvRVAuMS52MC4xNjM5MTc0MzgyLjcyMHAubXA0P3Rva2VuPVdIRVVaOUVjd0lJLU9iUXAwcGhXTXcmZXhwaXJlcz0xNjQxMjcyNDA4JmlkPTkwNzk5"
my_session = cloudscraper.create_scraper()
for_cookies = my_session.get("https://www.gogoplay1.com/") #don't know whether to use gogoanime or gogoplay1
cookies = for_cookies.cookies

response = my_session.get(url, headers={"User-Agent":UserAgent().chrome}, cookies=cookies)
print(response.status_code)

Lastly, I should mention that I have a decent grasp on Python, but next to none when it comes to scraping/web stuff. So, jargon and complicated processes might go right over my head. Thanks for reading this long post and thanks in advance for any help.

python-requests http-status-code-403 https download python
2 Answers

Try this,

from bs4 import BeautifulSoup
import cloudscraper

url = 'https://gogoplay.io/download?id=MTE3MTk0&title=One+Punch+Man+%28Dub%29&typesub=SUB&sub=&cover=Y292ZXIvb25lLXB1bmNoLW1hbi1kdWIucG5n&refer=https://gogoplay1.com/&ch=d41d8cd98f00b204e9800998ecf8427e'

scraper = cloudscraper.create_scraper(browser = 'chrome')
response = scraper.get(url)
if response.status_code == 200:
    print("downloads page")
    soup = BeautifulSoup(scraper.get(url).text, 'lxml')
    download_low_p_url = soup.find('div', {'class': 'mirror_link'}).find_all('div', {'class': 'dowload'})[0].find('a')['href'] # List contains all the links [360, 480, 780, 1080]. I choose 0 (first, 360p).
    print("getting cdn link for lowest quality", download_low_p_url)
    response = scraper.get(download_low_p_url)
    if response.status_code == 302:
        print("downloading from cdn", response.url)
        response = scraper.get(response.url, headers = {'Referer': 'https://gogoplay1.com/'}) #Referer is necessary else it throws 403
        if response.status_code == 200:
            with open('ep1.mp4', 'wb') as file:
                file.write(response.content)
            print("download complete")
        else:
            print("download error", response.status_code)
    else:
        print("cdn error", response.status_code)
else:
    print("base error", response.status_code)

On checking the requests trail, the download links has a redirecting url with a time bound token included as args. So, I initially sent a request to the downloads page, then to the lowest quality (just to save time on download, server is slow) which have a location in the response header (also known as redirecting url). Finally, we can send the redirect url a request and write the contents to a file.

This is like a skeleton. Add your features (such as option to select quality). Happy learning!

user_77624700

Ran into the same problem, although I was using selenium. Im currently using this workaround.

from selenium import webdriver
from selenium.webdriver import ActionChains
from selenium.webdriver.common.by import By
from selenium.webdriver.firefox.options import Options

url = "https://gogoplay.io/download?id=NzEzNjE=&typesub=Gogoanime-SUB&title=Boku+no+Hero+Academia+Episode+1"
download_dir = "some\\directory" #NOTE: This path MUST use backslashes

opt = Options()#A bunch of options to set the download directory and stop the browser from asking where to download to
opt.set_preference("browser.download.folderList", 2)
opt.set_preference('browser.download.dir', os.path.abspath(download_dir))
opt.set_preference("browser.helperApps.alwaysAsk.force", False)
opt.set_preference("browser.download.manager.showWhenStarting", False)
opt.set_preference("browser.download.manager.showAlertOnComplete", False)
opt.set_preference('browser.helperApps.neverAsk.saveToDisk','video/mp4')
driver = webdriver.Firefox(options=opt)
action = ActionChains(self.driver)

driver.get(url)
dl_button = driver.find_element(By.LINK_TEXT, "DOWNLOAD (1080P - MP4)") # Change to whatever quality you want

action.move_to_element(dl_button).click().perform()
action.move_to_element(dl_button).click().perform() # Because the first one triggers an ad, not the download


import sys
sys.exit(1) # Throw an error to stop the browser from closing before the download completes. 
# Pretty sure there's a cleaner way to stop the browser from closing but this works

The code clicks on the download button instead of trying to get the video directly from the source.

Some caveats though, selenium requires you to download the browser and web driver for that browser. Also, expect the download speed to be much slower than directly downloading from the source. Before when I could hit the source "gogo-cdn" directly, 1 episode took about 20 seconds to download. Now, using the download page, it takes me 10 minutes at best. Sometimes much slower. Not too big of a problem for me but it may be for you.

If you do find a way to get around the 403 issue, please do share though. Thanks!

user_67118510
Related