How to get a right video url of an Instagram post using python

user_13698622 0
Viewed 1092

I am trying to build a program which runs a function that input a url of a post, output the links of images and videos the post contain. It works really good for images. However, when it comes to get the links of videos, it return me a wrong url. I have no idea how to handle this situation.

https://scontent-lax3-2.cdninstagram.com/v/t50.2886-16/86731551_2762014420555254_3542675879337307555_n.mp4?efg=eyJ2ZW5jb2RlX3RhZyI6InZ0c192b2RfdXJsZ2VuLjcyMC5jYXJvdXNlbF9pdGVtIiwicWVfZ3JvdXBzIjoiW1wiaWdfd2ViX2RlbGl2ZXJ5X3Z0c19vdGZcIl0ifQ&_nc_ht=scontent-lax3-2.cdninstagram.com&_nc_cat=106&_nc_ohc=WDuXskvIuLEAX9rj7MU&vs=17877888256532912_3147883953&_nc_vs=HBksFQAYJEdCOXJLd1gyMVdhWUNkQUpBS090UGo3eEhDb3hia1lMQUFBRhUAAsgBABUAGCRHTFBXTUFVTXNPaG5XcW9EQU5KUEE5bEZVdVZxYmtZTEFBQUYVAgLIAQAoABgAGwGIB3VzZV9vaWwBMBUAABgAFuD4nJGH9sE%2FFQIoAkMzLBdAEszMzMzMzRgSZGFzaF9iYXNlbGluZV8xX3YxEQB17gcA&_nc_rid=97e769e058&oe=5EDF10A5&oh=3713c35f89fca1aa9554a281aa3421ed

https://scontent-gmp1-1.cdninstagram.com/v/t50.2886-16/0_0_0_\x00.mp4?_nc_ht=scontent-gmp1-1.cdninstagram.com&_nc_cat=100&_nc_ohc=Wnu_-GvKHJoAX9F_ui1&oe=5EDE8214&oh=7920ac3339d5bf313e898c3cbec85fa2

Here are two urls. The first one is copied from the sources of a web page, while the second one is copied from the data scraped by pyquery. They come from a same Instagram post, same path, but they are totally different. The first one works well, but the second one doesn't. How can I solve this? How can I get a right video url?

I am searching for a long time on net. But no use. Please help or try to give some ideas how to achieve this.

Here is my code related to the question

def getUrls(url):
    URL = str(url)
    html = get_html(URL)
    doc = pq(html)
    urls = []
    items = doc('script[type="text/javascript"]').items()
    for item in items:
        if item.text().strip().startswith('window._sharedData'):
            js_data = json.loads(item.text()[21:-1], encoding='utf-8')
            shortcode_media = js_data["entry_data"]["PostPage"][0]["graphql"]["shortcode_media"]
            edges = shortcode_media['edge_sidecar_to_children']['edges']

            for edge in edges:
                is_video = edge['node']['is_video']
                if is_video:
                    video_url = edge['node']['video_url']
                    video_url.replace(r'\u0026', "&")
                    urls.append(video_url)
                else:
                    display_url = edge['node']['display_resources'][-1]['src']
                    display_url.replace(r'\u0026', "&")
                    urls.append(display_url)


    return urls

Thanks in advance.

scrapy web-scraping python-3.x
2 Answers

I've seen this sometimes when using this Python module instead of HTML-scraping. At least with that module, edge["node"]["videos"]["standard_resolution"]["url"] usually (but not always) gives a non-corrupted value.

user_3404040

There's nothing wrong with your code. This is a known intermittent issue with Instagram, and other people have encountered it too: https://github.com/arc298/instagram-scraper/issues/545

There doesn't appear to be a known fix yet.

Also, while unrelated to your question, it's worth mentioning that you don't need to inspect the display_resources object to get the URL of the image:

display_url = edge['node']['display_resources'][-1]['src']

There is already a display_url property available (I'm guessing you saw this, based on the variable name?). So you can simply do:

display_url = edge['node']['display_url']
user_15765480
Related