Python script for downloading images and videos from social media websites using selenium?

If it is a url directly to the source. So something like (theoretical url) https://cdn.facebook.com/path/to/wanted/file.png and no login required to open this link (depending on profile / page / group privacy policy settings), then common cURL, wget, requests (python module), and other tools, whether system commands and programs or libraries, should suffice.

If login is required and there is no possibility to have header data (Authorization, X-some-auth-header or cookie) and it is necessary to iterate with the login and retention session, then Selenium is the solution. However, if I have the necessary header data, then the easier paths mentioned above can be used.

Hypothetically (I never downloaded files with Selenium): In case of selenium, it is necessary to open the source of images / videos directly and then copy the content of the page.

browser.get("https://cdn.facebook.com/path/to/wanted/file.png")
data = browser.page_source

with open("/path/to/save/file.png", "wb") as f:
    f.write(data)
    pass

Update: I looked at the Internet, I found this answer. It should work. Thus, the top snippet works when the picture / video is displayed, but the download does not start.

For videos, it can be quite tricky over some streaming HTTP and the like. If it becomes more complicated, the solution would be to use or inspire with youtube-dl to support countless video sources for download. Including login solutions.

Update 2: Based on comment: for example I had a link like this facebook.com/hkrugby Now I need to download videos and images sepeartely from this url using python script how?

3 options:

1) Find all the elements (img, video) using the Selenia interface and collect the URLs they link to.

2) Take the page source - HTML - and either find a regular expression or other way of searching to find the right elements (img, video) and extract URLs from them.

3) Take the page source, but parse it (for example, Beatiful Soup), then get links from the elements again.

He sets the problem for things like "infinite scroll" and "lazy loading", where the page is done continuously by scrolling down. Resources are not loaded until you are zoomed in / on the page. Depending on the implementation of the technology, often there is an element such as data-url="htttps://...', which is copied to the URL when the script is started and the resource is loaded. Or, the original element is temporarily replaced by another to prevent the resource from loading unnecessarily.

Scrolling can be done via JS in the browser, but you never know how far you have to scroll and if we are talking about a page or group, so we can scroll and find the url that we have already downloaded. Then it would be nice, for example, to keep an url index that has already been downloaded.

Some suggestions on how to do this:

1) Open the page - browser.get("...")

2) Save the current page_source to a temporary variable

3) Scroll down and wait X seconds (depending on the speed of the machine and the Internet, some time when we are sure that the content is already full)

4) Verify the current page_source against the variable from step 2, if different then repeat the steps from step 2 again, repeat until page_source matches the variable from step 2

5) Find elements on the page (by CSS selector, XPATH, etc)

6) Extract any url or other metadata needed for download

7) Then download the extracted url

Optimization Enhancements:

We store index of downloaded URLs, load it when restarted and interleave page scrolling by searching elements and URL parsing. If we find the URL we have already downloaded, there is no need to scroll anymore.

Update 3: Demo