How can I quickly get the follower count for a large list of Instagram users?

Viewed 2027

I have the following program in python that reads in a list of 1390680 URLS of Instagram accounts and gets the follower count for each user. It utilizes the instaloader. Here's the code:

import pandas as pd
from instaloader import Instaloader, Profile
# 1. Loading in the data
# Reading the data from the csv
data = pd.read_csv('IG_Audience.csv')
# Getting the profile urls
urls = data['Profile URL']

def getFollowerCount(PROFILE):
    # using the instaloader module to get follower counts from this programmer
    # https://stackoverflow.com/questions/52225334/webscraping-instagram-follower-count-beautifulsoup
    try:
        L = Instaloader()
        profile = Profile.from_username(L.context, PROFILE)
        print(PROFILE, 'has', profile.followers, 'followers')
        return(profile.followers)
    except Exception as exception :
        print(exception, False)
        return(0)

# Follower count List
followerCounts = []
# This loop will fetch the follower count for each user
for url in urls:
    # Getting the profile username from the URL by removing the instagram.com
    # portion and the backslash at the end of the url
    url_dirty = url.replace('https://www.instagram.com/', '')
    url_clean = url_dirty[:-1]
    followerCounts.append(getFollowerCount(url_clean))

# Converting the list to a series, adding it to the dataframe, and writing it to
# a csv
data['Follower Count'] = pd.Series(followerCounts)
data.to_csv('IG_Audience.csv')

The main issue I have with this is that it is taking a very long time to read through the entire list. It took 14 hours just to get the follower counts for 3035 users. Is there any way to speed up this process?

1 Answers

First I wanna say I'm sorry for being VERY late but hopefully this can help someone in the future. I'm having a similar issue and I believe I found out why, when you get the followers you instaloader doesn't just go to the profiles page and read the number but it gets the URL and profile ID for each account and can only get so many at a time, the best way I can think to get around this would be to make a request to the page and just read the follower count on their main page issue with this however is after I believe 9999 followers it will start saying "10k" or "10.1k" so you'll be off by 100 and just gets worse if the person has over a million because then its off by even more.

Related