Multiple iterations of requests saved for future processing

Viewed 22

My problem, I have a multistep process that requests data from API and then run another API over each response from first API. Problem is with correct saving and retrieving this information between APIs

My Flow:

  1. Request data from first API (100 rows 6 columns)
  2. Save to df
  3. Run each row of the df through second API
  4. Save to another df
  5. Data processing and cleaning

My issue is with step no.4

#Request data from second API 
try: 
        response_users = requests.post('https://deep-index.moralis.io/api/v2/0xFFE811714ab35360b67eE195acE7C10D93f89D8C/function', headers=headers, params=params, json=json_data)
            response_data = response_users.json()
            response_data["walletaddress"] = address
            response_data["block_number"] = updated_df.loc[updated_df['buddy'] == address, 'block_number'].iloc[0]
            response_data["updatedAt"] = updated_df.loc[updated_df['buddy'] == address, 'updatedAt'].iloc[0]
            dct = {k:[v] for k,v in response_data.items()}
            df2 = pd.DataFrame(dct)
            return(df2)
    
#get data for each row 
with tqdm(total=len(updated_df)) as pbar:
              for i in range(len(updated_df)) :
                team_info(updated_df.loc[i, 'buddy_str'])
                time.sleep(0.1)
                pbar.update(1)

Right now I can only get first iteration and should be 100 as this the number of rows After First API and in first df.

First DF enter image description here

0 Answers
Related