How to read the next page on API using python?

Viewed 9103

I need help on how to do a loop so each time I make a GET request, it will always be the new page from the API.

I start with getting the first response. It includes a parameter to the next page next_key

{
  "result": [
    {
      ...,
      ...
    }
  ],
  "next_key": 123
 }

Below is my current attempt

import requests
import json

url = "https://flespi.io/gw/channels/all/messages"
headers = {"Authorization": "FlespiToken 23ggh45"}

def getFirst():
    data = {"limit_count":100, "limit_size":10000}
    params = {"data":json.dumps(data, separators=(",", ":"))}

    reqFirst = requests.get(url, params=params, headers=headers).json()
    return reqFirst["next_key"] ## this returns "123"

def getDataNext():
    data = {"limit_count":100, "limit_size":10000, "curr_key":getFirst()}
    params = {"data":json.dumps(data, separators=(",", ":"))}

    reqNext = requests.get(url, params=params, headers=headers)

    jsonData = reqNext.json()

    while True:
        if "next_key" in jsonData:
            data = {"limit_count":100, "limit_size":10000,"curr_key":jsonData["next_key"]}
            params = {"data":json.dumps(data, separators=(",", ":"))}

            req = requests.get(url, params=params, headers=headers).json()  ## this should do GET request for the third page and so on...

            print req["next_key"] # this returns "3321" which is the value for "next_key" in second page

        else:
            pass

getDataNext()

The full url including limit count, limit size and curr key is as follows https://flespi.io/gw/channels/all/messages?data=%7B%22curr_key%22%123%2C%22limit_count%22%3A100%2C%22limit_size%22%3A10000%7D

As you can see this only returns the second page that is jsonData["next_key"]. What I want to do is that for each GET request, the program will read the next_key and put it on the next GET request.

I am thinking to use increment on the curr_key but the key is random and also I do not know how many page there is.

I believe there must be just a simple solution for this but apparently I could not think about it. Thank you for your help and suggestion.

1 Answers
Related