Listing IBM Cloud Resources using ResourceControllerV2 and pagination issues

user_2402246 0
Viewed 74

I'm using the Python ibm-cloud-sdk in an attempt to iterate all resources in a particular IBM Cloud account. My trouble has been that pagination doesn't appear to "work for me". When I pass in the "next_url" I still get the same list coming back from the call.

Here is my test code. I successfully print many of my COS instances, but I only seem to be able to print the first page....maybe I've been looking at this too long and just missed something obvious...anyone have any clue why I can't retrieve the next page?

try:
    ####### authenticate and set the service url
    auth = IAMAuthenticator(RESOURCE_CONTROLLER_APIKEY)
    service = ResourceControllerV2(authenticator=auth)
    service.set_service_url(RESOURCE_CONTROLLER_URL)

    ####### Retrieve the resource instance listing
    r = service.list_resource_instances().get_result()
    ####### get the row count and resources list
    rows_count = r['rows_count']
    resources = r['resources']

    while rows_count > 0:
        print('Number of rows_count {}'.format(rows_count))

        next_url = r['next_url']

        for i, resource in enumerate(resources):
            type = resource['id'].split(':')[4]
            if type == 'cloud-object-storage':
                instance_name = resource['name']
                instance_id = resource['guid']
                crn = resource['crn']
                print('Found instance id : name - {} : {}'.format(instance_id, instance_name))

        ############### this is SUPPOSED to get the next page
        r = service.list_resource_instances(start=next_url).get_result()
        rows_count = r['rows_count']
        resources = r['resources']

except Exception as e:
    Error = 'Error : {}'.format(e)
    print(Error)
    exit(1)
ibm-cloud python
2 Answers

From looking at the API documentation for listing resource instances, the value of next_url includes the URL path and the start parameter including its token for start.

To retrieve the next page, you would only need to pass in the parameter start with the token as value. IMHO this is not ideal.

I typically do not use the SDK, but a simply Python request. Then, I can use the endpoint (base) URI + next_url as full URI.

If you stick with the SDK, use urllib.parse to extract the query parameter. Not tested, but something like:

from urllib.parse import urlparse,parse_qs
o=urlparse(next_url)
q=parse_qs(o.query)
r = service.list_resource_instances(start=q['start'][0]).get_result()
user_49237550

Could you use the Search API for listing the resources in your account rather than the resource controller? The search index is set up for exactly that operation, whereas paginating results from the resource controller seems much more brute force.

https://cloud.ibm.com/apidocs/search#search

user_194377740
Related