How to Scrape Json Text Format using Python

Question

As i wanted to scrape the data from this website but it gives me an error with no result

 import requests
    import time

headers={
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.79 Safari/537.36'
}

url='https://api.github.com/repositories?since=0'
res = requests.get(url,headers=headers)
data =res.json()
print(data["id"])

print(data["node_id"])

F.Hoque · Answer

Json data is in an array. So to grab id value you have to iterate over the array.

import requests
   
headers={
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.79 Safari/537.36'
}
url='https://api.github.com/repositories?since=0'
res = requests.get(url,headers=headers)
data =res.json()

for id in data:
    print(id['id'])

Output:

How to Scrape Json Text Format using Python

1 Answers