URL request returning empty list

Viewed 35

I'm using the code below to access the data of an image on my company's website

import requests
import time

requ = requests.get('https://modularaspects.co.uk/wp-json/wp/v2/media?search=assetsfullsizephoto36647')
time.sleep(2)
print(requ.text)

However it just returns an empty list, whereas if you follow the link you can clearly see data within the 0th index. Any clues?

Please let me know if there's any further information i can provide.

3 Answers

Gave up with Requests and used Selenium instead. I retrieved the source html and then parsed it with the json library.

from selenium import webdriver
from selenium.webdriver.firefox.options import Options
import json

options = Options()
options.headless = True
options.add_argument("--window-size=1920,1200")

sel_driver = webdriver.Firefox(options=options)

website_url = 'view-source:https://modularaspects.co.uk/wp-json/wp/v2'
imagename = 'assetsfullsizephoto36647'



sel_driver.get(website_url + '/media?search=' + imagename)

output = sel_driver.page_source
output = sel_driver.find_element_by_tag_name('pre').text

parsedjson = json.loads(output)

print(parsedjson[0]['id'])

Hopefully this is helpful to someone.

Do you want to check again? I literally copy-pasted your code and it showed me the response data.

What's the response code you're getting? when you try requ or requ.status_code?

If it isn't 200 it could be an issue with your user-agent and the site is blocking access to the information

If you're able to access the data through your browser, then you can try using the same user-agent with the following:

url = 'https://modularaspects.co.uk/wp-json/wp/v2/media?search=assetsfullsizephoto36647'
user_agent = {your user agent here}
requ = requests.get(url, headers = user_agent)

A quick google search will give you your user agent

Related