Mars 2020 Web Scraping

user_13054451 0
Viewed 119

I'm trying to scrape data from 'Mars Weather Report from Perseverance' (https://mars.nasa.gov/mars2020/weather/). But when I try to scrape, e.g, the Mars Date, it returns a none item in a list. Like this:

import requests
from bs4 import BeautifulSoup as soup

html = requests.get('https://mars.nasa.gov/mars2020/weather/')
html.status_code

marsDat= []

for date in bsobj.findAll('span', {'class':'marsDate'}):
    marsDat.append(date.text.strip())
    
print(marsDat)
#returns: ['']

Can someone give me a hint? It is happening because we can't do web scraping on this site?

web-scraping html python
1 Answers

The data you see is loaded from external URL. To print current Mars date, you can do:

import json
import requests

url = "https://mars.nasa.gov/rss/api/?feed=weather&category=mars2020&feedtype=json"
data = requests.get(url).json()

# uncomment to print all data:
# print(json.dumps(data, indent=4))

max_sol = max(data["sols"], key=lambda k: int(k["sol"]))
print(max_sol["sol"])

Prints:

62
user_100359850
Related