I am trying to scrape some vessel attributes from the marinetraffic vessels database using Python and Selenium(sample URL). As an example, I'm trying a list of country names for the fist five vessels in the table.
The code I'm using is the following:
!pip install selenium
!apt-get update
!apt install chromium-chromedriver
!cp /usr/lib/chromium-browser/chromedriver /usr/bin
from selenium import webdriver
from selenium.webdriver.common.by import By
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
options.add_argument("lang=en")
options.add_argument("start-maximized")
options.add_argument("disable-infobars")
options.add_argument("--disable-extensions")
options.add_argument("--incognito")
options.add_argument("--disable-blink-features=AutomationControlled")
driver = webdriver.Chrome(options=options)
url ='https://www.marinetraffic.com/en/data/?asset_type=vessels&columns=flag,shipname,photo,recognized_next_port,reported_eta,reported_destination,current_port,imo,ship_type,show_on_live_map,time_of_latest_position,lat_of_latest_position,lon_of_latest_position,distance,notes&near_me|coordinates|near_me=37.745,23.535,50'
driver.get(url)
driver.implicitly_wait(10)
vessel_count = 0
vessel_left_container = driver.find_element(By.CLASS_NAME, 'ag-pinned-left-cols-container')
vessels_left = vessel_left_container.find_elements(By.CSS_SELECTOR, 'div[role="row"]')
while vessel_count <= 5:
for i in range(len(vessels_left)):
vessel_country_list = vessels_left[i].find_elements(By.CLASS_NAME, 'flag-icon')
if len(vessel_country_list) == 0:
vessel_country = 'Unknown'
else:
vessel_country = vessel_country_list[0].get_attribute('title')
vessel_count += 1
print(vessel_country)
I have verified that the class names and the "title" attribute of the div class="flag-icon" does exist, however I'm not able to fetch the country name. Can you help me understand what I'm doing wrong here?