Scraping wikipedia table by Selenium getting a SyntaxError

user_15115239 0
Viewed 162

I'm trying to scrape table info from wikipedia, but I'm getting a SyntaxError.

Please tell me how to fix this problem.

Thank you.

ignore_index=True

SyntaxError: invalid syntax

Code as follows.

from selenium import webdriver
import pandas as pd

url = 'https://en.wikipedia.org/wiki/List_of_future_tallest_buildings'
driver = webdriver.Chrome('C:\Program Files\Python39\chromedriver.exe')
driver.get(url)
driver.maximize_window()

df_building = pd.DataFrame(columns=['Building', 'City', 'Country', 'Height in M', 'Height in Ft', 'Floors'])

building = driver.find_elements_by_xpath('//table[@class="wikitable sortable jquery-tablesorter"][1]/tbody/tr/td[2]')
city = driver.find_elements_by_xpath('//table[@class="wikitable sortable jquery-tablesorter"][1]/tbody/tr/td[3]')
country = driver.find_elements_by_xpath('//table[@class="wikitable sortable jquery-tablesorter"][1]/tbody/tr/td[4]')
height_meters = driver.find_elements_by_xpath('//table[@class="wikitable sortable jquery-tablesorter"][1]/tbody/tr/td[5]')
height_ft = driver.find_elements_by_xpath('//table[@class="wikitable sortable jquery-tablesorter"][1]/tbody/tr/td[6]')
floors = driver.find_elements_by_xpath('//table[@class="wikitable sortable jquery-tablesorter"][1]/tbody/tr/td[7]')

for i in range(len(building)):
    df_building = df_building.append({
        'Building': building[i].text,
        'City': city[i].text,
        'Country': country[i].text,
        'Height in M': height_meters[i].text,
        'Height in Ft': height_ft[i].text,
        'Floors': floors[i].text,
        ignore_index=True
    })

df_building.to_excel('buildings_scraped.xlsx', index=False)
selenium python
1 Answers

Take the ignore_index=True out of {} containing the data.
Instead of

df_building = df_building.append({
        'Building': building[i].text,
        'City': city[i].text,
        'Country': country[i].text,
        'Height in M': height_meters[i].text,
        'Height in Ft': height_ft[i].text,
        'Floors': floors[i].text,
        ignore_index=True
    })

it should be

df_building = df_building.append({
        'Building': building[i].text,
        'City': city[i].text,
        'Country': country[i].text,
        'Height in M': height_meters[i].text,
        'Height in Ft': height_ft[i].text,
        'Floors': floors[i].text        
    },ignore_index=True)
user_34854340
Related