How can I get selenium to interact with fivethirtyeight search bars?

user_14858010 0
Viewed 37

I am working with this website (https://projects.fivethirtyeight.com/2020-nba-player-projections/) and trying to automate a search of players. I have a list of players but cannot seem to connect to the correct element. I have tried xpath, id, and css_selector. Below is an example code:

driver.get('https://projects.fivethirtyeight.com/2020-nba-player-projections/')
search_bar = driver.find_element_by_id('player-selectbox')
search_bar.click() 
search_bar.send_keys('lebron james')
search_bar.send_keys(Keys.RETURN)

Whichever way I try, I get the following error:

Message: element not interactable

Below is a link to a screenshot of the box I'm trying to access.

screenshot

Thanks!

selenium search python
3 Answers
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time


driver.get("https://projects.fivethirtyeight.com/2020-nba-player- projections/")
    
driver.find_element_by_xpath("//input[@placeholder='player name']").click()
    
driver.find_element_by_xpath("//input[@placeholder='player name']").send_keys('Kyrie Irving')

driver.find_element_by_xpath("//input[@placeholder='player name']").send_keys(Keys.ARROW_RIGHT)
time.sleep(10)
driver.close()

This code works perfectly, the complexity is that it is an autofill type search bar, I found an easy workaround using Keys.ARROW_RIGHT to take care of it

Attaching screenshot below,

pic contains proof of working automation

user_151319740

This should work

search_bar = driver.find_element_by_xpath("//input[@placeholder='player name']")
search_bar.click()
search_bar.send_keys('lebron james')
time.sleep(2)
driver.find_element_by_xpath("//*[@class='tt-suggestion tt-selectable']").click()

Feel free to replace time.sleep with explicit wait

user_177982390

Why not just get the data directly from the json? Would be far more efficient than Selenium.

import requests
import re
import json
import pandas as pd
from fuzzywuzzy import process

# Get Player Slugs from javascript
response = requests.get('https://projects.fivethirtyeight.com/2020-nba-player-projections/js/bundle.js')

js = response.text
p = re.compile('{t\.exports=(\[.*}]},{}\])')
result = p.search(js).group(1).rsplit('},{}',1)[0]

result = result.replace('team_abbr', '"team_abbr"')
result = result.replace('name', '"name"')
result = result.replace('full_team', '"full_team"')
result = result.replace('short_team', '"short_team"')
result = result.replace('slug', '"slug"')
result = result.replace(',randable:!1', '')
result = result.replace(',randable:!0', '')

playerSearch = json.loads(result)
df = pd.DataFrame(playerSearch)

players = list(df['name'])
slugs = list(df['slug'])

playerDict = dict(zip(players, slugs))



# Get the Player Data
# You could also make this as a user input

list_of_players = ['lebron james', 'Derik RoSe', 'LEamarkcus Allridge']

data = {}
for player in list_of_players:
    playerMatch = process.extractOne(player, list(playerDict.keys()))[0]
    player_slug = playerDict[playerMatch]

    url = f'https://projects.fivethirtyeight.com/2020-nba-player-projections/{player_slug}.json'
    jsonData = requests.get(url).json()
    
    data[playerMatch] = jsonData
    print(f'Returned: {playerMatch}')

Output:

enter image description here

user_77522960
Related