get all entries of a cell in a javascript table in one single element of a list if there are identical classes with different information in one cell

user_15716700 0
Viewed 34

I have the problem that with the link below, I get 15 seperate elements of clubs the player played for instead of one element in the list, that indicates that the player changed clubs and played for two different clubs but during the same period.

url = 'https://www.transfermarkt.ch/emre-can/verletzungen/spieler/119296'
response = requests.get(url, headers={'User-Agent': 'Custom5'})
print(response.status_code)

injury_data = response.text
soup = BeautifulSoup(injury_data, 'html.parser')
table = soup.find(id="yw1")

to get the clubs the player played for I did the following:

teamlist = []
for item in table.select('img', class_='tiny_wappen'): 
            team_list.append(item.get('alt'))

this leads to the following result:

['Borussia Dortmund',
 'Borussia Dortmund',
 'Borussia Dortmund',
 'Juventus Turin',
 'Borussia Dortmund',
 'Juventus Turin',
 'Juventus Turin',
 'Juventus Turin',
 'FC Liverpool',
 'FC Liverpool',
 'FC Liverpool',
 'FC Liverpool',
 'FC Liverpool',
 'FC Bayern München',
 'FC Bayern München II']

But what I want in the end ist the following:

['Borussia Dortmund',
 'Borussia Dortmund',
 'Borussia Dortmund',
 'Juventus Turin / Borussia Dortmund',
 'Juventus Turin',
 'Juventus Turin',
 'Juventus Turin',
 'FC Liverpool',
 'FC Liverpool',
 'FC Liverpool',
 'FC Liverpool',
 'FC Liverpool',
 'FC Bayern München',
 'FC Bayern München II']

in the end, I want 14 elements in the team list instead of 15, indicating that during certain periods the player was changing the clus / was playing for two clubs.

I tried the following:

teams = ['/'.join([i.get("alt") for i in item.find_all_next('a') if i.get("alt")]) for item in table.select('img', class_='tiny_wappen')

but I received a list with empty entries and it didn't work.

web-scraping python
1 Answers

Given the structure of this table, you need more refined css selectors to get there:

teams = []
for entry in soup.select('table tr:is(.odd, .even) td[class="rechts hauptlink wappen_verletzung"]:has(img)'):
    images = entry.select('a img')
    row = [image.get('alt') for image in images]
    teams.extend(['/'.join(row)])
print(teams)

Output should be your desired output.

user_94480900
Related