I have a little scraping script that worked fine, all ok. The aim is to scrape the results for each sporting round and create a row like this: Round 6, 12/11, 20.45, Liverpool, Chelsea, 1, 0. Scrape the results of the last round and also the results of all the previous rounds. Then 10 games are scraped for each round (rounds are 38 in total) which were saved in the Score_of_Scraping table. All ok, this work good (code scraping and database attached below)
The problem arose when I decided I wanted to enter the id number for each round as well. However, this id is copied from another table (called All_Round).So for every single scraped staff, I would like to copy the corresponding round id.
I already own the All_Round table. The All_Round table contains the TOTAL number of rounds, therefore: previous rounds, current rounds and next / future rounds. In the All_Round table there are (empty) days 1 to 38. Each round of this table must be filled and matched with the data of the corresponding rounds of the Score_of_Scraping table. This is one example. It has different data, but it is intuitive. It's the same

WHAT DO I WANT TO GET? For example I would like to obtain:
- Round 6, 12/11, 20.45, Liverpool, Chelsea, 1, 0, 435
- Round 6, 12/11, 8.45pm, Manchester City, Aston Villa, 3, 1, 435
- Round 6, 12/11, 8.45pm, West Ham, Everton, 2, 2, 435
435 will correspond to round 6, which would be ID_Round in the All_Round table
MY SOLUTION How can I achieve this in your opinion? I tried this by adding:
- current_round_ID = 0 under current_round = '?'
- into if: cursor.execute ("SELECT All_Round (Number_Round)", (..., ...)) and current_round_ID = cursor.lastrowid
- into Else: cursor.execute('INSERT INTO Scraping_Score (ID_Round)', (..., current_round_ID))
So I did as current_round- when getting 'event__round' then SELECT in All_Round table and keep its ID in variables (e.g. current_round_IDE when getting scored data then use) current_round_ID to insert data into Scraping_Score table.
PROBLEM I believe the problems are with the arguments in the two cursor.execute where I have left the dots, ie (..., ...)) and then (..., current_round_ID)). What should I write there to solve the problem? In case I am wrong, if the problem are not (..., ...)) and then (..., current_round_ID)), how do you think you can solve the problem? Thanks
Database code
Table All_Round
CREATE TABLE "All_Round" (
"ID_Round" INTEGER,
"Number_Round" INTEGER,
"ID_Tournment" INTEGER,
PRIMARY KEY("ID_Round" AUTOINCREMENT)
);
#Table Score_of_Scraping
CREATE TABLE "Scraping_Score" (
"Id" INTEGER,
"current_round" INTEGER,
"date" INTEGER,
"time" INTEGER,
"team_home" INTEGER,
"team_away" INTEGER,
"score_home" INTEGER,
"score_away" INTEGER,
#"new_field_id_round"
PRIMARY KEY("Id" AUTOINCREMENT)
);
Scraping code
driver.get("link")
driver.minimize_window()
driver.implicitly_wait(12)
wait = WebDriverWait(driver, 12)
all_rows = driver.find_elements(By.CSS_SELECTOR, "div[class^='event__round'],div[class^='event__match']")
current_round = '?'
#current_round_ID = 0 # <-- to keep ID of last inserted `round`
for PremierLeague in all_rows:
classes = PremierLeague.get_attribute('class')
if 'event__round' in classes:
current_round = PremierLeague.text.split(" ")[-1] # only `20` without `Round`
#cursor.execute("SELECT All_Round (Number_Round)", (..., ...))
#current_round_ID = cursor.lastrowid
else:
datetime = PremierLeague.find_element(By.CSS_SELECTOR, "[class^='event__time']")
#Divide la data e l'ora
date, time = datetime.text.split(" ")
date = date.rstrip('.') # right-strip to remove `.` at the end of date
team_home = PremierLeague.find_element(By.CSS_SELECTOR, "[class^='event__participant event__participant--home']")
team_away = PremierLeague.find_element(By.CSS_SELECTOR, "[class^='event__participant event__participant--away']")
score_home = PremierLeague.find_element(By.CSS_SELECTOR, "[class^='event__score event__score--home']")
score_away = PremierLeague.find_element(By.CSS_SELECTOR, "[class^='event__score event__score--away']")
PremierLeague = [current_round, date, time, team_home.text, team_away.text, score_home.text, score_away.text]
results_PremierLeague.append(PremierLeague)
#cursor.execute('INSERT INTO Scraping_Score (ID_Round)', (..., current_round_ID))
print(PremierLeague)