I am trying to extract all the text from various chats on an application. I have the code working but noticed when the chats are too long, the chat doesn't load in all the li tags at once within the ul, instead you have to scroll to have previous messages exposed in the li tags.
This is where my problem arises, I cannot simply keep scrolling to the top of the chat as the bottom li tags will disappear. Therefore, what I am trying to do is store all the li tags in a list, then scroll to the last li tag the ul has (so that it loads more) and then to .extend the already existing list of li tags with the rest.
My code is as follows:
chat_frame = driver.find_element_by_xpath("//iframe[@class='embedded-electron-webview embedded-page-content']")
driver.switch_to.frame(chat_frame)
up_top = False
wait_click.until(EC.element_to_be_clickable((By.XPATH, '//ul[@aria-label="Chat content"]')))
the_ul =driver.find_element(By.XPATH,'//ul[@aria-label="Chat content"]')
lis =the_ul.find_elements(By.TAG_NAME,'li')
print(f"Archiving Chat with {name} ...\n")
while up_top == False:
driver.execute_script("arguments[0].scrollIntoView();", lis[0])
print("Scrolled to top")
the_ul =driver.find_element(By.XPATH,'//ul[@aria-label="Chat content"]')
new_lis =the_ul.find_elements(By.TAG_NAME,'li')
if new_lis[0].text == lis[0].text:
up_top = True
lis.extend(new_lis)
The 2 problems I can't seem to find a solution to:
- How do I know when I'm at the top of the ul (the actual top, where the first every message was sent).
- How do I extract every li tag along the way do that I don't get duplicates in the list (lis)
As you can see currently I'm trying to compare the last elements text in both lists to see if they are the same and then assuming that's the top of the chat (this is a flawed assumption but I can't think of a better way)
As with not having duplicates, I'm not sure if selenium webelements can be compared?
Any help would be appreciated!