I'm trying to read a dynamic table, which is updated 1-3 times per second. I'm using Selenium, in Python 3.x, but if you have a solution for other languages I can work it out as well.
My question is: what is the best practice for reading frequently updated tables?
What I've tried:
driver.wait.untilalong withexpected_conditions- re-read the table with a call to
find_elementsif a stale exception is thrown
Neither of them is working, due to the high refreshing rate. I can successfully retrieve the table for a moment, but when I try to access its rows the moment after, I get a stale exception. It's worth to say that when I try the same code in the same table when there are less frequent updates everything works fine.
I'm not posting any code for the moment, as I'd be interested in knowing what more experienced people do in this case.
My naive thinking: Being non-expert (but keen to learn) in web scraping nor in any web-related languages, I'd say that if this was a problem with dynamic data, I'd take a pointer or a reference to the actual table (and then looping dynamically on the rows). Is that possible in this framework?