Selenium Get An Element Quickly

Viewed 113

Currently I'm making a python bot for whatsapp manually without APIs or that sort because I am clueless. As such, I'm using Selenium to take in messages and auto reply. Currently, I'm noticing that every few messages, one message doesn't get picked up because the loops ran are too slow and my computer is already pretty fast. Here's the code:

def incoming_msges():
    msges = driver.find_elements_by_class_name("message-in")
    msgq = []
    tq = []
    try:
        for msg in msges:
            txt_msg = msg.find_elements_by_class_name("copyable-text")
            time = msg.find_elements_by_class_name("_18lLQ")
            for t in time:
                tq.append(t.text.lower())
            for txt in txt_msg:
                msgq.append(txt.text.lower())
        msgq = msgq[-1]
        tq = tq[-1]
        if len(msgq) > 0:
            return (msgq, tq)
    except StaleElementReferenceException:
        pass

    return False

Previously, I didn't add the time check thing, and the message sent would be saved, with this loop continuously running such that even if the other party sent the same thing again, the code would not recognise it as a new message because it thinks it's the same one as before. So now, the problem is that my code is super time consuming and I have no idea how to speed it up. I tried doing this:

def incoming_msges():
    msges = browser.find_elements_by_class_name("message-in")
    
    try:
        msg = msges[-1]
        txt_msg = msg.find_element_by_xpath("/span[@class=\"copyable-text\"]").text.lower()
        time = msg.find_element_by_xpath("/span[@class=\"_18lLQ\"]").text.lower()
        return (txt_msg, time)
    except Exception:
        pass

    return False

However, like this, the code just doesn't find any messages. I have gotten the elements' types and classes correct according to the whatsapp web website but it just doesn't run. What's the correct way of rewriting my first code block as it is still correct? Thanks in advance.

1 Answers

First thing first ...

I definitely recommend using API ... Because what you are trying to do here is to reinvent the wheel. API has the power of telling you if there is a change in your status and you can queue these changes ... So I definitely recommend to use API ... It might be hard at the beginning, but trust me, its worth it ...

Next I would recommend you to use normal variable names. msges msgq tq (these are kindof unreadable and I still dont get what they are supposed to be after reading the code twice ...)

But to your speed problem ... "try - catch (aka except)" blocks are really heavy on a performance ... I would recommend to use safe programming if possible (20 if statements might be faster, but might not a same time) ... Also I think you are kind of unaware of a python language (atleast from what i can see here)

msgq = msgq[-1]   # you are telling it to take the last element and change array variable to string .. to be more specific...
msgq ([1,2,3,4]) = msgq[-1] (4) will result to -> msgq = 4 (which in my option hits you performance as well)
tq = tq[-1]    # same here

This would be better :)

if len(msgq[-1]) > 0:
   return (msgq[-1], tq[-1])

If I understand your code correctly, you are trying to scrape the messages, but if its like you are saying that you want to make auto-reply bot, I would recommend you to eighter get ready for some JS magic or switch tool. I personally noticed that the selenium has a problem with dynamic content ... to be more specific ... once its at the end of the file it does not scrape it again ... so if you do not want to auto refresh every 5-10 seconds to get the latest HTML file, I recommend eighter to create this bot in JS (that will trigger everytime that an element changes) or use the API and use selenium just for responses. I was told that Selenium was created to simulate the common user to check if user interface works as it should (if buttons exists, if the website contains all what it should etc.) ... I think that selenium is for this job something like a flower small sponge for a car clean ... you can do it ... buts gonna cost you alot of time and you might miss some spots (like you missed those messages) ...

Lastly ... the work with strings in general is really costly. you are doing O(n^2) of operations in a try block ... which i can imagine can be really costly ... if its possible, I would reduce the number of inner for loops.

I wish you good luck in this project and I hope you find the answer you seek, while I hope my answer was at least a little helpful.

Related