Trouble accessing Instagram post caption in Python Selenium

user_12172685 0
Viewed 125

Good afternoon: I'm currently attempting to access the description (caption) section of certain instagram posts using python selenium. For instance, in the picture below it refers to the part that starts with the word "Thanks" and that goes all the way down to the hashtags.

enter image description here

Well, it caught my attention the following things:

Consider this post (https://www.instagram.com/p/CNYQkxADO8p/). When the inspector is checked, we can see that the comment is inside a class call "Edited" so we can retrieve all the text with the following code:

desc = driver.find_element_by_xpath("//span[@title='Edited']").text

The code above works well for the mentioned post. However, consider this second post: (https://www.instagram.com/p/CN5dOopjNEk/). The description seems to be located in a different HTML class, and the line of code above throws a NoSuchElementException. I was therefore wondering how can I access the caption of that post, considering that it is not located in a section by the title "Edited". How can I access said part of the post?

Thanks for your attention

scrape instagram selenium python javascript
1 Answers

For that specific post you can try using:

desc = driver.find_element_by_xpath("//div[@class='C4VMK']/h2/following-sibling::span")

print(desc.text)

Since there is no direct way to access that element, we drill down from the class to h2 and then take the next element sibling of span, which is where the text resides.

I am not sure why it would be different from the other posts that have //span[@title='Edited'], but it seems to be.

user_44390190
Related