How to format texts so they look just like shown in the website?

user_11726134 0
Viewed 55

I am web scraping a website with Python selenium. This is the code on the website:

enter image description here

and I'd like texts to look like shown on the website, i.e. easy to 'read' with line breaks, in an organized format.

I tried using

driver.find_element_by_class_name('record-content.record-information.record-content_j').text

but there are \n \n characters inside. I tried print(text), and looks better. But is there a way to store the text in a dataframe or something so it is saved in an organized format. The website looks like this: enter image description here When I tried

rawData=driver.find_element_by_class_name('record-content.record-information.record-content_j').text
sanitizedData = rawData.replace('\n','')
print(BeautifulSoup(sanitizedData, 'html.parser').prettify())

The output looks like this:

enter image description here The 'br' tags for line breaks just disappeared.

selenium python
1 Answers

As we've extracted via .text content with no <br> tags included, we can use BeautifulSoup to actually prittify our output. Similarly, we could use element.get_attribute('innerHTML') if you'd like to maintain html, and strip any new line characters if needed. Hope this helps :)

from selenium import webdriver
from bs4 import BeautifulSoup


def main():
    """ Start Firefox instance """
    driver = webdriver.Firefox()
    driver.get("http://yoursite.com")

    """ Extract content """
    rawData = driver.find_element_by_class_name('record-content.record-information.record-content_j').text

    """ Print to console """
    print(BeautifulSoup(rawData, 'html.parser').prettify())


if __name__ == "__main__":
    main()
user_61687920
Related