Beautiful soup- how can we get elements before the <head> element?

user_703139 0
Viewed 599

If I have a html as follows, and I use beautiful soup to parse it, how can I access the lines before <head> element. 

<?xml version="1.0" encoding="utf-8" standalone="no"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
  "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
<head>

For instance the standard way to access the head element is soup.head or the body is soup.body. I assume that's because head and body are both standard tags.

Is there a way to access elements before <head>?

beautifulsoup python
1 Answers

You can by selecting the head tag and looping over previous_elements:

from bs4 import BeautifulSoup
from w3lib.html import remove_tags

html= '<?xml version="1.0" encoding="utf-8" standalone="no"?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"  "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head>'

soup= BeautifulSoup(html,"html.parser")
x= soup.head

while x.previous_element != None:   

if not isinstance(x.previous_element, bs4.element.Tag):
    p = x.previous_element.PREFIX + str(x.previous_element) + 

x.previous_element.SUFFIX
        prev_head = prev_head + p
    else:
        prev_head = str(x.previous_element) + prev_head

    x = x.previous_element

prev_head = remove_tags(prev_head, which_ones= ("head",))

BeautifulSoup(prev_head)

After this process you will have all the code above <head> in prev_head as a string .You can then BeautifulSoup(prev_head) to get a BS object for posterior use.

PS: Notice that I deleted the <head> tag because <html> its the first previous_element. I've also formatted the non-tag elements because their flat str format doesn't include their prefix and suffix making them unavailable to use in a BS object.

user_73177890
Related