Python: how to extract "data-bind" html elements?

Question

I am trying to extract data from a website. The element is hidden. when i try to "view source", the header text is not displayed.

But when i try to inspect, there is text visible.

STM1F-1S-HC

The code used is:

def getlink(link):
    try:
        f = urllib.request.urlopen(link)
        soup0 = BeautifulSoup(f)
    except Exception as e:
        print (e)
        soup0 = 'abc'
    for row2 in soup0.findAll("h4",{"data-bind":"text: Name"}):
        Name = row2.text
        print(Name)

#code to find all links to the products for further processing.
i=1
global i
for row in r1.findAll('a', { "class" : "col-xs-12 col-sm-6" }):
    link = 'https://www.truemfg.com/USA-Foodservice/'+row['href']
    print(link)
    getlink(link)
print(productcount)

The output is:

https://www.truemfg.com/USA-Foodservice/Products/Traditional-Reach-Ins
C:\Users\Santosh\Anaconda3\lib\site-packages\bs4\__init__.py:181: UserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system ("lxml"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.

The code that caused this warning is on line 193 of the file C:\Users\Santosh\Anaconda3\lib\runpy.py. To get rid of this warning, change code that looks like this:

 BeautifulSoup([your markup])

to this:

 BeautifulSoup([your markup], "lxml")

  markup_type=markup_type))

https://www.truemfg.com/USA-Foodservice/Products/Specification-Series

https://www.truemfg.com/USA-Foodservice/Products/Food-Prep-Tables

https://www.truemfg.com/USA-Foodservice/Products/Undercounters

https://www.truemfg.com/USA-Foodservice/Products/Worktops

https://www.truemfg.com/USA-Foodservice/Products/Chef-Bases

https://www.truemfg.com/USA-Foodservice/Products/Milk-Coolers

https://www.truemfg.com/USA-Foodservice/Products/Glass-Door-Merchandisers

https://www.truemfg.com/USA-Foodservice/Products/Air-Curtains

https://www.truemfg.com/USA-Foodservice/Products/Display-Cases

https://www.truemfg.com/USA-Foodservice/Products/Underbar-Refrigeration

we find that there are no names printed.

Could someone let me know a solution to print the names as well.

Thanks, Santosh

Python: how to extract "data-bind" html elements?

1 Answers