Having Trouble extracting the Value I want with Selenium Web Driver

user_14197885 0
Viewed 59

I would like to extract part number values from a site:

https://www.mcmaster.com/rivet-nuts/twist-resistant-rivet-nuts-6/material~stainless-steel/

I have identified "data-mcm-partnbr" as the variable I need to locate the values I need. It is located in the "PartNbrLnk" class. There are 12 part numbers on this site. This is what I've been trying so far and I've been getting an empty set returned. What am I doing wrong?

driver.get(url)
for x in  part_number_list:
    part_number = driver.find_elements_by_class_name("PartNbrLnk")
    part_number_list.append(part_number)
print(part_number_list)

Any help is welcome. Thanks

selenium-chromedriver web-scraping selenium html python
1 Answers

To get value of an attribute, you use .get_attribute("name of attribute")

In your case you'll want to try:

myElement.get_attribute("data-mcm-partnbr")

You also seem to have a slight hiccup in your logic.

You need to get all the web elements first, then loop through to collect your value.

You might want to try:

driver.get(url)
#get all the parts
all_Parts = driver.find_elements_by_class_name("PartNbrLnk")

#create an array for storage
part_number_list = []

#loop through all your identified parts
for part in all_Parts:
    #extract the part number
    part_number = part.get_attribute("data-mcm-partnbr")
    #print it for good measure
    print(part_number)
    #Append it like you were doing before
    part_number_list.append(part_number)

End result is part_number_list is an array to do with as you will.

user_138412480
Related