How can I get all html from website with python

Viewed 27

I currently have a project to extract data from a website. 'https://www.geni.com/list?focus_id=6000000187119548853 I am working with the owner and they welcome the effort so no privacy issues

Code

from bs4 import BeautifulSoup
# 
# url='https://www.geni.com/list?focus_id=6000000050609245281'
# response = requests.get(url)
# soup = BeautifulSoup(response.text, "lxml")
# print(soup)

When I run I get fewer lines than are in the page source for the site.

in fact, I am missing all the rows that have the data I am after.

When I inspect the Elements Section of the Inspect Option, all the data is there.

I have searched and not not see solution.

Thank you

0 Answers
Related