nested for loop, python, export to pandas or export to XML (web scraper), problem with my web scraper

Viewed 29

please, can someone modify this nested for loop PLEASE it can output the entire <li class_='rows'

[code]
#!/usr/bin/python3
import requests
from bs4 import BeautifulSoup

# craigslist page, data for Python3 http://sandiego.craigslist.org/search/sof

response = requests.get("http://sandiego.craigslist.org/search/sof")
webpage = response.content
soup = BeautifulSoup(webpage, "html.parser")

# info: http://www.compjour.org/warmups/govt-text-releases/intro-to-bs4-lxml-parsing-wh-press-briefings/
# nested loop https://pynative.com/python-nested-loops/
def start_program():
    for counter in soup.find_all('form', class_='search-form'):
        for n, tag in enumerate(counter.find_all('ul', class_='rows')):
            description = [x for x in tag.find_all('li')]
        print("Description: ", description[n].text.strip())

start_program()
[/code]

instead of just the first, for example ++counter. Also, would this method be good start for exporting it to a xml or pandas dataframe (a nested loop) or another method. Thanks.

0 Answers
Related