Scraping tables with beautiful soup and python 3.x

Viewed 2110

So I am very new to python and am still trying to get my head around how everything works, right now I am working on scraping tables for data using beautiful soup. I can navigate to the specific table I want using beautiful soup but pulling the actual data out has stumped me and everything I try has failed.

this is my current code:

sauce = requests.get('https://www.investsmart.com.au/managed-funds/fund/cromwell-phoenix-opportunities-fund/40665')
soup = BeautifulSoup(sauce.text, 'html.parser')
tables = soup.findChildren('table')
my_table = tables[1]
rows = my_table.findChildren(['tr'])

for tds in rows[1]:
    print(tds)

this leave me with the output

 <td class="text-left">Total return</td>


<td>-2.79</td>


<td>-2.61</td>


<td>11.22</td>


<td>24.6</td>


<td>19.18</td>


<td>18.65</td>


<td>21.44</td>


<td>-</td>

All i want is the actual numbers inside the td tag, which eventually i would like to sort into their respective months and output it into an excel file.

however i'm not really sure how to grab just the returns without the tags, when i try:

for tds in rows[1]:
    print(tds.text)

i get this error: AttributeError: 'NavigableString' object has no attribute 'text'

so how can I go about grabbing this data in a way that I can sort into the months it comes from and output to excel because I am not sure what to do next.

3 Answers
Related