Beautifulsoup/JSON: How do I export all data into a dictionary to JSON?

user_16707466 0
Viewed 42

Currently have this code and it's storing every td into a JSON file but I want it filtered into 4 tags:

    find_table = soup.find("table", id="viewAllSeriesTable")
    rows = find_table.select("td")
    with open("data_file.json", "w") as write_file:
        data = [j.get_text(strip=True) for j in rows]
        json.dump(data, write_file)

HTML Code:

<table width="735" cellpadding="0" cellspacing="2" border="0"
   id="viewAllSeriesTable">
<thead>
   <tr>
      <th id="id">Series ID</th>
      <th
         id="seriesName">Series Name</th>
      <th id="clientName">Client Name</th>
      <th id="Brand">Brand</th>
   </tr>
</thead>
<tbody>
   <tr>
      <td id="9127"
         style="word-break: break-word;"><a
         href="seriesDefinition.html?id=9127">9127</a></td>
      <td
         style="word-break: break-word;">a</td>
      <td style="word-break:
         break-word;">A</td>
      <td style="word-break: break-word;">B</td>
   </tr>

How would I be able to do this?

{
   "id": 9127,
   "seriesName": "a",
   "clientName": "A",
   "Brand": "B"
 },
beautifulsoup json
2 Answers

Try this:

list = []
dic = ['Id: ', 'seriesName: ', 'clientName: ', 'clientName: ']
find_table = soup.find("table", id="viewAllSeriesTable")
rows = find_table.find("tbody")
tds = rows.find_all('td')
for i, j in enumerate(tds):
    dic[i] = dic[i] + j.get_text(strip=True)
    list.append(dic[i])
with open("data_file.json", "w") as write_file:
    json.dump(list, write_file)
user_166987180

You can try this out.

This code stores all <td> data of every <tr> in JSON format.

from bs4 import BeautifulSoup

s = '''<table width="735" cellpadding="0" cellspacing="2" border="0"
   id="viewAllSeriesTable">
<thead>
   <tr>
      <th id="id">Series ID</th>
      <th
         id="seriesName">Series Name</th>
      <th id="clientName">Client Name</th>
      <th id="Brand">Brand</th>
   </tr>
</thead>
<tbody>
   <tr>
      <td id="9127"
         style="word-break: break-word;"><a
         href="seriesDefinition.html?id=9127">9127</a></td>
      <td
         style="word-break: break-word;">a</td>
      <td style="word-break:
         break-word;">A</td>
      <td style="word-break: break-word;">B</td>
   </tr>
   <tr>
      <td id="9157"
         style="word-break: break-word;"><a
         href="seriesDefinition.html?id=9127">9157</a></td>
      <td
         style="word-break: break-word;">a-2</td>
      <td style="word-break:
         break-word;">A-2</td>
      <td style="word-break: break-word;">B-2</td>
   </tr>
   </tbody>
'''
import json
d = []
soup = BeautifulSoup(s, 'lxml')
trs = soup.find('tbody').find_all('tr')

for tr in trs:
    temp = {}
    tds = tr.find_all('td')
    temp['id'] = int(tds[0].text)
    temp['seriesName'] = tds[1].text
    temp['clientName'] = tds[2].text
    temp['Brand'] = tds[3].text

    d.append(temp)

json_str = json.loads(json.dumps(d))
print(json_str)

json_str looks like this.

[{'id': 9127, 'seriesName': 'a', 'clientName': 'A', 'Brand': 'B'}, {'id': 9157, 'seriesName': 'a-2', 'clientName': 'A-2', 'Brand': 'B-2'}]
user_27732060
Related