How to convert and change Json format

sunrise 0
Viewed 38

I am trying to do a webscraping, to get the url link and download it, I'm stuck at json formatting. I would like to change format_1 into format_2

format_1= {'result': [{'TITLE': 'title1<br>title2<br>title3',
                       'URL': 'url1<br>url2<br>url3'}]}


format_2 = {'result': [{'TITLE': 'title1', 'URL': 'url1'},
                       {'TITLE': 'title2', 'URL': 'url3'},
                       {'TITLE': 'title3', 'URL': 'url3'}]}

This is what I do, I split the <br> out.

for every_report in format_1['result']:
    titles = every_report['TITLE'].split('<br>')
    pdf_titles = ['This is a title: ' + title for title in titles]
    for x in range(len(pdf_titles)):
        print(pdf_titles[x])

for every_report in format_1['result']:
    urls = every_report['URL'].split('<br>')
    pdf_urls = ['http://static.sse.com.cn' + url for url in urls]
    for x in range(len(pdf_urls)):
        print(pdf_urls[x])

this is the output

title1
title2
title3
url1
url2
url3

What should I do next to make them becomes format_2

{'result': [{'TITLE': 'title1', 'URL': 'url1'},
            {'TITLE': 'title2', 'URL': 'url3'},
            {'TITLE': 'title3', 'URL': 'url3'}]}

Because I would like to get every the URL from the list, download it, and named it with the TITLE. The code for this part are something like this(may have error, coz I didnt get a chance to try it as I'm stuck at compiling the format I need)

for every_report in my_format['result']:
        pdf_url = 'this is ths url: ' + every_report['URL']
        print(pdf_url)
        file_name = every_report['TITLE'] + '.pdf'

        resource = requests.get(pdf_url, stream = True)
        with open(file_name, 'wb') as temp:
            for chunk in resource.iter_content(1024):
                temp.write(chunk)
            print("This is the title: " + file_name + " Download Completed")

The result should have: Downloaded file into projects Output for reference

Title1
url1
Title2
url2
Title3
url3

The url suppose to be a real website url, but its too long, so I replace it with url here.

web-scraping java json python
1 Answers

You can use this (I used list comprehention):

def change_json_format(json_format):
    return {'result': [
        {'TITLE': title, 'URL': url} 
                for title, url in zip(
                    json_format['result'][0]['TITLE'].split('<br>'),
                    json_format['result'][0]['URL'].split('<br>')
                    )]}


# testing ...
format_1 = {'result': [
    {'TITLE': 'title1<br>title2<br>title3',
     'URL': 'url1<br>url2<br>url3'}]}

print(change_json_format(format_1))

output:

{'result': [{'TITLE': 'title1', 'URL': 'url1'}, {'TITLE': 'title2', 'URL': 'url2'}, {'TITLE': 'title3', 'URL': 'url3'}]}
mrCopiCat0
Related