Transform script extracted via lxml into a python dictionary

Viewed 35

I read an html of a site by using the python lxml library and I got a string response. In the original site, my tree.xpath "data" is a Json inside a script.

I would like to transform my data[1].text into a python dictionary in order to extract the data contained inside. If there is a better o way to do it, let me know

import lxml.html
from lxml import etree

htmlparser = lxml.etree.HTMLParser()
tree = lxml.etree.fromstring(response, parser=htmlparser)
data = tree.xpath('//script[@type="application/ld+json"]')


data[1].text 

>>> '{\n   "@type" : "JobPosting",\n   "validThrough" : "2022-08-17T10:07:41",\n   "employmentType" : [\n      "FULL_TIME"\n   ],\n   "jobLocationType" : "",\n   "baseSalary" : {\n      "currency" : "USD",\n      "value" : {\n         "@type" : "QuantitativeValue",\n         "value" : "",\n         "unitText" : ""\n      },\n      "@type" : "MonetaryAmount"\n   },\n   "industry" : "Hospitality & Catering",\n   "jobLocation" : {\n      "@type" : "Place",\n      "address" : {\n         "@type" : "PostalAddress",\n         "addressRegion" : "GA",\n         "addressCountry" : "US",\n         "streetAddress" : "",\n         "addressLocality" : "Peachtree Corners, GA",\n         "postalCode" : ""\n      }\n   },\n   "title" : "Shift Managers",\n   "hiringOrganization" : {\n      "name" : "Checkers Drive In Restaurants, Inc.",\n      "sameAs" : "https://www.myjob.com/company/checkers-drive-in-restaurants-inc-",\n      "@type" : "Organization"\n   },\n   "description" : "<p></p>\\n<p>PURPOSE OF YOUR WORK</p>\\n<p>At Dominoes we make a difference in people&#39;s lives by serving our Teams, and Guests. The Shift Manager will support the restaurant and General Manager to ensure we are 1st Choice for our Guests and Employees.</p>\\n<p>HOW YOU MAKE A DIFFERENCE EVERYDAY</p>\\n<ul>\\n<li>Building sales and profits by promoting Guest satisfaction and managing restaurant operations</li>\\n<li>Escalating any concerns or recommendations with the restaurant, employees or Guests to the General Manager</li>\\n<li>Ensuring policy and procedures are being followed on shifts Training Team Members on Company operations, policies and procedures and Guest service</li>\\n<li>Identifying and responding to complaints and policy and procedures violations</li>\\n<li>Helping ensure the restaurant is a safe, clean and fun environment for our employees and Guests!</li>\\n</ul>\\n<p>YOU&#39;VE GOT THIS?</p>\\n<ul>\\n<li>1-2 years restaurant management experience, preferably in Quick Service Restaurants (QSR)</li>\\n<li>Basic math skills</li>\\n<li>Ability to work flexible schedule and extended hours</li>\\n<li>High energy to keep up with our fast-paced environment</li>\\n<li>Commitment to our core values of integrity, service, excellence, and courage to be bold &amp; grow</li>\\n</ul>\\n<p>WHAT&#39;S IN IT FOR YOU?</p>\\n<ul>\\n<li>Operations Excellence: Our team expects and delivers nothing but the best</li>\\n<li>Training &amp; Development: We bring out the best by ensuring everyone gets well trained</li>\\n<li>Personal &amp; Career Growth: You can do more, get more, and be more at Checkers</li>\\n<li>Benefits &amp; Rewards: We offer Comprehensive benefits such as Medical and Dental Plans as well as recognition programs including bonuses and opportunities for advancement</li>\\n<li>Compensation: Work today &amp; get paid tomorrow!</li>\\n</ul>\\n<p>Details available at the restaurant during your interview</p>",\n   "@context" : "http://schema.org/",\n   "directApply" : "False",\n   "datePosted" : "2022-07-30T20:00:00"\n}\n'

What I want:

my_dict = {
    "@type" : "JobPosting",
    "validThrough" : "2022-08-17T10:07:41"
    ...
}
1 Answers

You can always do it like this:

import json
string = '''
{\n   "@type" : "JobPosting",\n   "validThrough" : "2022-08-17T10:07:41",\n   "employmentType" : [\n      "FULL_TIME"\n   ],\n   "jobLocationType" : "",\n   "baseSalary" : {\n      "currency" : "USD",\n      "value" : {\n         "@type" : "QuantitativeValue",\n         "value" : "",\n         "unitText" : ""\n      },\n      "@type" : "MonetaryAmount"\n   },\n   "industry" : "Hospitality & Catering",\n   "jobLocation" : {\n      "@type" : "Place",\n      "address" : {\n         "@type" : "PostalAddress",\n         "addressRegion" : "GA",\n         "addressCountry" : "US",\n         "streetAddress" : "",\n         "addressLocality" : "Peachtree Corners, GA",\n         "postalCode" : ""\n      }\n   },\n   "title" : "Shift Managers",\n   "hiringOrganization" : {\n      "name" : "Checkers Drive In Restaurants, Inc.",\n      "sameAs" : "https://www.myjob.com/company/checkers-drive-in-restaurants-inc-",\n      "@type" : "Organization"\n   },\n   "description" : "<p></p>\\n<p>PURPOSE OF YOUR WORK</p>\\n<p>At Dominoes we make a difference in people&#39;s lives by serving our Teams, and Guests. The Shift Manager will support the restaurant and General Manager to ensure we are 1st Choice for our Guests and Employees.</p>\\n<p>HOW YOU MAKE A DIFFERENCE EVERYDAY</p>\\n<ul>\\n<li>Building sales and profits by promoting Guest satisfaction and managing restaurant operations</li>\\n<li>Escalating any concerns or recommendations with the restaurant, employees or Guests to the General Manager</li>\\n<li>Ensuring policy and procedures are being followed on shifts Training Team Members on Company operations, policies and procedures and Guest service</li>\\n<li>Identifying and responding to complaints and policy and procedures violations</li>\\n<li>Helping ensure the restaurant is a safe, clean and fun environment for our employees and Guests!</li>\\n</ul>\\n<p>YOU&#39;VE GOT THIS?</p>\\n<ul>\\n<li>1-2 years restaurant management experience, preferably in Quick Service Restaurants (QSR)</li>\\n<li>Basic math skills</li>\\n<li>Ability to work flexible schedule and extended hours</li>\\n<li>High energy to keep up with our fast-paced environment</li>\\n<li>Commitment to our core values of integrity, service, excellence, and courage to be bold &amp; grow</li>\\n</ul>\\n<p>WHAT&#39;S IN IT FOR YOU?</p>\\n<ul>\\n<li>Operations Excellence: Our team expects and delivers nothing but the best</li>\\n<li>Training &amp; Development: We bring out the best by ensuring everyone gets well trained</li>\\n<li>Personal &amp; Career Growth: You can do more, get more, and be more at Checkers</li>\\n<li>Benefits &amp; Rewards: We offer Comprehensive benefits such as Medical and Dental Plans as well as recognition programs including bonuses and opportunities for advancement</li>\\n<li>Compensation: Work today &amp; get paid tomorrow!</li>\\n</ul>\\n<p>Details available at the restaurant during your interview</p>",\n   "@context" : "http://schema.org/",\n   "directApply" : "False",\n   "datePosted" : "2022-07-30T20:00:00"\n}\n
'''

el = json.loads(string)
print(el)
print('________________-')
print(el['validThrough'])

This results in an object which can be dissected further:

{'@type': 'JobPosting', 'validThrough': '2022-08-17T10:07:41', 'employmentType': ['FULL_TIME'], 'jobLocationType': '', 'baseSalary': {'currency': 'USD', 'value': {'@type': 'QuantitativeValue', 'value': '', 'unitText': ''}, '@type': 'MonetaryAmount'}, 'industry': 'Hospitality & Catering', 'jobLocation': {'@type': 'Place', 'address': {'@type': 'PostalAddress', 'addressRegion': 'GA', 'addressCountry': 'US', 'streetAddress': '', 'addressLocality': 'Peachtree Corners, GA', 'postalCode': ''}}, 'title': 'Shift Managers', 'hiringOrganization': {'name': 'Checkers Drive In Restaurants, Inc.', 'sameAs': 'https://www.myjob.com/company/checkers-drive-in-restaurants-inc-', '@type': 'Organization'}, 'description': '<p></p>\n<p>PURPOSE OF YOUR WORK</p>\n<p>At Dominoes we make a difference in people&#39;s lives by serving our Teams, and Guests. The Shift Manager will support the restaurant and General Manager to ensure we are 1st Choice for our Guests and Employees.</p>\n<p>HOW YOU MAKE A DIFFERENCE EVERYDAY</p>\n<ul>\n<li>Building sales and profits by promoting Guest satisfaction and managing restaurant operations</li>\n<li>Escalating any concerns or recommendations with the restaurant, employees or Guests to the General Manager</li>\n<li>Ensuring policy and procedures are being followed on shifts Training Team Members on Company operations, policies and procedures and Guest service</li>\n<li>Identifying and responding to complaints and policy and procedures violations</li>\n<li>Helping ensure the restaurant is a safe, clean and fun environment for our employees and Guests!</li>\n</ul>\n<p>YOU&#39;VE GOT THIS?</p>\n<ul>\n<li>1-2 years restaurant management experience, preferably in Quick Service Restaurants (QSR)</li>\n<li>Basic math skills</li>\n<li>Ability to work flexible schedule and extended hours</li>\n<li>High energy to keep up with our fast-paced environment</li>\n<li>Commitment to our core values of integrity, service, excellence, and courage to be bold &amp; grow</li>\n</ul>\n<p>WHAT&#39;S IN IT FOR YOU?</p>\n<ul>\n<li>Operations Excellence: Our team expects and delivers nothing but the best</li>\n<li>Training &amp; Development: We bring out the best by ensuring everyone gets well trained</li>\n<li>Personal &amp; Career Growth: You can do more, get more, and be more at Checkers</li>\n<li>Benefits &amp; Rewards: We offer Comprehensive benefits such as Medical and Dental Plans as well as recognition programs including bonuses and opportunities for advancement</li>\n<li>Compensation: Work today &amp; get paid tomorrow!</li>\n</ul>\n<p>Details available at the restaurant during your interview</p>', '@context': 'http://schema.org/', 'directApply': 'False', 'datePosted': '2022-07-30T20:00:00'}
________________-
2022-08-17T10:07:41
Related