I'm new with python, BeautifulSoup and other but I want to extract json data which are inside a javascript variable in the "script" tag of a web site.
Here is my code for now :
import re
from bs4 import BeautifulSoup
import json
import requests
url = 'myUrl'
page = requests.get(url).content
soup = BeautifulSoup(page, "html.parser")
pattern = re.compile(r"var hours = .")
script = soup.find("script",text=pattern)
print(script)
for now I can extract the data with the format like :
<script>
var hours = [{...dataIwant...}];
<\script>
But I only want the data without "script" or "var hours =". I want to change it in json and put it in apache nifi.
I have tried pretty much everything I have found here and on google. But most of the time when I try to extract the variable and change it to json format I have "None" or other errors.
So if you have some tips to help me get the data in json format it would be great !
Thank you !