How can I get frequently updated .php text from a website in Python using BeautifulSoup4?

user_15009676 0
Viewed 43

I would like create an automatic script to download a .php textfile from a webpage which is frequently updated. My program uses requests to get the webpage.

The code:

import os, pathlib, subprocess,requests, time, sys



url = 'http://metar.vatsim.net/metar.php?id=all'

current_dir = pathlib.Path(__file__).parent
os.chdir(current_dir)




icao = sys.argv[1]
fp = requests.get(url)
mybytes = fp.read()

mystr = mybytes.decode("utf8")
fp.close()

dict = {}

fls = str.splitlines(mystr)
for x in range(len(fls)):
    cur = str.split(fls[x])
    dict[cur[0]] = " ".join(cur)
    
try:
    print(dict[icao])
except:
    print('INCORRECT FORMAT OR AIRPORT ID\n')

When I try to read fp, it shows the err:

mybytes = fp.read()
AttributeError: 'Response' object has no attribute 'read'

Is there a better way to solve this, I am kind of stuck.

beautifulsoup python
2 Answers

What you are looking for is urllib.request, not requests.

Maybe this will work:

import urllib.request

fp = urllib.request.urlopen(url)
mybytes = fp.read()

mystr = mybytes.decode("utf8")
fp.close()

This will read the text present in http://metar.vatsim.net/metar.php?id=all.

user_157923990

You can absolutely use requests. You then want to extract the .text.

Also, don't overwrite inbuilt dict in the way you are doing.

import requests

url = 'http://metar.vatsim.net/metar.php?id=all'
fp = requests.get(url)
mystr = fp.text
a_dict = {}

fls = str.splitlines(mystr)

for x in range(len(fls)):
    cur = str.split(fls[x])
    a_dict[cur[0]] = " ".join(cur)
    
try:
    print(a_dict)
except:
    print('INCORRECT FORMAT OR AIRPORT ID\n')
user_62412350
Related