Fetch path from URL and use it within python script

user_19304418 0
Viewed 57

G'day guys, I'm working on a python project that pulls weather data from BOM (https://bom.gov.au). The script works correctly, however I would like for it to be able to use part of the URL within the post request. i.e., the user navigates to https://example.com/taf/ymml, the script runs and uses YMML within the POST.

the script I am using is below. I would like to swap out 'YSSY' in myobj for something that pulls it from the url that the user navigates to.

import requests
import re

url = 'http://www.bom.gov.au/aviation/php/process.php'
myobj = {'keyword': 'YSSY', 'type': 'search', 'page': 'TAF'}
headers = {'User-Agent': 'Chrome/102.0.0.0'}

x = requests.post(url, data = myobj, headers=headers)

content = x.text

stripped = re.sub('<[^<]+?>', ' ', content)
split_string = stripped.split("METAR", 1)
substring = split_string[0]

print(substring)

Any ideas?

python-requests post url python
1 Answers

Ok so I've managed to get this working using fastapi. When a user navigates to example.com/taf/ymml, the site will return in plain text the taf for ymml. it can be substituted for any Australian Aerodrome. One thing I haven't figured out is how to remove the the square brackets around the taf, but that is a problem for another time.

from fastapi import FastAPI
import requests
from bs4 import BeautifulSoup


app = FastAPI()

@app.get("/taf/{icao}")
async def read_icao(icao):
    url = 'http://www.bom.gov.au/aviation/php/process.php'
    myobj = {'keyword': icao, 'type': 'search', 'page': 'TAF'}
    headers = {'User-Agent': 'Chrome/102.0.0.0'}

    x = requests.post(url, data = myobj, headers=headers)

    content = x.text

    split_string = content.split("METAR", 1)
    substring = split_string[0]

    soup = BeautifulSoup(substring, 'html.parser')

    for br in soup('br'):
        br.replace_with(' ')

    #Create TAFs array.


    tafs = []

    for taf in soup.find_all('p', class_="product"):
        full_taf = taf.get_text()


        tafs.append(full_taf.rstrip())



    return {tuple(tafs)}
user_193044180
Related