I'm trying to get data from http://www.headyversion.com/. The HTML for website's search bar is: <form action="/search/" method="post" id="search_songs"> <input id="id_title" type="text" placeholder="Search songs.." class="searchbox" name="title" /> <div style='display:none'><input type='hidden' name='csrfmiddlewaretoken' value='1P9wkkThey8CETtstsK0UbxAnMdcyH65' /></div> <input type="submit" id="top_bar_search" class="btn btn-primary" value="go"/> </form>
A successful search done in browser has this form (using the song Dark Star as an example):
HTML Form URL Encoded: application/x-www-form-urlencoded
Form item: "title" = "dark star"
Key: title
Value: dark star
Form item: "csrfmiddlewaretoken" = "1P9wkkThey8CETtstsK0UbxAnMdcyH65"
Key: csrfmiddlewaretoken
Value: 1P9wkkThey8CETtstsK0UbxAnMdcyH65
I'm trying to perform a search on the website in Python, using aiohttp.
async with aiohttp.ClientSession() as session:
async with session.post("http://headyversion.com/search", data={"title":"dark star"}) as response:
content = await response.read()
print(content)
print(response.url)
The response URL just links back to http://www.headyversion.com. I've also tried performing a GET request first to get the CSRF middleware token, and then adding that to the data dict in the POST request, but that gives the same result.