Hi i am making a program that scrapes jobs from stackoverflow accordingly when i enter a specific keyword.
To do that, I need to know the maximum number of pages when you enter a specific keyword. However, the html content viewed through the developer tool and the html imported through beautifulsoup are different and are having a problem.
For example, let's call the keyword'vue'.
The url is as follows: https://stackoverflow.com/jobs?q=vue
The number of pages checked through developer tools is 2.
However, if you check the number of pages after entering the following code and outputting the result, the maximum number of pages is 43.
from bs4 import BeautifulSoup
import requests
url = "https://stackoverflow.com/jobs?q=vue"
result = requests.get(url)
soup = BeautifulSoup(result.text,"html.parser")
print(soup)
I need to load the content corresponding to 2 pages related to vue, but I am having a problem of scraping not only this content but also unrelated content.
I did not know where the problem occurred, so I asked questions. Any help would be appreciated.
