I am scraping this property listing website in order to build a predictive model for the price and to analyse the trend of pricing. I can scrape everything out of the present view for the website from page 1 to 100 without any trouble which means this will be my 2022 property dataset.
Unfortuantely, the listings caps at page 100 which means I cant retrieve data from 2021. I am not sure how to extract data from listings posted in previous years or dates. I would like to extract data when the website was in 13 September 2021 using rvest or any other tools in r.
This is the website https://www.iproperty.com.my/sale/all-residential/?page=1
At the bottom of the page 100, we can see that it displays Viewing 1981-2000 of 359514 property results. Is there a way to reach all the 359514 results.
The aim of the question is to find a way get results of previous years starting from 2010.
Any assistance will be appreciated. Thank you