Im trying to do a scraping to a website... Im already logged into the website and got access to some of the pages, but when I try to access a page with .do extension then it throws me an error like its not getting the rights parameters... The web Site has some Pages with the .do extension... this is my code:
Connection.Response loginForm = Jsoup.connect("urlLogin")
.referrer("urlRefer")
.userAgent("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36")
.timeout(10 * 1000)
.followRedirects(true)
.method(Connection.Method.GET)
.execute();
Map<String, String> mapLoginPageCookies = loginForm.cookies();
Map<String, String> mapParams = new HashMap<String, String>();
mapParams.put("FormName", "MyFormName");
mapParams.put("seclogin", "on");
mapParams.put("j_username", "My User Name");
mapParams.put("j_password", "My Password.");
mapParams.put("remember", "1");
mapParams.put("proceed", "Go");
I login using the j_security_check.
Connection.Response mainPage = Jsoup.connect("urlLogin/j_security_check")
.referrer("urlRefer")
.userAgent("Mozilla/5.0")
.timeout(10 * 1000)
.data(mapParams)
.cookies(mapLoginPageCookies)
.followRedirects(true)
//.execute();
.method(Connection.Method.POST).execute();
Document document = mainPage.parse();
Map<String, String> mapLoggedInCookies = mainPage.cookies();
then i tryed to get the document
Element loggedIn = Jsoup.connect("urlData/Page.do?Parameter1=Value1&Parameter2=Value2")
.followRedirects(true)
.userAgent("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36")
.cookies(mapLoggedInCookies)
.post();