I want to get a page using it's IP instead host(www.example.com) to avoiding DNS lookup , but how do I achieve that? Right now I am using
URL myURL;
try {
myURL = new URL(url.getUrl() + "/robots.txt");
HttpURLConnection myURLConnection = (HttpURLConnection) myURL.openConnection();
myURLConnection.setRequestMethod("GET");
myURLConnection.setUseCaches(false);
BufferedReader in = new BufferedReader(new InputStreamReader(myURLConnection.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
System.out.println(response.toString());
File folder = new File("src/workspace/" + url.getHost());
folder.mkdirs();
folder.createNewFile();
RunCrawler.writeFile("src/workspace/" + url.getHost() + "/robots.txt", response.toString());
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (ProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
I want to use for example 172.217.18.164 instead www.google.com
Can anyone please help me? Thanks