How to specify User Agent and Referer in FileUtils.copyURLToFile(URL, File) method?

user_462347 0
Viewed 2949

I'm using FileUtils.copyURLToFile(URL, File), an Apache Commons IO 2.4 part, to download and save the file on my computer. The problem is that some sites refuse connection without referrer and user agent data.

My questions:

  1. Is there any way to specify user agent and referrer to the copyURLToFile method?
  2. Or should I use another approach to download a file and then save a given InputStream to file?
apache-commons http-headers apache java
3 Answers

Completing the accepted answer on how to handle timeouts:

If you want to set timeouts, you have to create the CloseableHttpClient like this:

RequestConfig config = RequestConfig.custom()
                 .setConnectTimeout(connectionTimeout)
                 .setConnectionRequestTimeout(readDataTimeout)
                 .setSocketTimeout(readDataTimeout)
                 .build();

CloseableHttpClient httpClient = HttpClientBuilder
                 .create()
                 .setDefaultRequestConfig(config)
                 .build();

And, it may be a good idea to create your CloseableHttpClient using a try-with-resource statement to handle its closing:

try (CloseableHttpClient httpClient = HttpClientBuilder.create().setDefaultRequestConfig(config).build()) {
  ... rest of the code using httpClient
}
user_50465860
Related