How Can I use proxy properly on Java?

Viewed 28

I try to crawl some app-data from google-play-store (https://play-fe.googleapis.com). When I try to fetch it without proxy, it works properly according to my location. But when I try to fetch it with proxy and with specific country, it doesn’t work properly.

As a proxy provider, I used Bright-Data, Smart-Proxy and https://proxy.webshare.io/. All of them behave similar.

  1. Proxy Type: Data Center

  2. URL: https://play-fe.googleapis.com

My java code looks like:

    CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
    credentialsProvider.setCredentials(new AuthScope(PROXY_HOST, PROXY_PORT), new UsernamePasswordCredentials(PROXY_USER, PROXY_PASSWORD));

    var client = HttpClientBuilder.create()
            .useSystemProperties()
            .setProxy(new HttpHost(PROXY_HOST, PROXY_PORT))
            .setDefaultCredentialsProvider(credentialsProvider)
            .setProxyAuthenticationStrategy(new ProxyAuthenticationStrategy())
            .setDefaultSocketConfig(SocketConfig.custom().setSoTimeout(30000).build())
            .build();
    var requestFactory = new HttpComponentsClientHttpRequestFactory(client); 

Note: By the way, I check the ip country via https://api.myip.com/ and proxy provides proper ip-address.

Note: And I know that I can handle it via residential-proxy but residential-proxy is not cheap.

Is there any suggestion or comment?

1 Answers

Data Center proxies aren’t good idea for country based requests. Use residential-proxy for these kind of requests. Best options according to price look like:

  1. shifter.io
  2. Smart-Proxy
  3. Bright-Data
Related