I try to crawl some app-data from google-play-store (https://play-fe.googleapis.com). When I try to fetch it without proxy, it works properly according to my location. But when I try to fetch it with proxy and with specific country, it doesn’t work properly.
As a proxy provider, I used Bright-Data, Smart-Proxy and https://proxy.webshare.io/. All of them behave similar.
Proxy Type: Data Center
My java code looks like:
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(new AuthScope(PROXY_HOST, PROXY_PORT), new UsernamePasswordCredentials(PROXY_USER, PROXY_PASSWORD));
var client = HttpClientBuilder.create()
.useSystemProperties()
.setProxy(new HttpHost(PROXY_HOST, PROXY_PORT))
.setDefaultCredentialsProvider(credentialsProvider)
.setProxyAuthenticationStrategy(new ProxyAuthenticationStrategy())
.setDefaultSocketConfig(SocketConfig.custom().setSoTimeout(30000).build())
.build();
var requestFactory = new HttpComponentsClientHttpRequestFactory(client);
Note: By the way, I check the ip country via https://api.myip.com/ and proxy provides proper ip-address.
Note: And I know that I can handle it via residential-proxy but residential-proxy is not cheap.
Is there any suggestion or comment?