Best Practice to Use HttpClient in Multithreaded Environment

Viewed 130742

For a while, I have been using HttpClient in a multithreaded environment. For every thread, when it initiates a connection, it will create a completely new HttpClient instance.

Recently, I have discovered that, by using this approach, it can cause the user to have too many ports being opened, and most of the connections are in TIME_WAIT state.

http://www.opensubscriber.com/message/commons-httpclient-dev@jakarta.apache.org/86045.html

Hence, instead of each thread doing :

HttpClient c = new HttpClient();
try {
    c.executeMethod(method);
}
catch(...) {
}
finally {
    method.releaseConnection();
}

We plan to have :

[METHOD A]

// global_c is initialized once through
// HttpClient global_c = new HttpClient(new MultiThreadedHttpConnectionManager());

try {
    global_c.executeMethod(method);
}
catch(...) {
}
finally {
    method.releaseConnection();
}

In a normal situation, global_c will be accessed by 50++ threads concurrently. I was wondering, will this create any performance issues? Is MultiThreadedHttpConnectionManager using a lock-free mechanism to implement its thread safe policy?

If 10 threads are using global_c, will the other 40 threads be locked?

Or would it be better if, in every thread, I create an instance of an HttpClient, but release the connection manager explicitly?

[METHOD B]

MultiThreadedHttpConnectionManager connman = new MultiThreadedHttpConnectionManager();
HttpClient c = new HttpClient(connman);
try {
      c.executeMethod(method);
}
catch(...) {
}
finally {
    method.releaseConnection();
    connman.shutdown();
}

Will connman.shutdown() suffer performance issues?

May I know which method (A or B) is better, for application using an 50++ threads?

5 Answers

With HttpClient 4.5 you can do this:

CloseableHttpClient httpClient = HttpClients.custom().setConnectionManager(new PoolingHttpClientConnectionManager()).build();

Note that this one implements Closeable (for shutting down of the connection manager).

Related