I noticed that in Golang http.client, there is connections pool settings, like: MaxIdleConns or DefaultMaxIdleConnsPerHost, check the source code here.
I will expect that with the increase idle connections pool, the performance will be improved dramatically.
So I wrote a simple program try to proof that, but unfortunately, it shows that the pool size is not relevant to the performance at all. Could anyone help to explain? Also I post the program code below, please also check if there is an error there? Thank you!
// In this example, there are 3 routers:
// * /per-client-per-request: this route a new http.Client instance for each request.
// * /2pool: this route uses the default transport which has 2 maxIdlConnsPerHost
// * /100pool: this route uses a modified transport which has 100 maxIdlConnsPerHost
// Ideally, the /per-client-per-request should has the worst performance, /2pool is better, and /100pool should be the best.
// However, the result is totally random. Sometimes /2pool is the best, sometimes /100pool is the best, and sometimes /per-client-per-request is the best.
package main
import (
"fmt"
"io"
"log"
"net/http"
"os"
"github.com/gorilla/mux"
)
func startResourceServer() {
r := mux.NewRouter()
r.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
// time.Sleep(5 * time.Millisecond)
w.Write([]byte("Hello World"))
})
srv := &http.Server{
Addr: "0.0.0.0:8081",
Handler: r,
}
go func() {
if err := srv.ListenAndServe(); err != nil {
log.Println(err)
}
}()
}
type Handler struct {
httpClientWith2Pool *http.Client
httpClientWith100Pool *http.Client
}
func main() {
startResourceServer()
transportWithMax100Pool := http.DefaultTransport.(*http.Transport).Clone()
transportWithMax100Pool.MaxIdleConnsPerHost = 100
handler := &Handler{
httpClientWith100Pool: &http.Client{Transport: transportWithMax100Pool},
httpClientWith2Pool: &http.Client{},
}
r := mux.NewRouter()
r.HandleFunc("/2pool", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte(fmt.Sprintf("res len: %d", visitRemoteServer(handler.httpClientWith2Pool))))
})
r.HandleFunc("/100pool", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte(fmt.Sprintf("res len: %d", visitRemoteServer(handler.httpClientWith100Pool))))
})
r.HandleFunc("/per-client-per-request", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte(fmt.Sprintf("res len: %d", visitRemoteServer(&http.Client{}))))
})
srv := &http.Server{
Addr: "0.0.0.0:8080",
Handler: r,
}
log.Fatal(srv.ListenAndServe())
}
func visitRemoteServer(httpClient *http.Client) int {
res, err := httpClient.Get("http://localhost:8081")
if err != nil {
fmt.Printf("error making http request: %s", err)
os.Exit(1)
}
body, _ := io.ReadAll(res.Body)
return len(body)
}
And I use vegeta tool to test:
echo "GET http://localhost:8080/per-client-per-request" | vegeta attack -duration=5s -rate=200 | vegeta report
echo "GET http://localhost:8080/2pool" | vegeta attack -duration=5s -rate=200 | vegeta report
echo "GET http://localhost:8080/100pool" | vegeta attack -duration=5s -rate=200 | vegeta report
I expected that the results should gets better and better, but unfortunately, it is very random, and even the last load test which has 100 connections pool usually was the worst case. Check the below typical result:
Requests [total, rate, throughput] 1000, 200.22, 200.18
Duration [total, attack, wait] 4.995s, 4.995s, 857.5µs
Latencies [min, mean, 50, 90, 95, 99, max] 206.583µs, 833.203µs, 599.667µs, 1.577ms, 2.167ms, 3.871ms, 8.396ms
Bytes In [total, mean] 11000, 11.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:1000
Error Set:
Requests [total, rate, throughput] 1000, 200.21, 200.17
Duration [total, attack, wait] 4.996s, 4.995s, 1.004ms
Latencies [min, mean, 50, 90, 95, 99, max] 258.166µs, 862.518µs, 700.689µs, 1.505ms, 2.052ms, 3.312ms, 7.735ms
Bytes In [total, mean] 11000, 11.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:1000
Error Set:
Requests [total, rate, throughput] 1000, 200.20, 200.18
Duration [total, attack, wait] 4.996s, 4.995s, 594.958µs
Latencies [min, mean, 50, 90, 95, 99, max] 238µs, 916.789µs, 698.426µs, 1.542ms, 2.252ms, 3.896ms, 9.544ms
Bytes In [total, mean] 11000, 11.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:1000
Error Set:
Update #1
Based on the @steffen-ullrich comment, I changed my program as below. The new code will use http.Client directly send request to google. Two cases are using maxIdleConnectionsPerHost to 2 and 100 respectively.
package main
import (
"fmt"
"net/http"
"sync"
"time"
)
const TIMES = 300
var wg sync.WaitGroup
func check(maxIdleConnsPerHost int) {
transport := http.DefaultTransport.(*http.Transport).Clone()
transport.MaxIdleConns = 100
transport.MaxIdleConnsPerHost = maxIdleConnsPerHost
c := make(chan int)
httpClient := &http.Client{Transport: transport}
results := make([]int, 0)
go func() {
for i := 0; i < TIMES; i++ {
c <- i
time.Sleep(10 * time.Millisecond)
}
close(c)
}()
wg.Add(TIMES)
for range c {
go func(httpClient *http.Client) {
start := time.Now()
defer wg.Done()
defer func() {
results = append(results, int(time.Since(start).Milliseconds()))
}()
_, err := httpClient.Get("https://www.google.com")
if err != nil {
panic(err)
}
}(httpClient)
}
wg.Wait()
sum := 0
for _, v := range results {
sum += v
}
// fmt.Printf("list: %v \n", results)
fmt.Printf("When maxIdleConnectionsPerHost is %d, ", maxIdleConnsPerHost)
fmt.Printf("Average elapse ms: %d \n", sum/TIMES)
}
func main() {
check(2)
check(100)
}
However, the result still showing the connection pool size is irrelevant to the performance. The below is the results I ran 4 times.
When maxIdleConnectionsPerHost is 2, Average elapse ms: 649
When maxIdleConnectionsPerHost is 100, Average elapse ms: 772
When maxIdleConnectionsPerHost is 2, Average elapse ms: 755
When maxIdleConnectionsPerHost is 100, Average elapse ms: 739
When maxIdleConnectionsPerHost is 2, Average elapse ms: 748
When maxIdleConnectionsPerHost is 100, Average elapse ms: 756
When maxIdleConnectionsPerHost is 2, Average elapse ms: 810
When maxIdleConnectionsPerHost is 100, Average elapse ms: 780