http: read on closed response body - httptest.NewServer

Question

I am trying to get to grips with testing using the httptest.NewServer and I am hitting a roadblock.

In my code I am making a GET request to an external API and I want to write a test for this using httptest.NewServer.

Here is my code making the request (main.go):

package main

import (
    "fmt"
    "io"
    "io/ioutil"
    "log"
    "net/http"
)

type HTTPClient interface {
    Do(req *http.Request) (*http.Response, error)
}

type NewRequest interface {
    NewRequest(method string, url string, body io.Reader) (*http.Request, error)
}

var (
    Client HTTPClient
)

func init() {
    Client = &http.Client{}
}

func main() {
    url := "https://httpbin.org/get"
    GetData(url)
}

func GetData(url string) (*http.Response, error) {
    req, err := http.NewRequest(http.MethodGet, url, nil)
    if err != nil {
        log.Fatalln(err)
        return nil, err
    }

    resp, err := Client.Do(req)

    if err != nil {
        log.Fatalln(err)
        return nil, err
    }

    defer resp.Body.Close()

    responseBody, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        log.Fatal(err)
        return nil, err
    }

    fmt.Println(resp.Status)
    fmt.Println(string(responseBody))
    return resp, nil
}

When I run this it works fine.

Here is my test file:

package main

import (
    "fmt"
    "io/ioutil"
    "log"
    "net/http"
    "net/http/httptest"
    "testing"
)

func TestYourHTTPGet(t *testing.T){

    ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintln(w, `response from the mock server goes here`)
    }))
    defer ts.Close()

    mockServerURL := ts.URL

    resp, err := GetData(mockServerURL)
    if err != nil {
        fmt.Println("Error 1: ", err)
    }
    defer resp.Body.Close()

    responseBody, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        log.Fatal("Error 2: ", err)
    }

    fmt.Println(resp.Status)
    fmt.Println(string(responseBody))
}

When I run go test I receive the error: http: read on closed response body. If I remove defer resp.Body.Close() from main.go the test passes correctly.

I am not sure why this is happening and was hoping that someone could explain what is going on here?

user_4000206 · Answer

As @Cerise Limón says you call resp.Body.Close() twice and then try to read closed body. To fix yor code you can remove body processing from GetData function and do it outside GetData or return the body and do not read it in test.

main.go:

package main

import (
    "fmt"
    "io/ioutil"
    "log"
    "net/http"
)

var Client = &http.Client{}

func main() {
    url := "https://httpbin.org/get"

    status, data, err := GetData(url)
    if err != nil {
        log.Fatalln(err)
    }

    fmt.Println(status)
    fmt.Println(string(data))
}

func GetData(url string) (status string, body []byte, err error) {
    req, err := http.NewRequest(http.MethodGet, url, nil)
    if err != nil {
        return
    }

    resp, err := Client.Do(req)
    if err != nil {
        return
    }

    defer resp.Body.Close()

    body, err = ioutil.ReadAll(resp.Body)
    if err != nil {
        log.Fatalln(err)
    }

    return resp.Status, body, nil
}

main_test.go:

package main

import (
    "fmt"
    "net/http"
    "net/http/httptest"
    "testing"
)

func TestYourHTTPGet(t *testing.T){
    ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintln(w, `response from the mock server goes here`)
    }))
    defer ts.Close()

    mockServerURL := ts.URL

    status, data, err := GetData(mockServerURL)
    if err != nil {
        fmt.Println("Error 1: ", err)
    }

    fmt.Println(status)
    fmt.Println(string(data))
}

user_16100647 · Answer

Your GetData()'s return is a pointer. You run GetData() in main.go, when retun, it will close the resp.body. And if you read it again, it cause http: read on closed response body

So if you want read the body again, you should not return *http.Response, you should clone the resp.body to return

http: read on closed response body - httptest.NewServer

2 Answers