HTTP request without last byte?

Question

I'm looking to test load my app in Golang. I haven't found this functionality in already existing tools, I tried all of them. Here is what I'm trying to do:

Create 100 exactly the same HTTP requests (as goroutines)
From each goroutine connect to HTTP server and send the body of the response (which can be up to few MB), except the last byte
Synchronize between all goroutines - pretty much wait until all threads are at the point where there is only 1 byte left to send
Based on input from Terminal (for example, when I hit Enter), send the remaining byte, so I can test how the server handles this type of load - 100 large requests at the same time

I looked at the docs of the standard HTTP library, and I don't think it's possible wit standard tools. I'm looking to rewrite some parts of HTTP library to have this support, or maybe even use the plain old OS sockets to perform this type of functionality. It will require a lot of time just to implement that.

I'm wondering if I'm missing something here, some kind of HTTP library feature that allows to do that easily? Appreiate any suggestion that might work without a full rewrite.

user_16964027 · Answer

To my understanding there is no way to send part of a http request then the rest at the end, but I believe I can help with the concurrency part.

Two variables here, threads (mind the python terminology) = number of simultaneous goroutines, number = number of times to

func main() {
    fmt.Println("Input # of times to run")
    var number int
    fmt.Scan(&number)
    fmt.Println("Input # of threads")
    var threads int
    fmt.Scan(&threads)
    swg := sizedwaitgroup.New(threads)
    for i := 0; i < number; i++ {
        swg.Add()
        go func(i int) {
            defer swg.Done()//Ensure to put your request after this line

            //Do request


        }(i)
    }
    swg.Wait()

}

This code uses the github.com/remeh/sizedwaitgroup library

Bear in mind, if one of the first requests is completed, it will start another without waiting for others to finish.

Here's it in practice: https://codeshare.io/3A3dj4 https://pastebin.com/DP1sn1m4

Edit: If you further and manage to send all but the last byte of the http request, you'll be wanted to use channels to communicate when to send the last byte, I'm not too good at them but this guide is great: https://go.dev/blog/pipelines

HTTP request without last byte?

1 Answers