Stop all running nested loops inside for -> go func -> for loop

LAD 0
Viewed 65

The best way I figured out how to "multi-thread" requests with a different proxy each time was to nest a go func and for loop inside of another for loop, but I can't figure out how to stop all loops like a break normally would, I tried a regular break also tried break out and added out: above the loops but that didn't stop it.

package main

import (
    "log"
    "encoding/json"
    "github.com/parnurzeal/gorequest"
)

func main(){
    rep := 100 
    for i := 0; i < rep; i++ { 
        log.Println("starting loop")
        go func() { 
            for{
                request := gorequest.New()
                resp, body, errs := request.Get("https://discord.com/api/v9/invites/family").End()
                if errs != nil {
                    return
                }
                if resp.StatusCode == 200{
                    var result map[string]interface{}
                    json.Unmarshal([]byte(body), &result)
                    serverName := result["guild"].(map[string]interface{})["name"]       
                    log.Println(sererName +" response 200, closing all loops")

                    //break all loops and goroutine here
                }
            }
        }
    }
    log.Println("response 200,closed all loops")
go
1 Answers

Answering this would be complicated by your use of parnurzeal/gorequest because that package does not provide any obvious way to cancel requests (see this issue). Because your focus appears to be on the process rather than the specific function I've just used the standard library (http) instead (if you do need to use gorequest then perhaps ask a question specifically about that).

Anyway the below solution demonstrates a few things:

  • Uses a Waitgroup so it knows when all go routines are done (not essential here but often you want to know you have shutdown cleanly)
  • Passes the result out via a channel (updating shared variables from a goroutine leads to data races).
  • Uses a context for cancellation. The cancel function is called when we have a result and this will stop in progress requests.
package main

import (
    "context"
    "encoding/json"
    "errors"
    "fmt"
    "log"
    "net/http"
    "sync"
)

func main() {
    // Get the context and a function to cancel it
    ctx, cancel := context.WithCancel(context.Background())
    defer cancel() // Not really required here but its good practice to ensure context is cancelled eventually.

    results := make(chan string)

    const goRoutineCount = 100
    var wg sync.WaitGroup
    wg.Add(goRoutineCount) // we will be waiting on 100 goRoutines

    for i := 0; i < goRoutineCount; i++ {
        go func() {
            defer wg.Done() // Decrement WaitGroup when goRoutine exits

            req, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://discord.com/api/v9/invites/family", nil)
            if err != nil {
                panic(err)
            }
            resp, err := http.DefaultClient.Do(req)
            if err != nil {
                if errors.Is(err, context.Canceled) {
                    return // The error is due to the context being cancelled so just shutdown
                }
                panic(err)
            }
            defer resp.Body.Close() // Ensure body is closed
            if resp.StatusCode == 200 {
                var result map[string]interface{}
                if err = json.NewDecoder(resp.Body).Decode(&result); err != nil {
                    panic(err)
                }
                serverName := result["guild"].(map[string]interface{})["name"]
                results <- serverName.(string) // Should error check this...
                cancel()                       // We have a result so all goroutines can stop now!
            }
        }()
    }

    // We need to process results until everything has shutdown; simple approach is to just close the channel when done
    go func() {
        wg.Wait()
        close(results)
    }()

    var firstResult string
    requestsProcessed := 0
    for x := range results {
        fmt.Println("got result")
        if requestsProcessed == 0 {
            firstResult = x
        }
        requestsProcessed++ // Possible that we will get more than one result (remember that requests are running in parallel)
    }

    // At this point all goroutines have shutdown
    if requestsProcessed == 0 {
        log.Println("No results received")
    } else {
        log.Printf("xx%s response 200, closing all loops (requests processed: %d)", firstResult, requestsProcessed)
    }
}
Brits0
Related