Append not thread-safe?

user_7824245 0
Viewed 28735

I noticed that if I tried appending to a slice using goroutines inside a for loop, there would be instances where I would get missing/blank data:

destSlice := make([]myClass, 0)

var wg sync.WaitGroup
for _, myObject := range sourceSlice {
    wg.Add(1)
    go func(closureMyObject myClass) {
        defer wg.Done()
        var tmpObj myClass
        tmpObj.AttributeName = closureMyObject.AttributeName
        destSlice = append(destSlice, tmpObj)
    }(myObject)
}
wg.Wait()

Sometimes, when I print all AttributeNames from destSlice, some elements are empty strings (""), and other times, some elements from sourceSlice are not present in destSlice.

Does my code have a data race, and does this mean that append is not thread-safe for concurrent use by multiple goroutines?

goroutine go slice append concurrency
4 Answers

It's quite an old question but there is another small improvement that helps to get rid of mutex. You can use index to add to array. Each go routine will use it's own index. In this case synchronization is not necessary. 

destSlice := make([]myClass, len(sourceSlice))

var wg sync.WaitGroup
for i, myObject := range sourceSlice {
    wg.Add(1)
    go func(idx int, closureMyObject myClass) {
        defer wg.Done()
        var tmpObj myClass
        tmpObj.AttributeName = closureMyObject.AttributeName

        destSlice[idx] = tmpObj
     }(i, myObject)
}
wg.Wait()
user_11605810
user_36617480

Question has been answered, but my favorite way to solve this problem is with errgroup. One of the examples in the docs is this exact problem plus one nice addition the handling of errors.

Below is the meat of the example from the docs:

g, ctx := errgroup.WithContext(ctx)

searches := []Search{Web, Image, Video}
results := make([]Result, len(searches))
for i, search := range searches {
    i, search := i, search // https://golang.org/doc/faq#closures_and_goroutines
    g.Go(func() error {
        result, err := search(ctx, query)
        if err == nil {
            results[i] = result
        }
        return err
    })
}
if err := g.Wait(); err != nil {
    return nil, err
}
return results, nil

Hope this is helpful to those who are not aware of the errgroup package.

user_11106150
Related