Thread-safe operation with Stdout and Stderr (exec. Cmd)

user_12270548 0
Viewed 396

I have code and it works in the correct way, but it isn’t thread-safety https://play.golang.org/p/8EY3i1Uk_aO in these rows race happens here.

stdout := cmd.Stdout.(*bytes.Buffer).String()
stderr := cmd.Stderr.(*bytes.Buffer).String()

I rewrote it in this way

readout, _ := cmd.StdoutPipe()
readerr, _ := cmd.StderrPipe()

The link https://play.golang.org/p/htbn2zXXeQk

I don’t like that MultiReader is used here and I cannot separate data stdout from stderr

r, _ := bufio.NewReader(io.MultiReader(readout, readerr)).ReadString('\n')

Also the second example doesn’t work (it is commented in the code). I expected stdout not to be empty (like here https://play.golang.org/p/8EY3i1Uk_aO)

How to make the logic like in the first example, but it should be thread-safety?

go stdout
3 Answers

You have to pump cmd.Stdout and cmd.Stderr in separate goroutines until they are closed, for example, like you did with cmd.Stdin (but in other direction, of course). Otherwise there's a risk of deadlock - the process is blocked waiting to write to stdout/stderr, and your program is blocked waiting for the process to finish.

Or, like @JimB said, just assign string buffers to cmd.Stdout and cmd.Stderr, they will be filled as the process runs.

func invoke(cmd *exec.Cmd) (stdout string, stderr string, err error) {
    stdoutbuf, stderrbuf := new(strings.Builder), new(strings.Builder)
    cmd.Stdout = stdoutbuf
    cmd.Stderr = stderrbuf
    err = cmd.Start()
    if err != nil {
        return
    }
    err = cmd.Wait()
    return stdoutbuf.String(), stderrbuf.String(), err
}

Live demo:

https://play.golang.org/p/hakSVNbqirB

user_4853430

I used the advice of @rusty, anyway the race line `runner.go:264 ' is

append(normalizeEncoding(stdoutbuf.String()), normalizeEncoding(stderrbuf.String()), false)

enter image description here

user_122705480

solution: create your own Writer wrapper

type lockedWriter struct {
    sync.RWMutex

    buf []byte
    w   io.Writer
}


func (w *lockedWriter) Write(b []byte) (n int, err error) {
    w.Lock()
    defer w.Unlock()

    w.buf = append(w.buf, b...)
    return w.w.Write(b)
}

func (w *lockedWriter) String() string {
    w.RLock()
    defer w.RUnlock()

    return string(w.buf)
}

usage

stdoutbuf, stderrbuf := &lockedWriter{w: new(strings.Builder)}, &lockedWriter{w: new(strings.Builder)}
cmd.Stdout = stdoutbuf
cmd.Stderr = stderrbuf
user_122705480
Related