Why does golang strings.Builder implements String() like this?

user_9701267 0
Viewed 1808

The implementation is

// String returns the accumulated string.
func (b *Builder) String() string {
    return *(*string)(unsafe.Pointer(&b.buf))
}

According to my test, converting []byte to string uses "copy on write", or the compiler generates the deep copy instructions if either one is changing the insided slice:

{
        a := []byte{'a'}
        s1 := string(a)
        a[0] = 'b'
        fmt.Println(s1) // a
    }

    {
        a := "a"
        b := []byte(a)
        b[0] = 'b'
        fmt.Println(a) // a
    }

So what happens if it's implemented as below?

// String returns the accumulated string.
func (b *Builder) String() string {
    return string(b.buf)
}
go
2 Answers

Given large enough strings the type conversion requires memory allocations whereas the conversion using the unsafe package does not:

package main

import (
    "testing"
    "unsafe"
)

func BenchmarkConversion(b *testing.B) {
    buf := make([]byte, 16<<10)
    b.ResetTimer()

    for i := 0; i < b.N; i++ {
        var _ string = string(buf)
    }
}

func BenchmarkUnsafe(b *testing.B) {
    buf := make([]byte, 16<<10)
    b.ResetTimer()

    for i := 0; i < b.N; i++ {
        var _ string = *(*string)(unsafe.Pointer(&buf))
    }
}

$ go test -bench=. -benchmem
goos: linux
goarch: amd64
BenchmarkConversion-8            307087      3897 ns/op     16384 B/op     1 allocs/op
BenchmarkUnsafe-8            1000000000     0.299 ns/op         0 B/op     0 allocs/op
PASS
ok      _/tmp/tmp.KECLzZwkUn    1.579s
user_6033160

You can view the discussion on the changelist that introduced the strings.Builder api here: https://go-review.googlesource.com/c/go/+/74931/4/src/strings/builder.go#30

As you'd expect, it's a discussion of API mechanics, correctness, and efficiency.

If you replace the code with string(b.buf), you'll cause a copy of the built string. It may be that the compiler optimizes away the copy in simple cases of converting a byte slice to a string, but it's very unlikely that the compiler can do that here in general (because it would require a proof that the buffer inside the string builder is never used again).

Note that the (standard-library) code looks dangerous, because if you write this:

var b strings.Builder
b.WriteString("hello world")
c := b.String()
b.WriteString("a")
d := b.String()

then c and d will end up pointing to the same memory. But that's fine, because strings contain the length of their buffer. And there's no way to mutate a string, because even though in theory the memory backing the string is accessible via the buf in the strings.Builder, the only apis provided append to the backed memory.

user_14007930
Related