golang slice allocation performance

user_6092351 0
Viewed 3701

I stumbled upon an interesting thing while checking performance of memory allocation in GO.

package main

import (
      "fmt"
      "time"
    )

func main(){
   const alloc int = 65536
   now := time.Now()
   loop := 50000
   for i := 0; i<loop;i++{
      sl := make([]byte, alloc)
      i += len(sl) * 0
   }
   elpased := time.Since(now)
   fmt.Printf("took %s to allocate %d bytes %d times", elpased, alloc, loop) 
}

I am running this on a Core-i7 2600 with go version 1.6 64bit (also same results on 32bit) and 16GB of RAM (on WINDOWS 10) so when alloc is 65536 (exactly 64K) it runs for 30 seconds (!!!!). When alloc is 65535 it takes ~200ms. Can someone explain this to me please? I tried the same code at home with my core i7-920 @ 3.8GHZ but it didn't show same results (both took around 200ms). Anyone has an idea what's going on?

go memory performance
3 Answers

Note/Update 2021: as Tapir Liui notes in Go101 with this tweet:

As of Go 1.17, Go runtime will allocate the elements of slice x on stack if the compiler proves they are only used in the current goroutine and N <= 64KB:

var x = make([]byte, N)

And Go runtime will allocate the array y on stack if the compiler proves it is only used in the current goroutine and N <= 10MB:

var y [N]byte

Then how to allocated (the elements of) a slice which size is larger than 64KB but not larger than 10MB on stack (and the slice is only used in one goroutine)?

Just use the following way:

var y [N]byte
var x = y[:]

Considering stack allocation is faster than heap allocation, that would have a direct effect on your test, for alloc equals to 65536 and more.

Tapir adds:

In fact, we could allocate slices with arbitrary sum element sizes on stack.

const N = 500 * 1024 * 1024 // 500M
var v byte = 123

func createSlice() byte {
 var s = []byte{N: 0}
 for i := range s { s[i] = v }
 return s[v]
}

Changing 500 to 512 make program crash.

VonC_63090

the reason is very simple.

const alloc int = 65535

0x0000 00000 (example.go:8) TEXT "".main(SB), ABIInternal, $65784-0

const alloc int = 65536

0x0000 00000 (example.go:8) TEXT "".main(SB), ABIInternal, $248-0

the difference is where the slice are created.

user_51598910
Related