How to check if []byte is all zeros in go

user_1208557 0
Viewed 13000

Is there a way to check if a byte slice is empty or 0 without checking each element or using reflect?

theByteVar := make([]byte, 128)

if "theByteVar is empty or zeroes" {
   doSomething()
}

One solution which seems weird that I found was to keep an empty byte array for comparison.

theByteVar := make([]byte, 128)
emptyByteVar := make([]byte, 128)

// fill with anything
theByteVar[1] = 2

if reflect.DeepEqual(theByteVar,empty) == false {
   doSomething(theByteVar)
}

For sure there must be a better/quicker solution.

Thanks

UPDATE did some comparison for 1000 loops and the reflect way is the worst by far...

Equal Loops: 1000 in true in 19.197µs
Contains Loops: 1000 in true in 34.507µs
AllZero Loops: 1000 in true in 117.275µs
Reflect Loops: 1000 in true in 14.616277ms
go slice
4 Answers

Another solution borrows an idea from C. It could be achieved by using the unsafe package in Go.

The idea is simple, instead of checking each byte from []byte, we can check the value of byte[i:i+8], which is a uint64 value, in each steps. By doing this we can check 8 bytes instead of checking only one byte in each iteration.

Below codes are not best practice but only show the idea.

const (
    len8 int = 0xFFFFFFF8
)

func IsAllBytesZero(data []byte) bool {
    n := len(data)

    // Magic to get largest length which could be divided by 8. 
    nlen8 := n & len8
    i := 0

    for ; i < nlen8; i += 8 {
        b := *(*uint64)(unsafe.Pointer(uintptr(unsafe.Pointer(&data[0])) + 8*uintptr(i)))
        if b != 0 {
            return false
        }
    }

    for ; i < n; i++ {
        if data[i] != 0 {
            return false
        }
    }

    return true
}

Benchmark

Testcases:

Only test for worst cases (all elements are zero)

Methods:

  • IsAllBytesZero: unsafe package solution
  • NaiveCheckAllBytesAreZero: a loop to iterate the whole byte array and check it.
  • CompareAllBytesWithFixedEmptyArray: using bytes.Compare solution with pre-allocated fixed size empty byte array.
  • CompareAllBytesWithDynamicEmptyArray: using bytes.Compare solution without pre-allocated fixed size empty byte array.

Results

BenchmarkIsAllBytesZero10-8                                 254072224            4.68 ns/op
BenchmarkIsAllBytesZero100-8                                132266841            9.09 ns/op
BenchmarkIsAllBytesZero1000-8                               19989015            55.6 ns/op
BenchmarkIsAllBytesZero10000-8                               2344436           507 ns/op
BenchmarkIsAllBytesZero100000-8                              1727826           679 ns/op
BenchmarkNaiveCheckAllBytesAreZero10-8                      234153582            5.15 ns/op
BenchmarkNaiveCheckAllBytesAreZero100-8                     30038720            38.2 ns/op
BenchmarkNaiveCheckAllBytesAreZero1000-8                     4300405           291 ns/op
BenchmarkNaiveCheckAllBytesAreZero10000-8                     407547          2666 ns/op
BenchmarkNaiveCheckAllBytesAreZero100000-8                     43382         27265 ns/op
BenchmarkCompareAllBytesWithFixedEmptyArray10-8             415171356            2.71 ns/op
BenchmarkCompareAllBytesWithFixedEmptyArray100-8            218871330            5.51 ns/op
BenchmarkCompareAllBytesWithFixedEmptyArray1000-8           56569351            21.0 ns/op
BenchmarkCompareAllBytesWithFixedEmptyArray10000-8           6592575           177 ns/op
BenchmarkCompareAllBytesWithFixedEmptyArray100000-8           567784          2104 ns/op
BenchmarkCompareAllBytesWithDynamicEmptyArray10-8           64215448            19.8 ns/op
BenchmarkCompareAllBytesWithDynamicEmptyArray100-8          32875428            35.4 ns/op
BenchmarkCompareAllBytesWithDynamicEmptyArray1000-8          8580890           140 ns/op
BenchmarkCompareAllBytesWithDynamicEmptyArray10000-8         1277070           938 ns/op
BenchmarkCompareAllBytesWithDynamicEmptyArray100000-8         121256         10355 ns/op

Summary

Assumed that we're talking about the condition in sparse zero byte array. According to the benchmark, if performance is an issue, the naive check solution would be a bad idea. And, if you don't want to use unsafe package in your project, then consider using bytes.Compare solution with pre-allocated empty array as an alternative.

An interesting point could be pointed out is that the performance comes from unsafe package varies a lot, but it basically outperform all other solution mentioned above. I think it was relevant to the CPU cache mechanism.

user_75404460

I think it is better (faster) if binary or is used instead of if condition inside loop:

func isZero(bytes []byte) bool {
    b := byte(0)
    for _, s := range bytes {
        b |= s
    }
    return b == 0
}

One can optimize this even more by using idea with uint64 mentioned in previous answers

user_14608300
Related