Modify an array that is the value of a map

If I have a map whose value is an array, how can I modify one element of the array?

Something like this:

m := make(map[string][4]int)
m["a"]=[...]int{0,1,2,3}
m["a"][2]=10

It won't compile: prog.go:8: cannot assign to m["a"][2]

I could copy the variable to an array, modify it and then copying it back to the map, but it seems to be very slow, specially for large arrays.

// what I like to avoid.
m := make(map[string][4]int)
m["a"] = [...]int{0, 1, 2, 3}
b := m["a"]
b[2] = 10
m["a"] = b

Any idea?