Golang: appending slices with or w/o allocation

Go's append() function only allocates new slice data, when the capacity of the given slice is not sufficient (see also: https://stackoverflow.com/a/28143457/802833). This can lead to unexpected behavior (at least for me as a golang newbie):

package main

import (
    "fmt"
)

func main() {

    a1 := make([][]int, 3)
    a2 := make([][]int, 3)
    b := [][]int{{1, 1, 1}, {2, 2, 2}, {3, 3, 3}}
    common1 := make([]int, 0)
    common2 := make([]int, 0, 12) // provide sufficient capacity
    common1 = append(common1, []int{10, 20}...)
    common2 = append(common2, []int{10, 20}...)

    idx := 0
    for _, k := range b {
        a1[idx] = append(common1, k...) // new slice is allocated
        a2[idx] = append(common2, k...) // no allocation
        idx++
    }

    fmt.Println(a1)
    fmt.Println(a2) // surprise!!!
}

output:

> [[10 20 1 1 1] [10 20 2 2 2] [10 20 3 3 3]]
>
> [[10 20 3 3 3] [10 20 3 3 3] [10 20 3 3 3]]

https://play.golang.org/p/8PEqFxAsMt

So, what ist the (idomatic) way in Go to force allocation of new slice data or more precisely to make sure that the slice argument to append() remains unchanged?

You might maintain a wrong idea of how slices work in Go.

When you append elements to a slice, the call to append() returns a new slice. If reallocation did not happen, both slice values — the one you called append() on and the one it returned back — share the same backing array but they will have different lengths; observe:

package main

import "fmt"

func main() {
    a := make([]int, 0, 10)
    b := append(a, 1, 2, 3)
    c := append(a, 4, 3, 2)
    fmt.Printf("a=%#v\nb=%#v\nc=%#v\n", a, b, c)
}

outputs:

a=[]int{}
b=[]int{4, 3, 2}
c=[]int{4, 3, 2}

So, len(a) == 0, len(b) == 3, len(c) == 3, and the second call to append() owerwrote what the first one did because all the slices share the same underlying array.

As to reallocation of the backing array, the spec is clear:

> If the capacity of s is not large enough to fit the additional values, append allocates a new, sufficiently large underlying array that fits both the existing slice elements and the additional values. Otherwise, append re-uses the underlying array.

From this, it follows that:

1. append() never copies the underlying storage if the capacity of the slice being appeneded to is sufficient.
2. If there's not enough capacity, the array will be reallocated.

That is, given a slice s to which you want to append N elements, the reallocation won't be done iff cap(s) - len(s) ≥ N.

Hence I suspect your problem is not about unexpected reallocation results but rather about the concept of slices as implemented in Go. The code idea to absorb is that append() returns the resulting slice value, which you're supposed to be using after the call unless you fully understand the repercussions.

I recommend starting with this to fully understand them.

Thanx for your feedback.

So the solution to gain control of the memory allocation is to do it explicitely (which remembers me that Go is a more a system language than other (scripting) langs):

package main

import (
    "fmt"
)

func main() {

    a1 := make([][]int, 3)
    a2 := make([][]int, 3)
    b := [][]int{{1, 1, 1}, {2, 2, 2}, {3, 3, 3}}
    common1 := make([]int, 0)
    common2 := make([]int, 0, 12) // provide sufficient capacity
    common1 = append(common1, []int{10, 20}...)
    common2 = append(common2, []int{10, 20}...)

    idx := 0
    for _, k := range b {
        a1[idx] = append(common1, k...) // new slice is allocated
        
        a2[idx] = make([]int, len(common2), len(common2)+len(k))
        copy(a2[idx], common2)      // copy & append could probably be
        a2[idx] = append(a2[idx], k...) // combined into a single copy step
        
        idx++
    }

    fmt.Println(a1)
    fmt.Println(a2)
}

output:
> [[10 20 1 1 1] [10 20 2 2 2] [10 20 3 3 3]]
>
> [[10 20 1 1 1] [10 20 2 2 2] [10 20 3 3 3]]

https://play.golang.org/p/Id_wSZwb84