unexpected slice append behaviour

I encountered weird behaviour in go code today: when I append elements to slice in loop and then try to create new slices based on the result of the loop, last append overrides slices from previous appends.

In this particular example it means that sliceFromLoop j,g and h slice's last element are not 100,101 and 102 respectively, but...always 102!

Second example - sliceFromLiteral behaves as expected.

package main

import "fmt"

func create(iterations int) []int {
	a := make([]int, 0)
	for i := 0; i < iterations; i++ {
		a = append(a, i)
	}
	return a
}

func main() {
	sliceFromLoop()
	sliceFromLiteral()

}

func sliceFromLoop() {
	fmt.Printf("** NOT working as expected: **\n\n")
	i := create(11)
	fmt.Println("initial slice: ", i)
	j := append(i, 100)
	g := append(i, 101)
	h := append(i, 102)
	fmt.Printf("i: %v\nj: %v\ng: %v\nh:%v\n", i, j, g, h)
}

func sliceFromLiteral() {
	fmt.Printf("\n\n** working as expected: **\n")
	i := []int{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
	fmt.Println("initial slice: ", i)
	j := append(i, 100)
	g := append(i, 101)
	h := append(i, 102)
	fmt.Printf("i: %v\nj: %v\ng: %v\nh:%v\n", i, j, g, h)
}

link to play.golang:
https://play.golang.org/p/INADVS3Ats

After some reading, digging and experimenting I found that this problem is originated in slices referencing the same underlaying array values and can be solved by copying slice to new one before appending anything, however it looks quite... hesitantly.

What's the idomatic way for creating many new slices based on old ones and not worrying about changing values of old slices?

Don't assign append to anything other than itself.

As you mention in the question, the confusion is due to the fact that append both changes the underlying array and returns a new slice (since the length might be changed). You'd imagine that it copies that backing array, but it doesn't, it just allocates a new slice object that points at it. Since i never changes, all those appends keep changing the value of backingArray[12] to a different number.

Contrast this to appending to an array, which allocates a new literal array every time.

So yes, you need to copy the slice before you can work on it.

func makeFromSlice(sl []int) []int {
    result := make([]int, len(sl))
    copy(result, sl)
    return result
}

func main() {
    i := make([]int, 0)
    for ii:=0; ii<11; ii++ {
        i = append(i, ii)
    }
    j := append(makeFromSlice(i), 100)  // works fine
}

The slice literal behavior is explained because a new array is allocated if the append would exceed the cap of the backing array. This has nothing to do with slice literals and everything to do with the internals of how exceeding the cap works.

a := []int{1,2,3,4,5,6,7}
fmt.Printf("len(a) %d, cap(a) %d\n", a, len(a), cap(a))
// len(a) 7, cap(a) 7

b := make([]int, 0)
for i:=1; i<8, i++ {
    b = append(b, i)
}  // b := []int{1,2,3,4,5,6,7}
// len(b) 7, cap(b) 8

b = append(b, 1)  // any number, just so it hits cap

i := append(b, 100)
j := append(b, 101)
k := append(b, 102)  // these work as expected now

If you need a copy of a slice, there's no other way to do it other than, copying the slice. You should almost never assign the result of append to a variable other than the first argument of append. It leads to hard to find bugs, and will behave differently depending on whether the slice has the required capacity or not.

This isn't a commonly needed pattern, but as with all things of this nature if you need to repeate a few lines of code multiple times, then you can use a small helper function:

func copyAndAppend(i []int, vals ...int) []int {
	j := make([]int, len(i), len(i)+len(vals))
	copy(j, i)
	return append(j, vals...)
}

https://play.golang.org/p/J99_xEbaWo

There is also a little bit simpler way to implement copyAndAppend function:

func copyAndAppend(source []string, items ...string) []string {
    l := len(source)
    return append(source[:l:l], items...)
}

Here we just make sure that source has no available capacity and so copying is forced.