Passing custom slice types by reference

I'm having trouble wrapping my head around how pointers, slices, and interfaces interact in Go. This is what I currently have coded up:

type Loader interface {
  Load(string, string)
}

type Foo struct {
  a, b string
}

type FooList []Foo

func (l FooList) Load(a, b string) {
  l = append(l, Foo{a, b})
  // l contains 1 Foo here
}

func Load(list Loader) {
  list.Load("1", "2")
  // list is still nil here
}

Given this setup, I then try to do the following:

var list FooList
Load(list)
fmt.Println(list)

However, list is always nil here. My FooList.Load function does add an element to the l slice, but that's as far as it gets. The list in Load continues to be nil. I think I should be able to just pass the reference to my slice around and have things append to it. I'm obviously missing something on how to get it to work though.

(Code in http://play.golang.org/p/uuRKjtxs9D)

If you intend your method to make changes, you probably want to use a pointer receiver.

// We also define a method Load on a FooList pointer receiver.
func (l *FooList) Load(a, b string) {
	*l = append(*l, Foo{a, b})
}

This has a consequence, though, that a FooList value won't itself satisfy the Loader interface.

var list FooList
Load(list)      // You should see a compiler error at this point.

A pointer to a FooList value, though, will satisfy the Loader interface.

var list FooList
Load(&list)

Complete code below:

package main

import "fmt"

/////////////////////////////
type Loader interface {
  Load(string, string)
}

func Load(list Loader) {
	list.Load("1", "2")
}
/////////////////////////////


type Foo struct {
  a, b string
}

// We define a FooList to be a slice of Foo.
type FooList []Foo

// We also define a method Load on a FooList pointer receiver.
func (l *FooList) Load(a, b string) {
	*l = append(*l, Foo{a, b})
}

// Given that we've defined the method with a pointer receiver, then a plain
// old FooList won't satisfy the Loader interface... but a FooList pointer will.

func main() {
	var list FooList
	Load(&list)
	fmt.Println(list)
}

I'm going to simplify the problem so it's easier to understand. What is being done there is very similar to this, which also does not work (you can run it here):

type myInt int

func (a myInt) increment() { a = a + 1 }
func increment(b myInt)    { b.increment() }

func main() {
	var c myInt = 42
	increment(c)
	fmt.Println(c) // => 42
}

The reason why this does not work is because Go passes parameters by value, as the documentation describes:

> In a function call, the function value and arguments are evaluated in the usual
> order. After they are evaluated, the parameters of the call are passed by value
> to the function and the called function begins execution.

In practice, this means that each of a, b, and c in the example above are pointing to different int variables, with a and b being copies of the initial c value.

To fix it, we must use pointers so that we can refer to the same area of memory (runnable here):

type myInt int

func (a *myInt) increment() { *a = *a + 1 }
func increment(b *myInt)    { b.increment() }

func main() {
	var c myInt = 42
	increment(&c)
	fmt.Println(c) // => 43
}

Now a and b are both pointers that contain the address of variable c, allowing their respective logic to change the original value. Note that the documented behavior still holds here: a and b are still copies of the original value, but the original value provided as a parameter to the increment function is the address of c.

The case for slices is no different than this. They are references, but the reference itself is provided as a parameter by value, so if you change the reference, the call site will not observe the change since they are different variables.

There's also a different way to make it work, though: implementing an API that resembles that of the standard append function. Again using the simpler example, we might implement increment without mutating the original value, and without using a pointer, by returning the changed value instead:

func increment(i int) int {  return i+1 }

You can see that technique used in a number of places in the standard library, such as the strconv.AppendInt function.

It's worth keeping a mental model of how Go's data structures are implemented. That usually makes it easier to reason about behaviour like this.

http://research.swtch.com/godata is a good introduction to the high-level view.

Go is pass-by-value. This is true for both parameters and receivers. If you need to assign to the slice value, you need to use a pointer.

> Then I read somewhere that you shouldn't pass pointers to slices since
> they are already references

This is not entirely true, and is missing part of the story.

When we say something is a "reference type", including a map type, a channel type, etc., we mean that it is actually a pointer to an internal data structure. For example, you can think of a map type as basically defined as:

// pseudocode
type map *SomeInternalMapStructure

So to modify the "contents" of the associative array, you don't need to assign to a map variable; you can pass a map variable by value and that function can change the contents of the associative array pointed to by the map variable, and it will be visible to the caller. This makes sense when you realize it's a pointer to some internal data structure. You would only assign to a map variable if you want to change which internal associative array you want it to point to.

However, a slice is more complicated. It is a pointer (to an internal array), plus the length and capacity, two integers. So basically, you can think of it as:

// pseudocode
type slice struct {
    underlyingArray uintptr
    length int
    capacity int
}

So it's not "just" a pointer. It is a pointer with respect to the underlying array. But the length and capacity are "value" parts of the slice type.

So if you just need to change an element of the slice, then yes, it acts like a reference type, in that you can pass the slice by value and have the function change an element and it's visible to the caller.

However, when you append() (which is what you're doing in the question), it's different. First, appending affects the length of the slice, and length is one of the direct parts of the slice, not behind a pointer. Second, appending may produce a different underlying array (if the capacity of the original underlying array is not enough, it allocates a new one); thus the array pointer part of the slice might also be changed. Thus it is necessary to change the slice value. (This is why append() returns something.) In this sense, it cannot be regarded as a reference type, because we are not just "changing what it points to"; we are changing the slice directly.