Meaning of "don't move data over channels, move ownership of data over channels"

I'm learning that Golang channels are actually slower than many alternatives provided by the language. Of course, they are really easy to grasp but because they are a high level structure, they come with some overhead.

Reading some articles about it, I found someone benchmarking the channels here. He basically says that the channels can transfer 10 MB/s, which of course must be dependant on his hardware. He then says something that I haven't completely understood:

> If you just want to move data quickly using channels then moving it 1
> byte at a time is not sensible. What you really do with a channel is
> move ownership of the data, in which case the data rate can be
> effectively infinite, depending on the size of data block you
> transfer.

I've seen this "move ownership of data" in several places but I haven't seen a solid example illustrating how to do it instead of moving the data itself.

I wanted to see an example in order to understand this best practice.

Moving data over a channel:

c := make(chan [1000]int)

// spawn some goroutines that read from this channel

var data [1000]int
// populate the data

// write data to the channel
c <- data

The potential problem here, as you mentioned, is that you're moving a lot of data, so you might be doing an excessive amount of memory copying.

You could prevent that by sending a reference type, such as a pointer or slice over the channel:

c := make(chan []int)

// spawn some goroutines that read from this channel

var data [1000]int
// populate the data

// write a reference to data to the channel
c <- data[:]

So we just did the exact same data transfer, but reduced the memory copying, right? Well, here's a potential problem: You sent over the channel a reference to data, but that data value continues to be accessible in the current scope, even after the send:

// write a reference to data to the channel
c <- data[:]

// start messing with data
data[0] = 999
data[1] = 1234
...

This code might have just introduced a potential data race, because whoever read that slice from the channel might be working on it at the same time as you start modifying it.

The idea of passing ownership is that after you give out a reference to something, you are also conceding ownership of that thing, and will no use it. So long as we don't use data after giving out the reference (sending the slice on the channel), then we have properly passed ownership.

This problem is an extension of the general problem of shared state. Unlike, Rust, for example, Go doesn't have language constructs to properly control shared state. In order to reduce the chances of these errors, you could apply some strategies:

• Avoid passing references on channels: In the above example, the problem occurred once we started passing the data by reference, with a slice. This was only done to reduce the amount of memory coping done. Unless there was a pragmatic reason to do this optimization (a worthwhile performance difference was measured), it could be avoided entirely. Still, though, there are some data types in Go that are inherently a reference (e.g., maps and slices). If these types must be passed on a channel, then other strategies can be used.
• Separate the data creation logic into functions: In the example above, we could refactor the code:

func sendData(c chan []int) {
    var data [1000]int
    // populate the data

    // write a reference to data to the channel
    c <- data[:]
}

c := make(chan []int)

// spawn some goroutines that read from this channel

// send some data
sendData(c)

The possibility of incorrectly using data still exists, but now it's isolated to a small function with a clear intent. In theory, the isolation should make the code easier to understand, more obvious what the correct use of data is, and fewer changes would have potential interaction with it.

• Don't mix data pipelines with persistent state: By data pipeline, I mean two or more concurrent routines, between which data flows via channels. Expanding on the previous point, make the creation of owned references as close as possible to where they enter the data pipeline. Make space between where a goroutine receives data and where it sends it again or uses it, as tight as possible. In the general rules of ownership, you can only transfer ownership of something when you presently have full ownership of it. Due to this rule, you should avoid as much as possible, sending any reference on a channel that you didn't just create the referenced data immediately before sending. If you have a reference to any persistent or global state, it becomes much harder to ensure that ownership is respected.

By keeping the creation of the reference and the transfer of ownership in an isolated, global function, it should be harder to make errors. Then the only ways to violate the ownership rule are to:

1. Leak the reference to global state

• Try to eliminate global variables and global state

2. Leak the reference to a reference type parameter's state

• Don't take any reference type parameters in data sending functions

3. Modify the reference data after sending the reference

• Put the send operation at the very end of the function. If necessary, you could put the send inside a defered call.

There's no perfect solution to eliminate all shared state issues (even in Rust they sometimes exist in practice), but I hope these strategies will help you think about how to tackle this problem.

Hymns For Disco's answer is good, but I find writing good short answers an interesting challenge sometimes, and I think I have an analogy that will help here.

Imagine your data as occupying warehouses, each one a large city block in size. You have a thousand warehouses, scattered across many countries and cities.

You have five highly-skilled technicians, each of whom can do one thing really well. You need all the technicians to operate on all the data. Unfortunately, each technician hates the other four and will do no work (and maybe even try to kill the others) if any of them are present in the warehouse.

One way to deal with this is to build five extra warehouses, and put each of the five technicians in each of the new five warehouses. You can then ship the entire contents of each of the 1000 warehouses to the various spares, one at a time, and then move the contents back once each technician has finished with it; and you can optimize the warehouse-content moves somewhat by, perhaps, moving content to work-warehouse #1, then to #2, then to #3, etc., and only moving it back to its original warehouse after it's ready to leave #5. But obviously this requires a whole lot of shipping and logistics and takes huge amounts of time and money for all this bulk movement. It will be years, and lots of money, before everything is done, even if each technician can completely deal with a whole warehouse in just one day.

Alternatively, you can ship the five technicians around. Send them to warehouses (WHs) 1-5. When tech#1 is done with WH#1, move him to WH#2 unless tech#2 is still there; move him to WH#6 if that's the next free one instead.

We're moving the small and light "person who does the work" around, not the big and heavy "things that occupy the space where the person works". The overall cost is much lower. We do have to take care not to accidentally let the technicians encounter each other though.

Note, too, that this fancy solution—of being careful about who has access to which data at what time—doesn't help if the data themselves are small and light and easy to move around. In the small-data case we might as well move the data, instead of the workers.