Captured Closure (for Loop Variable) in Go

Shouldn't Go compiler capture for...range loop variables as a locally assigned closure variable?

Long Version:

This caused me some confusion in C# too and I was trying to understand it; that why it is fixed in C# 5.0 foreach (reason: the loop variable can not change inside the body of loop) and the reasoning for not fixing it in C# for loops (reason: the loop variable can change inside the body of loop).

Now (to me) for...range loops in Go seems pretty much like foreach loops in C#, but despite the fact that we can not alter those variables (like k and v in for k, v := range m { ... }); still we have to copy them to some local closures first, for them to behave as expected.

What is the reasoning behind this? (I suspect it's because Go treats any for loop the same way; but I'm not sure).

Here is some code to examine described behavior:

func main() {
lab1() // captured closure is not what is expected
fmt.Println(" ")
lab2() // captured closure is not what is expected
fmt.Println(" ")
lab3() // captured closure behaves ok
fmt.Println(" ")
}
func lab3() {
m := make(map[int32]int32)
var i int32
for i = 1; i <= 10; i++ {
m[i] = i
}
l := [](func() (int32, int32)){}
for k, v := range m {
kLocal, vLocal := k, v // (C) captures just the right values assigned to k and v
l = append(l, func() (int32, int32) {
return kLocal, vLocal
})
}
for _, x := range l {
k, v := x()
fmt.Println(k, v)
}
}
func lab2() {
m := make(map[int32]int32)
var i int32
for i = 1; i <= 10; i++ {
m[i] = i
}
l := [](func() (int32, int32)){}
for k, v := range m {
l = append(l, func() (int32, int32) {
kLocal, vLocal := k, v // (B) captures just the last values assigned to k and v from the range
return kLocal, vLocal
})
}
for _, x := range l {
k, v := x()
fmt.Println(k, v)
}
}
func lab1() {
m := make(map[int32]int32)
var i int32
for i = 1; i <= 10; i++ {
m[i] = i
}
l := [](func() (int32, int32)){}
for k, v := range m {
l = append(l, func() (int32, int32) { return k, v }) // (A) captures just the last values assigned to k and v from the range
}
for _, x := range l {
k, v := x()
fmt.Println(k, v)
}
}

As it is shown in lab1, at the comment // (A) we get just the last values from the range; the output is like printing 9,9 ten times instead of showing expected result like 1,1, 2,2, ... (and of-course maps are not necessarily sorted in Go so we may see 3,3 ten times as the last pair of values; instead of 10,10 ten times as the last pair of values). The same goes for code at comment // (B) at lab2, which was expected because we are trying to capture outer variables inside the inner scope (I put this one too just to try that). In lab3 at code at comment // (C) everything works fine and you will see ten pairs of numbers there like 1,1, 2,2, ....

I was trying to use closure+function as a replacement for tuples in Go.

Do you want the closure over the variable or the value? For example,

package main
import "fmt"
func VariableLoop() {
f := make([]func(), 3)
for i := 0; i < 3; i++ {
// closure over variable i
f[i] = func() {
fmt.Println(i)
}
}
fmt.Println("VariableLoop")
for _, f := range f {
f()
}
}
func ValueLoop() {
f := make([]func(), 3)
for i := 0; i < 3; i++ {
i := i
// closure over value of i
f[i] = func() {
fmt.Println(i)
}
}
fmt.Println("ValueLoop")
for _, f := range f {
f()
}
}
func VariableRange() {
f := make([]func(), 3)
for i := range f {
// closure over variable i
f[i] = func() {
fmt.Println(i)
}
}
fmt.Println("VariableRange")
for _, f := range f {
f()
}
}
func ValueRange() {
f := make([]func(), 3)
for i := range f {
i := i
// closure over value of i
f[i] = func() {
fmt.Println(i)
}
}
fmt.Println("ValueRange")
for _, f := range f {
f()
}
}
func main() {
VariableLoop()
ValueLoop()
VariableRange()
ValueRange()
}

Output:

<pre>
VariableLoop
3
3
3
ValueLoop
0
1
2
VariableRange
2
2
2
ValueRange
0
1
2
</pre>

References:

> The Go Programming Language Specification
>
> Function literals
>
> Function literals are closures: they may refer to variables defined in
> a surrounding function. Those variables are then shared between the
> surrounding function and the function literal, and they survive as
> long as they are accessible.
>
> Go FAQ: What happens with closures running as goroutines?
>
> To bind the current value of v to each closure as it is launched, one
> must modify the inner loop to create a new variable each iteration.
> One way is to pass the variable as an argument to the closure.
>
> Even easier is just to create a new variable, using a declaration
> style that may seem odd but works fine in Go.

The Common Mistake / Using reference to loop iterator variable wiki page document your process (copy the loop variable into a new variable)

But the proposal # 60078 spec: less error-prone loop variable scoping, May 2023, seek to address that.

> Changing the loop semantics would in essence insert this kind of v := v statement for every for loop variable declared with :=. It would fix this loop and many others to do what the author clearly intends.
>
> the new loop semantics would only apply in Go modules that have opted in to the release with the new loops. If that was Go 1.22, then only packages in a module with a go.mod that says go 1.22 would get the new loop semantics.

You can see the design being considered here:

> This proposal is about changing for loop variable scoping semantics, so that loop variables are per-iteration instead of per-loop.

The For statements with for clause would include:

> The init statement may be a short variable declaration (:=), but the post statement must not.
Each iteration has its own separate declared variable (or variables).
>
> - The variable used by the first iteration is declared by the init statement.
> - The variable used by each subsequent iteration is declared implicitly before executing the post statement and initialized to the value of the previous iteration's variable at that moment.
>
> go
> var prints []func()
> for i := 0; i < 3; i++ {
> prints = append(prints, func() { println(i) })
> }
> for _, p := range prints {
> p()
> }
>
> // Output:
> // 0
> // 1
> // 2
>

Since Go 1.21 is released, Russ Cox adds

> For anyone new to this issue and exploring what the change would mean, the Go playground now lets you experiment with the new semantics.
>
> To do that, use Go 1.21 and add // GOEXPERIMENT=loopvar at the top of your program.
>
> For example try https://go.dev/play/p/lDFLrPOcdz3 and then try deleting the comment.

// GOEXPERIMENT=loopvar

package main

func main() {
	var prints []func()
	for i := range make([]int, 5) {
		prints = append(prints, func() { println(i) })
	}
	for _, p := range prints {
		p()
	}
}

Output (with comment):

0
1
2
3
4

Output (without comment):

4
4
4
4
4