Golang pattern to kill multiple goroutines at once

I have two goroutines as shown in the snippet below. I want to synchronize them such that when one returns, the other one should also exit. What is best way in go to achieve this?

func main() {

  go func() {
    ...
    if err != nil {
      return
    }
  }()

  go func() {
    ...
    if err != nil {
      return
    }
  }()


}

I have simulated this scenario here https://play.golang.org/p/IqawStXt7rt and tried to solve it with a channel to signal a routine is done. This looks like there can be a write to closed channel resulting in a panic. What is the best way to solve this problem?

You can use context for communication between two go routines.
For example,

package main

import (
	"context"
	"sync"
)

func main() {

	ctx, cancel := context.WithCancel(context.Background())
	wg := sync.WaitGroup{}
	wg.Add(3)
	go func() {
		defer wg.Done()
		for {
			select {
			// msg from other goroutine finish
			case <-ctx.Done():
				// end
			}
		}
	}()

	go func() {
		defer wg.Done()
		for {
			select {
			// msg from other goroutine finish
			case <-ctx.Done():
				// end
			}
		}
	}()

	go func() {
		defer wg.Done()
		// your operation
		// call cancel when this goroutine ends
		cancel()
	}()
	wg.Wait()
}

Use close on a channel to signal completion. This allows multiple goroutines to check for completion by receiving on the channel.

Use one channel per goroutine to signal completion of the goroutine.

done1 := make(chan struct{}) // closed when goroutine 1 returns
done2 := make(chan struct{}) // closed when goroutine 2 returns

go func() {
	defer close(done1)

    timer1 := time.NewTicker(1 * time.Second)
    defer timer1.Stop()

    timer2 := time.NewTicker(2 * time.Second)
    defer timer2.Stop()

	for {
		select {
		case <-done2:
            // The other goroutine returned.
			fmt.Println("done func 1")
			return
		case <-timer1.C:
			fmt.Println("timer1 func 1")
		case <-timer2.C:
			fmt.Println("timer2 func 1")
			return
		}

	}
}()

go func() {
	defer close(done2)
	for {
		select {
		case <-done1:
            // The other goroutine returned.
			fmt.Println("done func 2")
			return
		default:
			time.Sleep(3 * time.Second)
			fmt.Println("sleep done from func 2")
			return
		}

	}
}()

fmt.Println("waiting for goroutines to complete")

// Wait for both goroutines to return. The order that
// we wait here does not matter. 
<-done1
<-done2

fmt.Println("all done")

Run it on the playground.

First separate the waiting on go-routines and the done channel.

Use a sync.WaitGroup to coordinate the goroutines.

func main() {
	wait := &sync.WaitGroup{}
	N := 3

	wait.Add(N)
	for i := 1; i <= N; i++ {
		go goFunc(wait, i, true)
	}

	wait.Wait()
	fmt.Println(`Exiting main`)
}

Each goroutine will look like this:

// code for the actual goroutine
func goFunc(wait *sync.WaitGroup, i int, closer bool) {
	defer wait.Done()
	defer fmt.Println(`Exiting `, i)

	T := time.Tick(time.Duration(100*i) * time.Millisecond)
	for {
		select {
		case <-T:
			fmt.Println(`Tick `, i)
			if closer {
				return
			}
		}
	}
}

(https://play.golang.org/p/mDO4P56lzBU)

Our main func is successfully waiting for the goroutines to exit before it exits. Each goroutine is closing itself, and we want a way to cancel all our goroutines at the same time.

We'll do this with a chan, and make use of this feature of receiving from channels:

QUOTE: A receive operation on a closed channel can always proceed immediately, yielding the element type's zero value after any previously sent values have been received.
(https://golang.org/ref/spec#Receive_operator)

We modify our goroutines to check for a CLOSE:

func goFunc(wait *sync.WaitGroup, i int, closer bool, CLOSE chan struct{}) {
	defer wait.Done()
	defer fmt.Println(`Exiting `, i)

	T := time.Tick(time.Duration(100*i) * time.Millisecond)
	for {
		select {
		case <-CLOSE:
			return
		case <-T:
			fmt.Println(`Tick `, i)
			if closer {
				close(CLOSE)
			}
		}
	}
}

and then we change our func main so that it passes the CLOSE channel through, and we'll set the closer variable so that only the last of our goroutines will trigger the close:

func main() {
	wait := &sync.WaitGroup{}
	N := 3
	CLOSE := make(chan struct{})

	// Launch the goroutines
	wait.Add(N)
	for i := 1; i <= N; i++ {
		go goFunc(wait, i, i == N, CLOSE)
	}

	// Wait for the goroutines to finish
	wait.Wait()
	fmt.Println(`Exiting main`)
}

(https://play.golang.org/p/E91CtRAHDp2)

Now it looks like everything is working.

But it isn't. Concurrency is hard. There's a bug lurking in this code, just waiting to bite you in production. Let's surface it.

Change our example so that every goroutine will close:

func main() {
	wait := &sync.WaitGroup{}
	N := 3
	CLOSE := make(chan struct{})

	// Launch the goroutines
	wait.Add(N)
	for i := 1; i <= N; i++ {
		go goFunc(wait, i, true /*** EVERY GOROUTINE WILL CLOSE ***/, CLOSE)
	}

	// Wait for the goroutines to finish
	wait.Wait()
	fmt.Println(`Exiting main`)
}

Change goroutine so that it takes a while before closing. We want two goroutines to be about to close at the same time:

// code for the actual goroutine
func goFunc(wait *sync.WaitGroup, i int, closer bool, CLOSE chan struct{}) {
	defer wait.Done()
	defer fmt.Println(`Exiting `, i)

	T := time.Tick(time.Duration(100*i) * time.Millisecond)
	for {
		select {
		case <-CLOSE:
			return
		case <-T:
			fmt.Println(`Tick `, i)
			if closer {
				/*** TAKE A WHILE BEFORE CLOSING ***/
				time.Sleep(time.Second)
				close(CLOSE)
			}
		}
	}
}

(https://play.golang.org/p/YHnbDpnJCks)

We crash with:

Tick  1
Tick  2
Tick  3
Exiting  1
Exiting  2
panic: close of closed channel

goroutine 7 [running]:
main.goFunc(0x40e020, 0x2, 0x68601, 0x430080)
	/tmp/sandbox558886627/prog.go:24 +0x2e0
created by main.main
	/tmp/sandbox558886627/prog.go:38 +0xc0

Program exited: status 2.

While a receive on a closed channel returns immediately, you cannot close a closed channel.

We need a little coordination. We can do this with a sync.Mutex and a bool to indicate whether we've closed the channel or not. Let's create a struct to do this:

type Close struct {
	C chan struct{}
	l sync.Mutex
	closed bool
}

func NewClose() *Close {
	return &Close {
		C: make(chan struct{}),
	}
}

func (c *Close) Close() {
	c.l.Lock()
	if (!c.closed) {
		c.closed=true
		close(c.C)
	}
	c.l.Unlock()
}

Rewrite our gofunc and our main to use our new Close struct, and we're good to go:
https://play.golang.org/p/eH3djHu8EXW

The problem with concurrency is that you always need to be wondering what would happen if another 'thread' was anywhere else in the code.

Your problem is that you want a single send on the DONE channel to be received by multiple listeners. You also need to consider whether a send on the done channel is received by your goroutines, or by your main func.

I suggest you rather separate the waiting on go-routines and the done channel.

import `sync`

// This code will wait for the two functions to complete before ending
func main {
   var wait sync.WaitGroup
   wait.Add(2)
   go func() {
     defer wait.Done()
   }()
   go g() {
     defer wait.Done()
   }()
   wait.Wait()
}

Now, how to manage the Done. Well, the solution is to use a sync.Cond and have each goroutine run its own goroutine to wait on the Cond. Here's an example:

package main

import (
	`fmt`
	`sync`
	`time`
)

// WaitForIt wraps a Cond and a Mutex for a simpler API:
// .WAIT() chan struct{} will return a channel that will be
//   signalled when the WaitForIt is done.
// .Done() will indicate that the WaitForIt is done.
type WaitForIt struct {
	L *sync.Mutex
	Cond *sync.Cond
}

func NewWaitForIt() *WaitForIt {
	l := &sync.Mutex{}
	c := sync.NewCond(l)
	return &WaitForIt{ l, c }
}

// WAIT returns a chan that will be signalled when
// the Cond is triggered.
func (w *WaitForIt) WAIT() chan struct{} {
	D := make(chan struct{})
	go func() {
		w.L.Lock()
		defer w.L.Unlock()
		w.Cond.Wait()
		D <- struct{}{}
		close(D)
	}()
	return D
}

// Done indicates that the Cond should be triggered.
func (w *WaitForIt) Done() {
	w.Cond.Broadcast()
}

// doneFunc launches the func f with a chan that will be signalled when the
// func should stop. It also handles WaitGroup synchronization
func doneFunc(wait *sync.WaitGroup, waitForIt *WaitForIt, f func(DONE chan struct{})) {
	defer wait.Done()
	f(waitForIt.WAIT())
}

func main() {
	// wait will coordinate all the goroutines at the level of main()
    // between themselves the waitForIt will do the coordination
	wait := &sync.WaitGroup{}
	// waitForIt indicates to the goroutines when they should shut
	waitForIt := NewWaitForIt()

    // goFunc generates each goroutine. Only the 3-second goroutine will 
    // shutdown all goroutines
	goFunc := func(seconds int) func(chan struct{}) {
		return func(DONE chan struct{}) {
            // this is the actual code of each goroutine
            // it makes a ticker for a number of seconds,
            // and prints the seconds after the ticker elapses,
            // or exits if DONE is triggered
			timer := time.NewTicker(time.Duration(seconds) * time.Second)
			defer timer.Stop()
			for {
				select {
				case <- DONE:
					return
				case <- timer.C:
					if (3==seconds) {
						waitForIt.Done()
                        // Don't shutdown here - we'll shutdown
                        // when our DONE is signalled
					}
				}
			}
		}
	}
	// launch 3 goroutines, each waiting on a shutdown signal
	for i:=1; i<=3; i++ {
		wait.Add(1)
		go doneFunc(wait, waitForIt, goFunc(i))
	}
	// wait for all the goroutines to complete, and we're done
	wait.Wait()
}

Here's your example implemented using WaitForIt: https://play.golang.org/p/llphW73G1xE
Note that I had to remove the Lock() call in WaitForIt.Done. Although the documentation says you're allowed to hold the lock, it was blocking your 2nd goroutine from completing.

package main

import (
	"fmt"
	"sync"
	"time"
)

func func1(done chan struct{}, wg *sync.WaitGroup) {
	defer wg.Done()
	timer1 := time.NewTicker(1 * time.Second)
	timer2 := time.NewTicker(2 * time.Second)
	for {
		select {
		case <-timer1.C:
			fmt.Println("timer1 func 1")
		case <-timer2.C:
			// Ask GC to sweep the tickers timer1, timer2
			// as goroutine should return
			timer1.Stop()
			timer2.Stop()

			fmt.Println("timer2 func 1")

			done <- struct{}{} // Signal the other goroutine to terminate

			fmt.Println("sent done from func 1")
			return
		case <-done:
			// Ask GC to sweep the tickers timer1, timer2
			// as goroutine should return
			timer1.Stop()
			timer2.Stop()

			fmt.Println("done func 1")
			return

		}

	}
}

func func2(done chan struct{}, wg *sync.WaitGroup) {
	defer wg.Done()
	timer3 := time.NewTicker(3 * time.Second)
	for {
		select {
		case <-timer3.C:
			// Ask GC to sweep the tickers timer3
			// as goroutine should return
			timer3.Stop()

			fmt.Println("timer3 func 2")

			done <- struct{}{} // Signal the other goroutine to terminate

			fmt.Println("sent done from func 2")
			return
		case <-done:
			// Ask GC to sweep the tickers timer3
			// as goroutine should return
			timer3.Stop()
			fmt.Println("done func 2")
			return
		}

	}
}

func main() {
	// Chan used for signalling between goroutines
	done := make(chan struct{})

	// WaitGroup
	wg := sync.WaitGroup{}

	wg.Add(2)

	// Spawn the goroutine for func1
	go func1(done, &wg)
	// Spawn the goroutine for func2
	go func2(done, &wg)

	fmt.Println("starting sleep")

	// Wait for the goroutines
	wg.Wait()

	// Wait for 15 seconds
	// If not required, please remove
	// the lines below
	time.Sleep(15 * time.Second)
	fmt.Println("waited 15 seconds")

}

You could use channel closing pattern to wait within multiple go routines.

package main

import (
	"os"
	"os/signal"
	"sync"
	"syscall"
)


type RunGroup struct {
  sync.WaitGroup
}

// Run handles wait group state
func (runGroup *RunGroup) Run(f func()) {
  runGroup.Add(1)
  go func() {
    f()
    runGroup.Done()
  }()
}

func doStuff(done <-chan any, id int) {
  println("Doing something", id)
  <-done
  println("DONE", id)
}

func main() {
  // Done channel
  done := make(chan any)
  // Setup Shutdown listeners
  sigChan := make(chan os.Signal, 1)
	signal.Notify(sigChan, syscall.SIGTERM)
	signal.Notify(sigChan, syscall.SIGINT)
  go func() {
    rawSig := <-sigChan
    sig := rawSig.String()
    println("Caught signal, shutting down.", sig)
    close(done)
  }()

  runGroup := RunGroup{}
  // Do some stuff
  runGroup.Run(func () {
    doStuff(done, 1)
  })
  runGroup.Run(func () {
    doStuff(done, 2)
  })
  runGroup.Run(func () {
    doStuff(done, 3)
  })

  // Wait mainthread until interrupt
  runGroup.Wait()
}

Output

go run ./main.go
Doing something 3
Doing something 2
Doing something 1
^CCaught signal, shutting down. interrupt
DONE 3
DONE 1
DONE 2