Go program exits before goroutine workers complete

I'm having an issue understanding how to properly block and close channels. I'm starting some arbitrary number of workers and I've discovered that my main function either exits before the workers have completed or hangs due to unclosed channel. I need a better way of blocking while the workers read the channel without main exiting and then gracefully close the channel to end the loop when it's done. Any attempt I've made ends in a deadlock.

I've tried a few things including using a wait group but the issue persists. I noticed that by adding time.Sleep, the program works as expected, but commenting it out results in no work done.

time.Sleep(time.Duration(10 * time.Second))

Here's a runable example https://go.dev/play/p/QHqNj-AJQBI with Sleep left in. And here's the broken code with the Sleep timeout commented out.

package main

import (
	"fmt"
	"sync"
	"time"
)

// some complicated work
func do(num int, ch chan<- int) {
	time.Sleep(time.Duration(500 * time.Millisecond))
	ch <- num
}

func main() {

	results := make(chan int)

	// for some number of required complicated work
	for i := 0; i < 53; i++ {
		go do(i, results)
	}

	var wg sync.WaitGroup

	// start 3 workers which can process results
	for i := 0; i < 3; i++ {
		wg.Add(1)
		go func(id int) {
			defer wg.Done()
			worker(id, results)
		}(i)
	}

	// handle closing the channel when all workers complete
	go func() {
		wg.Wait()
		close(results)
	}()

	//time.Sleep(time.Duration(10 * time.Second))

	fmt.Println("donezo")
}

// process the results of do() in a meaningful way
func worker(id int, ch <-chan int) {
	fmt.Println("starting worker", id)

	for i := range ch {
		fmt.Println("channel val:", i)
	}
}

I've also tried moving the defer wg.Done() to inside the worker() func but it's the same issue and doesn't work without sleep.

// process the results of do() in a meaningful way
func worker(wg *sync.WaitGroup, id int, ch <-chan int) {
	fmt.Println("starting worker", id)

	defer wg.Done()

	for i := range ch {
		fmt.Println("channel val:", i)
	}
}

Have I chosen the wrong paradigm or am I just using it wrong?

Use this code:

// Start the workers first. This allows you to feed
// work to the workers without using goroutines.
var wg sync.WaitGroup
for i := 0; i < 3; i++ {
wg.Add(1)
go func(id int) {
defer wg.Done()
worker(id, results)
}(i)
}
// Feed work to the workers.
for i := 0; i < 53; i++ {
do(i, results)
}
// Close channel to indicate no more work. 
// This must come after all work is fed in.
close(results)
// Wait for workers to complete.
wg.Wait()

I originally asked “Is there a small tweak I can make to my code to make this work or do I have to rethink the problem?” The answer I’ve discovered is, yes, there is a small tweak.

I had to learn an interesting fundamental concept about channels: You can read from a closed channel i.e. drain a channel. As mentioned the range in my original example never terminates because I couldn’t find a good place to close the channel and even when I forced it some other creative way the program was exhibiting undesired behaviour

• exiting short of processing all the contents within a channel
• deadlocks or send on closed channel

This is because of the nuances of the “real” code where processing the contents of a channel takes longer than it took to fill the channel and things were just out of sync.

Since there’s no clear practical way within my sender to close the channel which is recommended in 99% of channel tutorials, it’s actually acceptable to do it in main via a goroutine when you have multiple workers reading a channel and the workers are unaware of which of them read the last value.

solution

I’ve wrapped the workers in their own sync.WaitGroup and used worker.Wait() to block the progam from exiting, allowing the works to “finish”. Independently I close() the channel when there's no more data to send, i.e. I block by waiting for the writers to finish by using their own wait group. The close gives the range loop a terminating case as it’ll end when the default value of the channel is returned, i.e. and EOF type return when the channel end is reached. A blocking rendezvous channel has no end, until it's closed.

The way I think about this is go has no way of knowing the length of an unbuffered channel as it’s being ranged over if you don’t know how many values will be pushed to it in parrellel, until you close it. Since closing it says read whatever is left until the terminating value or end. And the workers.Wait() blocks until that's complete.

example of the solved op
https://go.dev/play/p/NzKIYOqaGGD

example of reading a closed channel
https://go.dev/play/p/HYZofbz0Xc5

output

filling 0
filling 1
filling 2
filling 3
filling 4
filling 5
filling 6
filling 7
filling 8
filling 9
closed
empyting 0
empyting 1
empyting 2
empyting 3
empyting 4
empyting 5
empyting 6
empyting 7
empyting 8
empyting 9