Does not produce the same output Concurrency Go worker pool

I am writing a program that concurrently reads word by word from a text file to compute the occurrences using channels and worker pool pattern

The program works in the following flow:

1. Read a text file (readText function)
2. readText function sends each word to the word channel
3. Each goroutine executes countWord function that counts word in a map
4. Each goroutine returns a map and the worker function passes the Result value of struct to the resultC channel
5. Test function creates a map based on the result values coming from the resultC channel
6. Print the map created from step 5

The program works, but when I try to put fmt.Println(0) to see the process as shown below

func computeTotal() {
	i := 0
	for e := range resultC {
		total[e.word] += e.count
		i += 1
		fmt.Println(i)
	}
}

The program terminates without showing/counting all the words

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 all goroutines finished 16 17 18 map[but:1 cat's:1 crouched:1 fur:1 he:2 imperturbable:1 it:1 pointed:1 sat:1 snow:1 stiffly:1 the:1 was:2 with:1] total words: 27 38 ... 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 Time taken for reading the book 5.8145ms

The program shows results correctly if I uncomment the fmt.println() in the compute Total function statement here and the output is as shown below

all goroutines finished
map[a:83 about:4 above:2 absolute:1 accepted:1 across:1 affection:1 after:1 again:5  wonder:2 wood:5 wooded:1 woody:1 work:1 worked:2 world:4 would:11 wrapped:1 wrong:1 yellow:2 yielded:1 yielding:1 counts continues ......]
total words:  856
Time taken for reading the book 5.9924ms

here is my implementation of readtext

//ensure close words at the right timing
func readText() {

	file, err := os.Open(FILENAME)
	if err != nil {
		log.Fatal(err)
	}
	defer file.Close()
	scanner := bufio.NewScanner(file)
	scanner.Split(bufio.ScanWords)

	for scanner.Scan() {
		word := strings.ToLower(scanner.Text())
		words <- strings.Trim(word, ".,:;")

	}
	//time.Sleep(1 * time.Second)
	close(words)
}

here is my count word implementation using worker pool

//call countWord func,
func workerPool() {
	var wg sync.WaitGroup
	for i := 1; i <= NUMOFWORKER; i++ {
		wg.Add(1)
		go worker(&wg)
	}
	wg.Wait()
	fmt.Println("all goroutines finished")
	close(resultC)
}

func worker(wg *sync.WaitGroup) {
	var tempMap = make(map[string]int)
	for w := range words {
		resultC <- countWord(w, tempMap) //retuns Result value
	}
	wg.Done()

}

//creates a map each word
func countWord(word string, tempMap map[string]int) Result {
	_, ok := tempMap[word]
	if ok {
		tempMap[word]++
		return Result{word, tempMap[word] + 1}

	}
	return Result{word, 1}

}

Finally, this is the main function

const FILENAME = "cat.txt"
const BUFFERSIZE = 3000
const NUMOFWORKER = 5

var words = make(chan string, BUFFERSIZE) //job
var resultC = make(chan Result, BUFFERSIZE)

var total = map[string]int{}

type Result struct {
	word  string
	count int
}

func main() {
	startTime := time.Now()
	go readText()
	go computeTotal()
	workerPool() //blocking
	fmt.Println(total)
	endTime := time.Now()
	timeTaken := endTime.Sub(startTime)
	fmt.Println("total words: ", len(total))
	fmt.Println("Time taken for reading the book", timeTaken)
}

I have been looking for why the program does not show consistent results but I could not figure it out yet. How can I make a change to the program so that it produces the same outcome?

The countWord function always returns a result with count == 1.

Here's a version of the function that increments the count:

func countWord(word string, tempMap map[string]int) Result {
count := tempMap[word] + 1
tempMap[word] = count
return Result{word, count}
}

But hold that thought! The computeTotal function assumes that the result count is 1. Given that the workers in the question always send Result{word, 1} as computeTotal expects, we can cut the workers out of the picture by sending Result{word, 1} directly from readText. Here's the code:

func computeTotal() {
i := 0
for e := range resultC {
total[e.word] += e.count
i += 1
fmt.Println(i)
}
}
func readText() {
file, err := os.Open(FILENAME)
if err != nil {
log.Fatal(err)
}
defer file.Close()
scanner := bufio.NewScanner(file)
scanner.Split(bufio.ScanWords)
for scanner.Scan() {
word := strings.ToLower(scanner.Text())
resultC <- Result{strings.Trim(word, ".,:;"), 1}
}
close(resultC)
}
main() {
...
go readText()
computeTotal()
fmt.Println(total)
...
}

Run it on the playground.

The overhead of the channel operations probably negates any benefit of running computeTotal and readText in separate goroutines. Here's the code combined into a single goroutine:

func main() {
file, err := os.Open(FILENAME)
if err != nil {
log.Fatal(err)
}
defer file.Close()
scanner := bufio.NewScanner(file)
scanner.Split(bufio.ScanWords)
var total = map[string]int{}
for scanner.Scan() {
word := strings.ToLower(strings.Trim(scanner.Text(), ".,:;"))
total[word]++
}
fmt.Println(total)
}

Run it on the playground.

The countWord function in the question makes me think your goal was to count words in each worker and merge the result for a total. Here's the code for that:

func computeTotal() {
for i := 1; i <= NUMOFWORKER; i++ {
m := <-resultC
for word, count := range m {
total[word] += count
}
}
}
func workerPool() {
for i := 1; i <= NUMOFWORKER; i++ {
go worker()
}
}
func worker() {
var tempMap = make(map[string]int)
for w := range words {
tempMap[w]++
}
resultC <- tempMap
}
...
var resultC = make(chan map[string]int)
...
func main() {
...
go readText()
workerPool()
computeTotal()
...
}

Run it on the playground.

You have to rewrite your computeTotal function in a following way:

func computeTotal(done chan struct{}) {
defer close(done)
i := 0
for e := range resultC {
total[e.word] += e.count
i += 1
fmt.Println(i)
}
}
func main() {
computeTotalDone := make(chan struct{})
go computeTotal(computeTotalDone)
...
workerPool() //blocking
<-computeTotalDone
fmt.Println(total)
}

The reason why adding fmt.Println leads to invalid result is that your implementation has a race condition. As printing total result in main function fmt.Println(total) and computeTotal function runs in parallel there is no guarantee that computeTotal handles all messages before fmt.Println(total) is invoked. Without fmt.Println the computeTotal function is fast enough on your computer to produce correct results.

The proposed solution ensures that computeTotal completes before fmt.Println(total) is invoked.