Improving testing.Benchmark for Go?

In learning about Go, I see that you can benchmark a function using fmt.Println(testing.Benchmark(BenchmarkFunction)), having set up the following:

func BenchmarkFunction(b *testing.B) {
	n := 42
	for i := 0; i < b.N; i++ {
		_ = Function(n)
	}
}

However, as the BenchmarkFunction code would be repeated for every benchmark of every method you would do this for (hence a code smell, in terms of DRY), is there a way that this could be rewritten using closures (or in some other manner) such that a benchmark of a function could be rewritten something like:

fmt.println(test.BenchmarkMyFunction(MyFunction(parameters...)))

and added either into my code or the testing library?

Actually, this is not the right way to do benchmarks in Go.

The actual standard is to put your benchmarking code into functions that have names BenchmarkXXX where XXX is whatever you like. Then you run go test -bench=. on the package that defines these files. go test is running all the benchmarks for you.

If you have similar benchmarks with slightly different parameters, you can write a generic benchmarking function that is just called by all the other benchmarks:

func genericBenchmarkFoo(b *testing.B, param int) { ... }

And then you write a boilerplate function for each specific benchmark:

func BenchmarkFoo1(b *testing.B) { genericBenchmarkFoo(b, 1) }
func BenchmarkFoo2(b *testing.B) { genericBenchmarkFoo(b, 2) }
func BenchmarkFoo3(b *testing.B) { genericBenchmarkFoo(b, 3) }
func BenchmarkFoo4(b *testing.B) { genericBenchmarkFoo(b, 4) }

You can find an example of this pattern in this package I wrote.

Of course this is not exactly beautiful but I am afraid there is not an easier solution. It's clean coding however to find a small set of benchmarks that represent what you want to do.

Here's a real, simple, and DRY Go benchmark, which uses closures. I wanted to know how the performance of a various Substr functions varied with the size (hi - lo) of the substring.

package main

import (
	"fmt"
	"strings"
	"testing"
)

func Substr1(str string, lo, hi int) string {
	return string([]byte(str[lo:hi]))
}

func Substr2(str string, lo, hi int) string {
	sub := str[lo:hi]
	return (sub + " ")[:len(sub)]
}

func Substr3(str string, lo, hi int) string {
	sub := str[lo:hi]
	if len(sub) == 0 {
		return ""
	}
	return sub[0:1] + sub[1:]
}

var substrFunctions = []struct {
	name     string
	function func(str string, lo, hi int) string
}{
	{"Substr1", Substr1},
	{"Substr2", Substr2},
	{"Substr3", Substr3},
}

var substrBenchmarks = []struct {
	name                 string
	strLen, subLo, subHi int
}{
	{"Zero  ", 1, 1, 1},
	{"Small ", 4, 1, 4},
	{"Medium", 256, 1, 256},
	{"Large ", 4096, 1, 4096},
}

func BenchmarkSubstrSize() {
	fmt.Println("BenchmarkSubstrSize:")
	for _, benchmark := range substrBenchmarks {
		str := strings.Repeat("abc", benchmark.strLen)
		for _, function := range substrFunctions {
			benchmarkFunc := func(b *testing.B) {
				b.ResetTimer()
				for i := 0; i < b.N; i++ {
					function.function(str, benchmark.subLo, benchmark.subHi)
				}
				b.StopTimer()
			}
			results := testing.Benchmark(benchmarkFunc)
			fmt.Println(benchmark.name, function.name, results)
		}
	}
}

func main() {
	BenchmarkSubstrSize()
}

Output:

BenchmarkSubstrSize:
Zero    Substr1   50000000    54.8 ns/op
Zero    Substr2  100000000    19.6 ns/op
Zero    Substr3  500000000     6.66 ns/op
Small   Substr1   20000000    95.7 ns/op
Small   Substr2   50000000    70.4 ns/op
Small   Substr3   50000000    70.1 ns/op
Medium  Substr1    5000000   380 ns/op
Medium  Substr2   10000000   229 ns/op
Medium  Substr3   10000000   213 ns/op
Large   Substr1     500000  4290 ns/op
Large   Substr2    1000000  2007 ns/op
Large   Substr3    1000000  2275 ns/op

I'm not sure if this can be considered a gain:

package main

import (
        "fmt"
        "testing"
)

func f1(n int) (s int) {
        for i := 0; i < n; i++ {
                s += i
        }
        return
}

func f2(n int) (s int) {
        for i := 0; i < n; i++ {
                s += 2 * i
        }
        return
}

func bench(f func()) func(b *testing.B) {
        return func(b *testing.B) {
                for i := 0; i < b.N; i++ {
                        f()
                }
        }
}

func main() {
        fmt.Printf("%v\n", testing.Benchmark(bench(func() { f1(42) })))
        fmt.Printf("%v\n", testing.Benchmark(bench(func() { f2(24) })))
}

Output:

(16:55) jnml@fsc-r550:~/src/tmp/SO/16920669$ go run main.go 
50000000	        68.4 ns/op
50000000	        35.8 ns/op
(16:55) jnml@fsc-r550:~/src/tmp/SO/16920669$