How to test an expected formatted error in idiomatic Go?

This question is NOT about validation. It is just an example, but could be any other thing with formatted errors.

Let's say I have a very simple Validate() method that validates an extension eg. .vcf from a filename and I want an error saying valid extension and current extension if they don't match. Ideally, this method would be extended to do other validations and return other validation errors.

Something like:

// main.go
package main

import (
	"fmt"
	"os"
	"path/filepath"
)

type ValidationError struct {
	Msg string
}

func (v ValidationError) Error() string {
	return v.Msg
}

func Validate(fileName string) error {
	// first validation
	wantExtension := ".vcf"
	gotExtension := filepath.Ext(fileName)
	if gotExtension != wantExtension {
		return ValidationError{Msg: fmt.Sprintf("Extension %q not accepted, please use a %s file.", gotExtension, wantExtension)}
	}
	// potentially other validations
	return nil
}

func main() {
	fileName := os.Args[1]
	Validate(fileName)
}

I want to test it in a way that I check not only the error type (in this case error, but also the message). Something like:

// main_test.go
package main

import (
	"errors"
	"testing"
)

func TestValidate(t *testing.T) {
	testCases := []struct {
		name        string
		fileName    string
		expectedErr error
	}{
		{
			name:     "happy-path",
			fileName: "file.vcf",
		},
		{
			name:        "wrong-extension",
			fileName:    "file.md",
			expectedErr: ValidationError{Msg: "Extension .md not accepted, please use a .vcf file."},
		},
	}
	for _, tc := range testCases {
		t.Run(tc.name, func(t *testing.T) {
			err := Validate(tc.fileName)
			if !errors.Is(err, tc.expectedErr) {
				t.Errorf("want (%v) got (%v)", tc.expectedErr, err)
			}
		})
	}
}

This test will get:

--- FAIL: TestValidate (0.00s)
    --- FAIL: TestValidate/wrong-extension (0.00s)
        main_test.go:28: want (Extension .md not accepted, please use a .vcf file.) got (Extension ".md" not accepted, please use a .vcf file.)

I does not work since errors.Is() expects same memory object. Also if I use errors.As() the test will pass but it will not really check if the message is correct.

I also tried defining a function to return an error with the message as parameter or simpler errors.New() instead of custom error, but I run into the same problem that the object in memory is different in the test side, also don't know which of the three approaches would be considered more idiomatic.

How would you implement that?

UPDATE: I updated the question code to use custom error instead of errors.New as it is probably more fit to the case as in the answer by @kingkupps.

I would recommend defining your own type and having that type implement the Error interface. In your tests you can use errors.As to determine whether the returned error is an instance of the new type or is an error wrapping an instance of the new type.

package main
import (
"errors"
"fmt"
"path/filepath"
)
type ValidationError struct {
WantExtension string
GotExtension  string
}
func (v ValidationError) Error() string {
return fmt.Sprintf("extension %q not accepted, please use a %s file.", v.GotExtension, v.WantExtension)
}
func Validate(fileName string) error {
fileExtension := filepath.Ext(fileName)
if fileExtension != ".vcf" {
return ValidationError{WantExtension: ".vcf", GotExtension: fileExtension}
}
return nil
}
func main() {
fileName := "something.jpg"
err := Validate(fileName)
var vErr ValidationError
if errors.As(err, &vErr) {
// you can now use vErr.WantExtension and vErr.GotExtension
fmt.Println(vErr)
fmt.Println(vErr.WantExtension)
fmt.Println(vErr.GotExtension)
}
}

Which prints:

extension ".jpg" not accepted, please use a .vcf file.
.vcf
.jpg

Then in your test, you can compare errors based on either the result of Error() or their fields:

func TestValidate(t *testing.T) {
testCases := []struct {
name        string
fileName    string
expectedErr error
}{
{
name:     "happy-path",
fileName: "file.vcf",
},
{
name:        "wrong-extension",
fileName:    "file.md",
expectedErr: ValidationError{WantExtension: ".vcf", GotExtension: ".md"},
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
err := Validate(tc.fileName)
if err != nil {
if tc.expectedErr == nil {
t.Errorf("unexpected error: %v", err)
return
}
var want ValidationError
var got ValidationError
if w, g := errors.As(tc.expectedErr, &want), errors.As(err, &got); w == g && w {
// Both errors are ValidationErrors
assertValidationErrorsEqual(t, want, got)
} else if w != g {
// The expected and actual error differ in type
t.Errorf("wanted error of type %T, got error of type %T", tc.expectedErr, err)
} else {
// Neither error is a ValidationError so we just assert that they produce the same error message
assertErrorMessagesEqual(t, tc.expectedErr, err)
}
}
})
}
}
func assertErrorMessagesEqual(t *testing.T, want error, got error) {
if w, g := want.Error(), got.Error(); w != g {
t.Errorf("want %q, got %q", w, g)
}
}
func assertValidationErrorsEqual(t *testing.T, want ValidationError, got ValidationError) {
if want.WantExtension != got.WantExtension || want.GotExtension != got.GotExtension {
t.Errorf("want %v, got %v", want, got)
}
}

EDIT: Thanks to @Daniel Farrell for the suggestion on using errors.As over a type assertion. Using errors.As will do what you want even if the returned error wraps an instance of ValidationError whereas a type assertion will not.

EDIT: Added what the test might look like in using the suggested approach.

there is no problem in using errors.Is. The code did not fail at using it.

It is the test code that is not setup correctly.

In the program, the error message is defined as Extension ".md" not accepted, please use a .vcf file., like in, return ValidationError{Msg: fmt.Sprintf("Extension %q not accepted, please use a %s file.", gotExtension, wantExtension)}

In the test, the error message is defined as expectedErr: ValidationError{Msg: "Extension .md not accepted, please use a .vcf file."},

Hence, when trying to compare the two error values, the state is similar to

	fmt.Println(
errors.Is(
ValidationError{Msg: `Extension ".md" not accepted, please use a .vcf file.`},
ValidationError{Msg: `Extension .md not accepted, please use a .vcf file.`},
),
)

Which is expected to return false using that api as it performs equality check of comparable values.

https://cs.opensource.google/go/go/+/refs/tags/go1.17.6:src/errors/wrap.go;l=46

For, reference, it is roughly equivalent to

	fmt.Println(
ValidationError{Msg: `Extension ".md" not accepted, please use a .vcf file.`} ==
ValidationError{Msg: `Extension .md not accepted, please use a .vcf file.`},
)

https://go.dev/ref/spec#Comparison_operators

> Struct values are comparable if all their fields are comparable. Two struct values are equal if their corresponding non-blank fields are equal.

Most certainly using errors, this one should be mentioned too

> Interface values are comparable. Two interface values are equal if they have identical dynamic types and equal dynamic values or if both have value nil.