布尔检查的模糊测试?

huangapple go评论71阅读模式
英文:

Fuzzing for boolean checks?

问题

我有一个函数,用于检查一个字符串是否是一个有效的十六进制数表示(例如'0xdEAdBEef'和类似的字符串)。现在我在isHexaString(string)上有一个检查,它返回一个布尔值,当字符串不是有效的十六进制数表示时为false。但是在使用一堆种子运行go -fuzz=Fuzz时,检查出现错误,我没有得到任何有意义的结果(没有恐慌)。这是否意味着我的代码(isHexaString)是正确的?

对于这种类型的单元测试,最好的模糊测试方法是什么?

哦,而且只在一个包上运行会触发运行整个单元测试(这可能是Go希望先运行单元测试的行为吗?)

英文:

I have a function that checks if a string is a valid representation of an hexadecimal number ('0xdEAdBEef' and that sort of strings.) Now I have a check on isHexaString(string) which returns a bool that is false when it's not. But running go -fuzz=Fuzz with a bunch of seeds and they the checks with errors, I am not getting anything meaningful (no panics). Does that mean my code (isHexaString) is okay?

func FuzzIsHexAddress(f *testing.F) {
	testcases := []string{
		"0x5aaeb6053f3e94c9b9a09f33669435e7ef1beaed",
		"5aaeb6053f3e94c9b9a09f33669435e7ef1beaed",
		"0X5aaeb6053f3e94c9b9a09f33669435e7ef1beaed",
		"0XAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
		"0xAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
	}
	for _, tc := range testcases {
		f.Add(tc) // Use f.Add to provide a seed corpus
	}

	f.Fuzz(func(t *testing.T, orig string) {
		if result := isHexaString(orig); !result {
			t.Errorf("IsHexAddress(%s) == %v",
				orig, result)
		}
	})
} 

What's the best way to fuzz test this type of unit tests?

Oh and running on one package only triggers the whole unit tests to be ran (must be a behaviour of Go to want to run unit tests first?)

答案1

得分: 1

我不认为这个模糊测试是正确的。这个条件result := isHexaString(orig); !resultorig不是有效的十六进制字符串时会评估为true。所以你基本上是在说"当输入字符串不是十六进制字符串时失败"。但是为什么测试应该失败呢?如果orig确实不是十六进制字符串,你的函数产生了正确的结果:false

关键在于你无法控制模糊输入(它是模糊的!),因此你不应仅仅依靠被测试函数的输出来确定通过或失败。

fuzz教程提供了一个字符串反转函数的示例:

rev := Reverse(orig)
doubleRev := Reverse(rev)
if orig != doubleRev {
    t.Errorf("Before: %q, after: %q", orig, doubleRev)
}

在这种特殊情况下,Reverse应该是一个自反函数,即它的逆反是它本身。你的函数isHexaString不是一个自反函数,甚至没有逆反(它只是满射的),但你可以将其输出与一个黄金标准进行比较,例如另一个已知始终正确识别有效十六进制字符串的函数。

或者,你可以继续使用条件result := isHexaString(orig); !result来找到一些有趣的边界情况,这些情况不是十六进制字符串,并将其包含在非模糊的单元测试中。

英文:

I don't think this fuzz test is correct. This condition result := isHexaString(orig); !result evaluates to true when orig is not a valid hex string. So you are basically saying "fail when the input string is not a hex string". But why the test should fail then? If orig was indeed not a hex string, your function yielded the correct result: false.

The point is that you are not in control of the fuzz input (it is fuzzed!), therefore you should not rely on the output of the function under test alone to determine a pass or fail.

The fuzz tutorial suggests the example of a string reverse function:

rev := Reverse(orig)
doubleRev := Reverse(rev)
if orig != doubleRev {
    t.Errorf("Before: %q, after: %q", orig, doubleRev)
}

In this particular case Reverse is supposed to be an involution, i.e. the reverse of itself. Your function isHexaString is not an involution, and it doesn't even have an inverse (it's only surjective) but you can check its output against a golden standard, e.g. some other function that is known to always correctly identify valid hex strings.

Or, you can keep using the condition result := isHexaString(orig); !result to find interesting edge cases that are not hex strings and include those into your non-fuzzed unit test.

huangapple
  • 本文由 发表于 2022年3月14日 17:34:50
  • 转载请务必保留本文链接:https://go.coder-hub.com/71465556.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定