英文:
Fuzzing for boolean checks?
问题
我有一个函数,用于检查一个字符串是否是一个有效的十六进制数表示(例如'0xdEAdBEef'和类似的字符串)。现在我在isHexaString(string)上有一个检查,它返回一个布尔值,当字符串不是有效的十六进制数表示时为false。但是在使用一堆种子运行go -fuzz=Fuzz时,检查出现错误,我没有得到任何有意义的结果(没有恐慌)。这是否意味着我的代码(isHexaString)是正确的?
对于这种类型的单元测试,最好的模糊测试方法是什么?
哦,而且只在一个包上运行会触发运行整个单元测试(这可能是Go希望先运行单元测试的行为吗?)
英文:
I have a function that checks if a string is a valid representation of an hexadecimal number ('0xdEAdBEef' and that sort of strings.) Now I have a check on isHexaString(string) which returns a bool that is false when it's not. But running go -fuzz=Fuzz with a bunch of seeds and they the checks with errors, I am not getting anything meaningful (no panics). Does that mean my code (isHexaString
) is okay?
func FuzzIsHexAddress(f *testing.F) {
testcases := []string{
"0x5aaeb6053f3e94c9b9a09f33669435e7ef1beaed",
"5aaeb6053f3e94c9b9a09f33669435e7ef1beaed",
"0X5aaeb6053f3e94c9b9a09f33669435e7ef1beaed",
"0XAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
"0xAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
}
for _, tc := range testcases {
f.Add(tc) // Use f.Add to provide a seed corpus
}
f.Fuzz(func(t *testing.T, orig string) {
if result := isHexaString(orig); !result {
t.Errorf("IsHexAddress(%s) == %v",
orig, result)
}
})
}
What's the best way to fuzz test this type of unit tests?
Oh and running on one package only triggers the whole unit tests to be ran (must be a behaviour of Go to want to run unit tests first?)
答案1
得分: 1
我不认为这个模糊测试是正确的。这个条件result := isHexaString(orig); !result
在orig
不是有效的十六进制字符串时会评估为true。所以你基本上是在说"当输入字符串不是十六进制字符串时失败"。但是为什么测试应该失败呢?如果orig
确实不是十六进制字符串,你的函数产生了正确的结果:false
。
关键在于你无法控制模糊输入(它是模糊的!),因此你不应仅仅依靠被测试函数的输出来确定通过或失败。
fuzz教程提供了一个字符串反转函数的示例:
rev := Reverse(orig)
doubleRev := Reverse(rev)
if orig != doubleRev {
t.Errorf("Before: %q, after: %q", orig, doubleRev)
}
在这种特殊情况下,Reverse
应该是一个自反函数,即它的逆反是它本身。你的函数isHexaString
不是一个自反函数,甚至没有逆反(它只是满射的),但你可以将其输出与一个黄金标准进行比较,例如另一个已知始终正确识别有效十六进制字符串的函数。
或者,你可以继续使用条件result := isHexaString(orig); !result
来找到一些有趣的边界情况,这些情况不是十六进制字符串,并将其包含在非模糊的单元测试中。
英文:
I don't think this fuzz test is correct. This condition result := isHexaString(orig); !result
evaluates to true when orig
is not a valid hex string. So you are basically saying "fail when the input string is not a hex string". But why the test should fail then? If orig
was indeed not a hex string, your function yielded the correct result: false
.
The point is that you are not in control of the fuzz input (it is fuzzed!), therefore you should not rely on the output of the function under test alone to determine a pass or fail.
The fuzz tutorial suggests the example of a string reverse function:
rev := Reverse(orig)
doubleRev := Reverse(rev)
if orig != doubleRev {
t.Errorf("Before: %q, after: %q", orig, doubleRev)
}
In this particular case Reverse
is supposed to be an involution, i.e. the reverse of itself. Your function isHexaString
is not an involution, and it doesn't even have an inverse (it's only surjective) but you can check its output against a golden standard, e.g. some other function that is known to always correctly identify valid hex strings.
Or, you can keep using the condition result := isHexaString(orig); !result
to find interesting edge cases that are not hex strings and include those into your non-fuzzed unit test.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论