Am I doing something wrong or is this a bug in Go's C compiler?

huangapple go评论98阅读模式
英文:

Am I doing something wrong or is this a bug in Go's C compiler?

问题

所以我正在将xxhash从使用cgo转换为Go的本地9p C,但是我遇到了一个非常奇怪的问题。

如果作为cgo函数调用,哈希函数完全正常工作,但是如果我尝试使用“本地”版本,它会返回错误的哈希值。

我了解足够的C知识来使其工作,但在报告问题之前,我想确保我没有做错任何事情。

gist

xxhash.go:

//#include "xxhash_9p.c"
//import "C" //取消注释此行并注释下一行以使用cgo版本
func XXH32_test(in unsafe.Pointer, l uint32, seed uint32) uint32


func GoXXH32(in []byte, seed uint32) (h uint32) {
	//省略,完整版本在上面的gist中
}

func main() {
	b := []byte("ABCDEFGLAALSDLSD:LSDL:DL:DL:SDL:SL:DSL:DL:DSL:DL:{W{EOQWExzghp[[")
	fmt.Println(XXH32_test(unsafe.Pointer(&b[0]), uint32(len(b)), 0)) //取消注释此行并注释下一行以使用cgo版本
	//fmt.Println(C.XXH32_test(unsafe.Pointer(&b[0]), C.uint(len(b)), 0))
	fmt.Println(GoXXH32(b, 0)) //这与C实现进行了测试,是正确的哈希值。
}

xxhash_9p.c:

#define PRIME32_1   2654435761U
#define PRIME32_2   2246822519U
#define PRIME32_3   3266489917U
#define PRIME32_4    668265263U
#define PRIME32_5    374761393U

#define U32 unsigned int
typedef struct _U32_S { U32 v; } U32_S;
#define A32(x) (((U32_S *)(x))->v)

U32 ·XXH32_test(const void* input, U32 len, U32 seed) {
//static U32 XXH32_test(const void* input, U32 len, U32 seed) {
	const char* p = (const char*)input;
	const char* bEnd = p + len;
	U32 h32;
	
	#define XXH_get32bits(p) A32(p)
	#define XXH_rotl32(x,r) ((x << r) | (x >> (32 - r)))

	if (len>=16) {
		const char* const limit = bEnd - 16;
		U32 v1 = seed + PRIME32_1 + PRIME32_2;
		U32 v2 = seed + PRIME32_2;
		U32 v3 = seed + 0;
		U32 v4 = seed - PRIME32_1;
		do
		{
			v1 += XXH_get32bits(p) * PRIME32_2; v1 = XXH_rotl32(v1, 13); v1 *= PRIME32_1; p+=4;
			v2 += XXH_get32bits(p) * PRIME32_2; v2 = XXH_rotl32(v2, 13); v2 *= PRIME32_1; p+=4;
			v3 += XXH_get32bits(p) * PRIME32_2; v3 = XXH_rotl32(v3, 13); v3 *= PRIME32_1; p+=4;
			v4 += XXH_get32bits(p) * PRIME32_2; v4 = XXH_rotl32(v4, 13); v4 *= PRIME32_1; p+=4;
		} while (p<=limit);

		h32 = XXH_rotl32(v1, 1) + XXH_rotl32(v2, 7) + XXH_rotl32(v3, 12) + XXH_rotl32(v4, 18);
	}
	else
	{
		h32  = seed + PRIME32_5;
	}

	h32 += (unsigned long) len;
	while (p<=bEnd-4) {
		h32 += XXH_get32bits(p) * PRIME32_3;
		h32  = XXH_rotl32(h32, 17) * PRIME32_4 ;
		p+=4;
	}

	while (p<bEnd) {
		h32 += (*p) * PRIME32_5;
		h32 = XXH_rotl32(h32, 11) * PRIME32_1 ;
		p++;
	}

	h32 ^= h32 >> 15;
	h32 *= PRIME32_2;
	h32 ^= h32 >> 13;
	h32 *= PRIME32_3;
	h32 ^= h32 >> 16;
	return h32;
}

运行:

$ go build && ./nocgo #9p native
134316512
981225178
$ go build && ./nocgo #cgo
981225178
981225178

TL;DR:

当通过Go的6c使用时,C函数返回错误的值,但是当通过CGO调用相同的C函数时,返回正确的值。

//编辑

我在问题上得到了回复,它不会被修复,9p工具链最终会消失。

来自mi...@golang.org的回复:

> C编译器最终会消失。计划做好准备,不要依赖它。
>
> 请注意,Plan 9 C编译器并不完全符合ANSI标准,我们不会修复其中的错误(因为我们控制编译器及其输入,我们只会解决其错误)。

英文:

So I'm porting xxhash from using cgo to Go's native 9p C, however I'm running into a rather weird problem.

The hash function works perfectly fine if called as a cgo function, however if I try to use the "native" version it returns the wrong hash.

I know enough C to get it working, but before reporting the issue, I want to make sure I'm not doing anything wrong.

gist

xxhash.go:

//#include &quot;xxhash_9p.c&quot;
//import &quot;C&quot; //uncomment this and comment the next line for the cgo version
func XXH32_test(in unsafe.Pointer, l uint32, seed uint32) uint32


func GoXXH32(in []byte, seed uint32) (h uint32) {
	//omitted, full version in the gist above
}

func main() {
	b := []byte(&quot;ABCDEFGLAALSDLSD:LSDL:DL:DL:SDL:SL:DSL:DL:DSL:DL:{W{EOQWExzghp[[&quot;)
	fmt.Println(XXH32_test(unsafe.Pointer(&amp;b[0]), uint32(len(b)), 0)) //uncomment this and comment the next line for the cgo version
	//fmt.Println(C.XXH32_test(unsafe.Pointer(&amp;b[0]), C.uint(len(b)), 0))
	fmt.Println(GoXXH32(b, 0)) //this is tested against the C implementation and it&#39;s the right hash.
}

xxhash_9p.c:

#define PRIME32_1   2654435761U
#define PRIME32_2   2246822519U
#define PRIME32_3   3266489917U
#define PRIME32_4    668265263U
#define PRIME32_5    374761393U

#define U32 unsigned int
typedef struct _U32_S { U32 v; } U32_S;
#define A32(x) (((U32_S *)(x))-&gt;v)

U32 &#183;XXH32_test(const void* input, U32 len, U32 seed) {
//static U32 XXH32_test(const void* input, U32 len, U32 seed) {
	const char* p = (const char*)input;
	const char* bEnd = p + len;
	U32 h32;
	
	#define XXH_get32bits(p) A32(p)
	#define XXH_rotl32(x,r) ((x &lt;&lt; r) | (x &gt;&gt; (32 - r)))

	if (len&gt;=16) {
		const char* const limit = bEnd - 16;
		U32 v1 = seed + PRIME32_1 + PRIME32_2;
		U32 v2 = seed + PRIME32_2;
		U32 v3 = seed + 0;
		U32 v4 = seed - PRIME32_1;
		do
		{
			v1 += XXH_get32bits(p) * PRIME32_2; v1 = XXH_rotl32(v1, 13); v1 *= PRIME32_1; p+=4;
			v2 += XXH_get32bits(p) * PRIME32_2; v2 = XXH_rotl32(v2, 13); v2 *= PRIME32_1; p+=4;
			v3 += XXH_get32bits(p) * PRIME32_2; v3 = XXH_rotl32(v3, 13); v3 *= PRIME32_1; p+=4;
			v4 += XXH_get32bits(p) * PRIME32_2; v4 = XXH_rotl32(v4, 13); v4 *= PRIME32_1; p+=4;
		} while (p&lt;=limit);

		h32 = XXH_rotl32(v1, 1) + XXH_rotl32(v2, 7) + XXH_rotl32(v3, 12) + XXH_rotl32(v4, 18);
	}
	else
	{
		h32  = seed + PRIME32_5;
	}

	h32 += (unsigned long) len;
	while (p&lt;=bEnd-4) {
		h32 += XXH_get32bits(p) * PRIME32_3;
		h32  = XXH_rotl32(h32, 17) * PRIME32_4 ;
		p+=4;
	}

	while (p&lt;bEnd) {
		h32 += (*p) * PRIME32_5;
		h32 = XXH_rotl32(h32, 11) * PRIME32_1 ;
		p++;
	}

	h32 ^= h32 &gt;&gt; 15;
	h32 *= PRIME32_2;
	h32 ^= h32 &gt;&gt; 13;
	h32 *= PRIME32_3;
	h32 ^= h32 &gt;&gt; 16;
	return h32;
}

Run:

$ go build &amp;&amp; ./nocgo #9p native
134316512
981225178
$ go build &amp;&amp; ./nocgo #cgo
981225178
981225178

TL;DR:

A C function returns the wrong value when used through Go's 6c, same exact C function returns the correct value when called through CGO.

//edit

I got a response on the issue, it's not gonna get fixed and the 9p toolchain is going away eventually.

From mi...@golang.org:

> the C compiler will eventually go away. Plan for that, so don't rely
> on it.
>
> Note the Plan 9 C compiler isn't fully ANSI compliant, and we're not
> going to fix bugs in it (because we control both the compiler and its
> input, we will just workaround its bugs).

答案1

得分: 3

经过一番调查,将函数签名从

U32 &#183;XXH32_test(const void* input, U32 len, U32 seed)

改为

void &#183;XXH32_test(const unsigned char* input, U32 len, U32 seed, U32 *ret)

并像这样调用它:

var u uint32
XXH32_test(unsafe.Pointer(&amp;b[0]), uint32(len(b)), 0, &amp;u)

可以返回正确的哈希值。

我仍然不确定发生了什么,它应该按照最初的方式工作,但我猜测运行时在幕后做了一些魔法。

英文:

After some digging, changing the function signature from

U32 &#183;XXH32_test(const void* input, U32 len, U32 seed)

to

void &#183;XXH32_test(const unsigned char* input, U32 len, U32 seed, U32 *ret)

And calling it like :

var u uint32
XXH32_test(unsafe.Pointer(&amp;b[0]), uint32(len(b)), 0, &amp;u)

Returns the correct hash.

I'm still not sure what's going on, it should work how it originally was, but I'm guessing the runtime is doing some magic behind the scenes.

huangapple
  • 本文由 发表于 2014年7月31日 07:44:58
  • 转载请务必保留本文链接:https://go.coder-hub.com/25048775.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定