使用binary.PutVarint(…)时出现索引超出范围的错误。

huangapple go评论82阅读模式
英文:

Index Out of Range when using binary.PutVarint(...)

问题

以下是翻译好的内容:

package main

import (
	"fmt"
	"math/rand"
	"encoding/binary"
)

func main() {
	buffer := []byte{0, 0, 0, 0, 0, 0, 0, 0}
	num := rand.Int63()
	count := binary.PutVarint(buffer, num)
	
	fmt.Println(count)
}

我之前在num只是一个递增的uint64并且使用binary.PutUvarint时,这段代码是可以工作的。但是现在num是一个随机的int64并且使用binary.PutVarint时,我遇到了一个错误:

panic: runtime error: index out of range

goroutine 1 [running]:
encoding/binary.PutUvarint(0x1042bf58, 0x8, 0x8, 0x6ccb, 0xff9faa4, 0x9acb0442, 0x7fcfd52, 0x4d658221)
	/usr/local/go/src/encoding/binary/varint.go:44 +0xc0
encoding/binary.PutVarint(0x1042bf58, 0x8, 0x8, 0x6ccb, 0x7fcfd52, 0x4d658221, 0x14f9e0, 0x104000e0)
	/usr/local/go/src/encoding/binary/varint.go:83 +0x60
main.main()
	/tmp/sandbox010341234/main.go:12 +0x100

我漏掉了什么?我本以为这只是一个微不足道的改变...

编辑:我刚刚尝试扩展了我的缓冲数组。出于某种奇怪的原因,它起作用了,并且我得到了一个count10的结果。这怎么可能?int64是64位 = 8字节,对吗?

英文:

http://play.golang.org/p/RqScJVvpS7

package main

import (
    "fmt"
    "math/rand"
    "encoding/binary"
)

func main() {
	buffer := []byte{0, 0, 0, 0, 0, 0, 0, 0}
	num := rand.Int63()
	count := binary.PutVarint(buffer, num)
	
	fmt.Println(count)
}

I had this working awhile ago when num was just an incrementing uint64 and I was using binary.PutUvarint but now that it's a random int64 and binary.PutVarint I get an error:

panic: runtime error: index out of range

goroutine 1 [running]:
encoding/binary.PutUvarint(0x1042bf58, 0x8, 0x8, 0x6ccb, 0xff9faa4, 0x9acb0442, 0x7fcfd52, 0x4d658221)
	/usr/local/go/src/encoding/binary/varint.go:44 +0xc0
encoding/binary.PutVarint(0x1042bf58, 0x8, 0x8, 0x6ccb, 0x7fcfd52, 0x4d658221, 0x14f9e0, 0x104000e0)
	/usr/local/go/src/encoding/binary/varint.go:83 +0x60
main.main()
	/tmp/sandbox010341234/main.go:12 +0x100

What am I missing? I would have thought this to be a trivial change...

EDIT: I just tried extending my buffer array. For some odd reason it works and I get a count of 10. How can that be? int64 is 64 bits = 8 bytes, right?

答案1

得分: 7

引用encoding/binary文档中的内容:

varint函数使用可变长度编码对单个整数值进行编码和解码;较小的值需要较少的字节。有关规范,请参阅https://developers.google.com/protocol-buffers/docs/encoding。

因此,binary.PutVarint()不是固定长度的编码,而是可变长度的编码。当传递一个int64时,对于大数值,它将需要超过8个字节,对于小数值,它将需要少于8个字节。由于您要编码的数字是一个随机数,即使在其最高字节中也会有随机位。

看看这个简单的例子:

buffer := make([]byte, 100)
for num := int64(1); num < 1<<60; num <<= 4 {
	count := binary.PutVarint(buffer, num)
	fmt.Printf("Num=%d, bytes=%d\n", num, count)
}

输出:

Num=1, bytes=1
Num=16, bytes=1
Num=256, bytes=2
Num=4096, bytes=2
Num=65536, bytes=3
Num=1048576, bytes=4
Num=16777216, bytes=4
Num=268435456, bytes=5
Num=4294967296, bytes=5
Num=68719476736, bytes=6
Num=1099511627776, bytes=6
Num=17592186044416, bytes=7
Num=281474976710656, bytes=8
Num=4503599627370496, bytes=8
Num=72057594037927936, bytes=9

可变长度编码的本质是小数值使用较少的字节,但只有在大数值使用超过8个字节(即int64的大小)时才能实现这一点。

有关具体编码的详细信息,请参考链接页面

一个非常简单的例子是:一个字节是8位。使用输出字节的7位作为编码数据/数字的“有用”位。如果最高位为1,表示需要更多的字节。如果最高位为0,表示完成。您可以看到,小数值可以使用1个输出字节进行编码(例如n=10),而对于每7位有用数据,我们使用1个额外的位,因此如果输入数字使用了所有的64位,我们将得到超过8个字节:需要10个组来覆盖64位,因此我们将需要10个字节(9个组只有9*7=63位)。

英文:

Quoting the doc of encoding/binary:

> The varint functions encode and decode single integer values using a variable-length encoding; smaller values require fewer bytes. For a specification, see https://developers.google.com/protocol-buffers/docs/encoding.

So the binary.PutVarint() is not a fixed, but a variable-length encoding. When passing an int64, it will need more than 8 bytes for large numbers, and less than 8 bytes for small numbers. Since the number you're encoding is a random number, it will have random bits even in its highest byte.

See this simple example:

buffer := make([]byte, 100)
for num := int64(1); num &lt; 1&lt;&lt;60; num &lt;&lt;= 4 {
	count := binary.PutVarint(buffer, num)
	fmt.Printf(&quot;Num=%d, bytes=%d\n&quot;, num, count)
}

Output:

Num=1, bytes=1
Num=16, bytes=1
Num=256, bytes=2
Num=4096, bytes=2
Num=65536, bytes=3
Num=1048576, bytes=4
Num=16777216, bytes=4
Num=268435456, bytes=5
Num=4294967296, bytes=5
Num=68719476736, bytes=6
Num=1099511627776, bytes=6
Num=17592186044416, bytes=7
Num=281474976710656, bytes=8
Num=4503599627370496, bytes=8
Num=72057594037927936, bytes=9

The essence of variable-length encoding is that small numbers use less bytes, but this can only be achieved if in turn big numbers may use more than 8 bytes (that would be size of int64).

Details of the specific encoding is on the linked page.

A very easy example would be: A byte is 8 bits. Use 7 bits of the output byte as the "useful" bits to encode the data/number. If the highest bit is 1, that means more bytes are required. If highest bit is 0, we're done. You can see that small numbers can be encoded using 1 output byte (e.g. n=10), while we're using 1 extra bit for every 7-bit useful data, so if the input number uses all the 64 bits, we will end up with more than 8 bytes: 10 groups are required to cover 64 bits, so we will need 10 bytes (9 groups is only 9*7=63 bits).

huangapple
  • 本文由 发表于 2015年9月22日 22:55:48
  • 转载请务必保留本文链接:https://go.coder-hub.com/32720218.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定