在源代码中直接使用gobs，这是可能的吗？

huangapple
go评论138阅读模式
英文:

use gobs directly in the source code, is it possible?

问题

我想知道是否可以直接在源代码中使用gob编码的数据（例如在函数中）。原因是通过不访问磁盘获取gob文件来提高性能。我知道有memcached、redis等工具，但我不需要TTL或其他高级功能，只需要在内存中使用映射。数据将在“设置/构建”过程中进行编码和转储到源代码中，以便在运行时只需要“解码”它。

这个Go应用程序基本上将作为一个小型的只读嵌入式数据库。我可以使用JSON来实现这一点（基本上声明一个带有原始JSON的变量），但我猜这可能会有性能损失，所以我想知道是否可以使用gob来实现。

我尝试了不同的方法，但我无法使其工作，因为我不知道如何定义gob变量（byte、[bytes]？？）而且解码器似乎期望一个io.Reader，所以在花费整天时间之前，我决定先问问你们SO的朋友们是否可能实现。

糟糕的尝试：

  1. var test string
  2. test = "hello"
  4. p := new(bytes.Buffer)
  5. e := gob.NewEncoder(p)
  6. e.Encode(test)
  7. ers := ioutil.WriteFile("test.gob", p.Bytes(), 0600)
  8. if ers != nil {
  9.     panic(ers)
  10. }

现在我想将test.gob添加到一个函数中。根据我看到的，test.gob的源代码如下：^H^L^@^Ehello

  1. var test string
  3. var b bytes.Buffer
  5. b = byte("^H^L^@^Ehello")
  7. de := gob.NewDecoder(b.Bytes())
  9. er := de.Decode(&test)
  10. if er != nil {
  11.     fmt.Printf("cannot decode")
  12.     panic(er)
  13. }
  15. fmt.Fprintf(w, test)
英文:

I would like to know if it's possible to have gob encoded data directly in the source code (e.g. in a function). The reason is to increase performance by not having to access the disk to get the gob file. I'm aware of memcached, redis and friends. i don't need TTL or any other fancy feature. Just maps in memory. The data would be encoded and dumped in the source code in a 'setup'/build process so that at runtime it would only need to 'decode' it.

The go application would basically serve as a small read-only, embedded database. I can do this using json (basically declare a var with the raw json) but I guess there would be a performance penalty so I'm wondering if it's possible with gob.

I've tried different things but I can't make make it work because basically I don't know how to define the gob var (byte, [bytes] ?? ) and the decoder seems to expect an io.Reader so before spending the whole day on this I've decided to ask you SO fellas at least if it's possible to do.

Miserable attempt:

  1. var test string
  2. test = "hello"
  4. p := new(bytes.Buffer)
  5. e := gob.NewEncoder(p)
  6. e.Encode(test)
  7. ers := ioutil.WriteFile("test.gob", p.Bytes(), 0600)
  8. if ers != nil {
  9. 	panic(ers)
  10. }

Now I would like to take test.gob and add it in a function. As I can see, the source of test.gob reads like ^H^L^@^Ehello

  1. var test string
  3. var b bytes.Buffer
  5. b = byte("^H^L^@^Ehello")
  7. de := gob.NewDecoder(b.Bytes())
  9. er := de.Decode(&test)
  10. if er != nil {
  11. 	fmt.Printf("cannot decode")
  12. 	panic(er)
  13. }
  15. fmt.Fprintf(w, test)

答案1

得分: 5

将数据存储在字节切片中。这是原始数据，你可以从文件中读取它。

你 gob 文件中的字符串不是 "^H^L^@^Ehello"！这是你的编辑器显示非可打印字符的方式。

  1. b := []byte("\b\f\x00\x05hello")
  2. // 这不是 hello 的字符串表示，
  3. // 你需要一个 []byte，而不是 byte。
  4. // 它应该是这样的
  6. de := gob.NewDecoder(b)
  7. // b.Bytes() 返回一个 []byte，你想要读取的是缓冲区本身。

这里有一个可工作的示例：http://play.golang.org/p/6pvt2ctwUq

  1. func main() {
  2.     buff := &bytes.Buffer{}
  3.     enc := gob.NewEncoder(buff)
  4.     enc.Encode("hello")
  6.     fmt.Printf("Encoded: %q\n", buff.Bytes())
  8.     // 现在，如果你想直接在源代码中使用它
  9.     encoded := []byte("\b\f\x00\x05hello")
  10.     // 或 []byte{0x8, 0xc, 0x0, 0x5, 0x68, 0x65, 0x6c, 0x6c, 0x6f}
  12.     de := gob.NewDecoder(bytes.NewReader(encoded))
  14.     var test string
  15.     er := de.Decode(&test)
  16.     if er != nil {
  17.         fmt.Println("cannot decode", er)
  18.         return
  19.     }
  21.     fmt.Println("Decoded:", test)
  22. }
英文:

Store the data in a byte slice. It's raw data, and that's how you would read it in from a file.

The string in your gob file is not "^H^L^@^Ehello"! That's how your editor is displaying the non-printable characters.

  1. b = byte("^H^L^@^Ehello")
  2. // This isn't the string equivalent of hello, 
  3. // and you want a []byte, not byte. 
  4. // It should look like 
  6. b = []byte("\b\f\x00\x05hello")
  7. // However, you already declared b as bytes.Buffer, 
  8. // so this assignment isn't valid anyway.
  11. de := gob.NewDecoder(b.Bytes())
  12. // b.Bytes() returns a []byte, you want to read the buffer itself.

Here's a working example http://play.golang.org/p/6pvt2ctwUq

  1. func main() {
  2. 	buff := &bytes.Buffer{}
  3. 	enc := gob.NewEncoder(buff)
  4. 	enc.Encode("hello")
  6. 	fmt.Printf("Encoded: %q\n", buff.Bytes())
  8. 	// now if you wanted it directly in your source
  9. 	encoded := []byte("\b\f\x00\x05hello")
  10. 	// or []byte{0x8, 0xc, 0x0, 0x5, 0x68, 0x65, 0x6c, 0x6c, 0x6f}
  12. 	de := gob.NewDecoder(bytes.NewReader(encoded))
  14. 	var test string
  15. 	er := de.Decode(&test)
  16. 	if er != nil {
  17. 		fmt.Println("cannot decode", er)
  18. 		return
  19. 	}
  21. 	fmt.Println("Decoded:", test)
  22. }

答案2

得分: 1

例如，

  1. package main
  3. import (
  4.     "bytes"
  5.     "encoding/gob"
  6.     "fmt"
  7.     "io"
  8. )
  10. func writeGOB(data []byte) (io.Reader, error) {
  11.     buf := new(bytes.Buffer)
  12.     err := gob.NewEncoder(buf).Encode(data)
  13.     return buf, err
  14. }
  16. func readGOB(r io.Reader) ([]byte, error) {
  17.     var data []byte
  18.     err := gob.NewDecoder(r).Decode(&data)
  19.     return data, err
  20. }
  22. func main() {
  23.     var in = []byte("hello")
  24.     fmt.Println(string(in))
  25.     r, err := writeGOB(in)
  26.     if err != nil {
  27.         fmt.Println(err)
  28.         return
  29.     }
  30.     out, err := readGOB(r)
  31.     if err != nil {
  32.         fmt.Println(err)
  33.         return
  34.     }
  35.     fmt.Println(string(out))
  36. }

输出：

  1. hello
  2. hello
英文:

For example,

  1. package main
  3. import (
  4. 	"bytes"
  5. 	"encoding/gob"
  6. 	"fmt"
  7. 	"io"
  8. )
  10. func writeGOB(data []byte) (io.Reader, error) {
  11. 	buf := new(bytes.Buffer)
  12. 	err := gob.NewEncoder(buf).Encode(data)
  13. 	return buf, err
  14. }
  16. func readGOB(r io.Reader) ([]byte, error) {
  17. 	var data []byte
  18. 	err := gob.NewDecoder(r).Decode(&data)
  19. 	return data, err
  20. }
  22. func main() {
  23. 	var in = []byte("hello")
  24. 	fmt.Println(string(in))
  25. 	r, err := writeGOB(in)
  26. 	if err != nil {
  27. 		fmt.Println(err)
  28. 		return
  29. 	}
  30. 	out, err := readGOB(r)
  31. 	if err != nil {
  32. 		fmt.Println(err)
  33. 		return
  34. 	}
  35. 	fmt.Println(string(out))
  36. }

Output:

  1. hello
  2. hello

huangapple
  • 本文由 发表于 2014年3月8日 21:05:39
  • 转载请务必保留本文链接：https://go.coder-hub.com/22269700.html
  • go
  • gob
  • performance
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定