英文:
Store map key/values in a persistent file
问题
我将创建一个类似以下结构的结构体:
type FileState struct {
LastModified int64
Hash string
Path string
}
我想将这些值写入文件,并在后续调用时读取它们。我的初始计划是将它们读入一个map中,并使用键(Path)查找值(Hash和LastModified)。在Go中有没有一种简洁的方法可以做到这一点?
如果没有,你能推荐什么文件格式吗?我在之前的项目中阅读过一些键值文件存储的内容,并进行了一些实验,但没有使用Go。现在,我的要求可能相当简单,所以一个大型数据库服务器系统会过度杀伤力。我只想要一个我可以快速、轻松和可移植地写入和读取的东西(Windows、Mac、Linux)。因为我必须在多个平台上部署,所以我试图将非Go依赖项保持最少。
我考虑过XML、CSV、JSON。我简要地看了一下Go中的gob包,并注意到Go包仪表板上有一个BSON包,但我不确定它们是否适用。
我在这里的主要目标是快速启动,这意味着我需要编写的代码越少越好,部署也要容易。
英文:
I will be creating a structure more or less of the form:
type FileState struct {
LastModified int64
Hash string
Path string
}
I want to write these values to a file and read them in on subsequent calls. My initial plan is to read them into a map and lookup values (Hash and LastModified) using the key (Path). Is there a slick way of doing this in Go?
If not, what file format can you recommend? I have read about and experimented with with some key/value file stores in previous projects, but not using Go. Right now, my requirements are probably fairly simple so a big database server system would be overkill. I just want something I can write to and read from quickly, easily, and portably (Windows, Mac, Linux). Because I have to deploy on multiple platforms I am trying to keep my non-go dependencies to a minimum.
I've considered XML, CSV, JSON. I've briefly looked at the gob package in Go and noticed a BSON package on the Go package dashboard, but I'm not sure if those apply.
My primary goal here is to get up and running quickly, which means the least amount of code I need to write along with ease of deployment.
答案1
得分: 11
只要你的整个数据适合内存,你就不应该有问题。使用内存映射并定期将快照写入磁盘(例如使用gob
包)是一个好主意。Andrew Gerrand在他的《实用Go编程》演讲中使用了这种技术。
如果你需要用不同的程序访问这些文件,使用像json或csv这样的流行编码可能是一个好主意。如果你只需要在Go内部访问这些文件,我会使用优秀的gob包,它有很多不错的功能。
一旦你的数据变得更大,总是在每次更改时将整个数据库写入磁盘就不是一个好主意了。此外,你的数据可能不再适合内存。在这种情况下,你可能想看看由另一位Go开发者Nigel Tao开发的leveldb键值数据库包。它目前正在积极开发中(但尚不可用),但它也将提供一些高级功能,如事务和自动压缩。此外,由于leveldb的设计,读/写吞吐量应该相当不错。
英文:
As long as your entiere data fits in memory, you should't have a problem. Using an in-memory map and writing snapshots to disk regularly (e.g. by using the gob
package) is a good idea. The Practical Go Programming talk by Andrew Gerrand uses this technique.
If you need to access those files with different programs, using a popular encoding like json or csv is probably a good idea. If you just have to access those file from within Go, I would use the excellent gob package, which has a lot of nice features.
As soon as your data becomes bigger, it's not a good idea to always write the whole database to disk on every change. Also, your data might not fit into the RAM anymore. In that case, you might want to take a look at the leveldb key-value database package by Nigel Tao, another Go developer. It's currently under active development (but not yet usable), but it will also offer some advanced features like transactions and automatic compression. Also, the read/write throughput should be quite good because of the leveldb design.
答案2
得分: 2
有一个我写的叫做gkvlite的有序键值持久化库,用于go语言 - https://github.com/steveyen/gkvlite
英文:
There's an ordered, key-value persistence library for the go that I wrote called gkvlite -
https://github.com/steveyen/gkvlite
答案3
得分: 1
JSON非常简单,但由于重复的变量名,会生成更大的文件。XML没有优势。你应该选择CSV,它也非常简单。你的程序将生成少于一页的内容。
但实际上,这取决于你的修改。如果你进行了大量的修改,并且必须将它们同步存储到磁盘上,你可能需要比单个文件更复杂的东西。如果你的地图主要是只读的,或者如果你可以偶尔将其转储到文件中(不是每秒都转储),一个单独的CSV文件和一个内存中的地图将保持事物简单和高效。
顺便说一下,使用Go语言的csv包来完成这个任务。
英文:
JSON is very simple but makes bigger files because of the repeated variable names. XML has no advantage. You should go with CSV, which is really simple too. Your program will make less than one page.
But it depends, in fact, upon your modifications. If you make a lot of modifications and must have them stored synchronously on disk, you may need something a little more complex that a single file. If your map is mainly read-only or if you can afford to dump it on file rarely (not every second) a single csv file along an in-memory map will keep things simple and efficient.
BTW, use the csv package of go to do this.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论