What is the appropriate way to multithread LevelDB in Go?

huangapple go评论85阅读模式
英文:

What is the appropriate way to multithread LevelDB in Go?

问题

我在我的项目中实现了levigo包装器,以便可以使用LevelDB。声明部分相当标准,如下所示:

func NewLeveldbStorage(dbPath string) *leveldbStorage {
    opts := levigo.NewOptions()
    opts.SetCache(levigo.NewLRUCache(3<<30))
    opts.SetCreateIfMissing(true)
    log.Debugf("Entering Open")
    db, err := levigo.Open(dbPath, opts)
    if err != nil {
        log.Fatal("BOOM %v", err)
    }
    log.Debugf("Finished calling open")
    opts.Close()
    return &leveldbStorage{db: db}
}

这是返回的结构体:

type leveldbStorage struct {
    db *levigo.DB
}

然后,我在这个结构体上进行了一些简单的GET和STORE命令,实际上只是使用s.db.Gets.db.Put。这在我的测试中运行良好,但是当我运行以下基准测试时:

func BenchmarkLeviDbGet(b *testing.B) {
    s := storage.NewLeveldbStorage("/path/to/db")
    value := "value"
    uid, _ := s.Store(value)

    b.ResetTimer()
    for i := 0; i < b.N; i++ {
        s.Get(uid)
    }
}

运行该基准测试时,返回以下错误信息:

2014/10/12 21:17:09 BOOM %vIO error: lock /path/to/db/LOCK: already held by process

有没有适当的方法可以使用levigo/leveldb实现多线程读取?写入呢?如果多线程写入不可行,我不会感到惊讶,但是多线程读取似乎是可行的。我在这里做错了什么?

英文:

I have implemented the levigo wrapper in my project so I can use LevelDB. The declaration is fairly boilerplate, like so:

func NewLeveldbStorage(dbPath string) *leveldbStorage {
     opts := levigo.NewOptions()
     opts.SetCache(levigo.NewLRUCache(3&lt;&lt;30))
     opts.SetCreateIfMissing(true)
     log.Debugf(&quot;Entering Open&quot;)
     db, err := levigo.Open(dbPath, opts); if err != nil {
         log.Fatal(&quot;BOOM %v&quot;, err)
     }
     log.Debugf(&quot;Finished calling open&quot;)
     opts.Close()
     return &amp;leveldbStorage{db:db}
 }

Here is the struct returned:

type leveldbStorage struct {
     db *levigo.DB
}

I then made a few simple GET and STORE commands on the struct that essentially just use s.db.Get and s.db.Put. This works fine in my tests, but when I run the following benchmark:

 func BenchmarkLeviDbGet(b *testing.B) {
     s := storage.NewLeveldbStorage(&quot;/path/to/db&quot;)
     value := &quot;value&quot;
     uid,_ := s.Store(value)

     b.ResetTimer()
     for i := 0; i &lt; b.N; i++ {
         s.Get(uid)
     }

This benchmark, when run, returns:

 2014/10/12 21:17:09 BOOM %vIO error: lock /path/to/db/LOCK: already held by process

Is there an appropriate way to use levigo/leveldb to enable multithreaded reading? What about writing? I would not be surprised if multithreaded writing is not possible, but multithreaded reading seems like it should be. What am I doing wrong here?

答案1

得分: 2

你需要关闭数据库文件或者使用一个全局实例来操作它,不能多次打开同一个文件,但是你可以在多个 goroutine 中访问同一个实例。

英文:

You either need to close the database file or use a global instance to it, you can't open the file multiple times, you can however access the same instance from multiple goroutines.

huangapple
  • 本文由 发表于 2014年10月13日 05:33:25
  • 转载请务必保留本文链接:https://go.coder-hub.com/26330216.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定