为什么 leveldb 错误只在 10% 的测试运行中出现，如何解决这个问题？

huangapple
go评论151阅读模式
英文:

Why is leveldb error appears only in 10% of test runs and how to solve it?

问题

我正在使用go版本的leveldb来存储我的数据。我编写了一些简单的辅助函数来操作数据库。同时，我为它们编写了一个测试（如下所示）。但是测试在大约10%的运行中会抛出错误（以下是堆栈跟踪）。这个错误的原因是什么，我该如何解决它？

辅助函数：

  1. package data
  3. import (
  4. 	"path"
  5. 	"runtime"
  7. 	"github.com/syndtr/goleveldb/leveldb"
  8. )
  10. var base = openDB()
  12. func openDB() *leveldb.DB {
  13. 	_, filename, _, _ := runtime.Caller(0)
  14. 	dbPath := path.Dir(filename) + "/leveldb"
  15. 	db, _ := leveldb.OpenFile(dbPath, nil)
  16. 	return db
  17. }
  19. // 从数据库中根据键获取值
  20. func Get(key []byte) []byte {
  21. 	output, getErr := base.Get(key, nil)
  22. 	if getErr != nil {
  23. 		return nil
  24. 	}
  25. 	return output
  26. }
  28. // 将键和值放入数据库（如果值存在，请使用Change()函数代替）
  29. func Put(key []byte, value []byte) {
  30. 	valueExists, _ := base.Has(key, nil)
  31. 	if valueExists {
  32. 		return
  33. 	}
  34. 	base.Put(key, value, nil)
  35. }
  37. // 此函数仅用于在测试后删除值，请勿在其他情况下调用它
  38. func TestRM(key []byte) {
  39. 	base.Delete(key, nil)
  40. }

测试：

  1. func TestPutValue(t *testing.T) {
  2. 	key := []byte{1, 2, 3}
  3. 	val := []byte{1, 2, 3, 4, 5}
  4. 	Put(key, val)
  5. 	val2 := Get(key)
  6. 	if !reflect.DeepEqual(val, val2) {
  7. 		t.Error("values are not equal")
  8. 	}
  9. 	TestRM(key)
  10. }

堆栈跟踪：

  1. --- FAIL: TestPutValue (0.00s)
  2. panic: runtime error: invalid memory address or nil pointer dereference [recovered]
  3. 	panic: runtime error: invalid memory address or nil pointer dereference
  4. [signal SIGSEGV: segmentation violation code=0x1 addr=0x188 pc=0x1149a7c]
  6. goroutine 6 [running]:
  7. testing.tRunner.func1.2(0x1188020, 0x12d05c0)
  8. 	/usr/local/go/src/testing/testing.go:1143 +0x332
  9. testing.tRunner.func1(0xc000001380)
  10. 	/usr/local/go/src/testing/testing.go:1146 +0x4b6
  11. panic(0x1188020, 0x12d05c0)
  12. 	/usr/local/go/src/runtime/panic.go:965 +0x1b9
  13. github.com/syndtr/goleveldb/leveldb.(*DB).isClosed(...)
  14. 	/Users/danilafominyh/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db_state.go:230
  15. github.com/syndtr/goleveldb/leveldb.(*DB).ok(...)
  16. 	/Users/danilafominyh/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db_state.go:235
  17. github.com/syndtr/goleveldb/leveldb.(*DB).Has(0x0, 0xc00001c1a8, 0x3, 0x3, 0x0, 0xc000038600, 0x0, 0x0)
  18. 	/Users/danilafominyh/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db.go:852 +0x5c
  19. sync_tree/data.Put(0xc00001c1a8, 0x3, 0x3, 0xc00001c1ab, 0x5, 0x5)
  20. 	/Users/danilafominyh/Documents/sync_tree_server/data/database.go:31 +0x58
  21. sync_tree/data.TestPutValue(0xc000001380)
  22. 	/Users/danilafominyh/Documents/sync_tree_server/data/database_test.go:11 +0xb3
  23. testing.tRunner(0xc000001380, 0x11c4b68)
  24. 	/usr/local/go/src/testing/testing.go:1193 +0xef
  25. created by testing.(*T).Run
  26. 	/usr/local/go/src/testing/testing.go:1238 +0x2b3
  27. FAIL	sync_tree/data	0.167s
  28. FAIL
  29. Error: Tests failed.
英文:

I am using go version of leveldb to store my data. I made some simple helper functions to manipulate the database. Also I wrote a test for them (below). But the test is throwing error in around 10% runs (stack trace below). What is the reason of that error, and how can I solve it?

Helpers:

  1. package data
  3. import (
  4. 	"path"
  5. 	"runtime"
  7. 	"github.com/syndtr/goleveldb/leveldb"
  8. )
  10. var base = openDB()
  12. func openDB() *leveldb.DB {
  13. 	_, filename, _, _ := runtime.Caller(0)
  14. 	dbPath := path.Dir(filename) + "/leveldb"
  15. 	db, _ := leveldb.OpenFile(dbPath, nil)
  16. 	return db
  17. }
  19. // get value by key from database
  20. func Get(key []byte) []byte {
  21. 	output, getErr := base.Get(key, nil)
  22. 	if getErr != nil {
  23. 		return nil
  24. 	}
  25. 	return output
  26. }
  28. // put key by some value to database (if value exists use Change()
  29. // func instead)
  30. func Put(key []byte, value []byte) {
  31. 	valueExists, _ := base.Has(key, nil)
  32. 	if valueExists {
  33. 		return
  34. 	}
  35. 	base.Put(key, value, nil)
  36. }
  38. // function is made only to remove values after testing, dont call it in
  39. // any other case
  40. func TestRM(key []byte) {
  41. 	base.Delete(key, nil)
  42. }

Test:

  2. func TestPutValue(t *testing.T) {
  3. 	key := []byte{1, 2, 3}
  4. 	val := []byte{1, 2, 3, 4, 5}
  5. 	Put(key, val)
  6. 	val2 := Get(key)
  7. 	if !reflect.DeepEqual(val, val2) {
  8. 		t.Error("values are not equal")
  9. 	}
  10. 	TestRM(key)
  11. }

StackTrace:

  1. --- FAIL: TestPutValue (0.00s)
  2. panic: runtime error: invalid memory address or nil pointer dereference [recovered]
  3. 	panic: runtime error: invalid memory address or nil pointer dereference
  4. [signal SIGSEGV: segmentation violation code=0x1 addr=0x188 pc=0x1149a7c]
  6. goroutine 6 [running]:
  7. testing.tRunner.func1.2(0x1188020, 0x12d05c0)
  8. 	/usr/local/go/src/testing/testing.go:1143 +0x332
  9. testing.tRunner.func1(0xc000001380)
  10. 	/usr/local/go/src/testing/testing.go:1146 +0x4b6
  11. panic(0x1188020, 0x12d05c0)
  12. 	/usr/local/go/src/runtime/panic.go:965 +0x1b9
  13. github.com/syndtr/goleveldb/leveldb.(*DB).isClosed(...)
  14. 	/Users/danilafominyh/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db_state.go:230
  15. github.com/syndtr/goleveldb/leveldb.(*DB).ok(...)
  16. 	/Users/danilafominyh/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db_state.go:235
  17. github.com/syndtr/goleveldb/leveldb.(*DB).Has(0x0, 0xc00001c1a8, 0x3, 0x3, 0x0, 0xc000038600, 0x0, 0x0)
  18. 	/Users/danilafominyh/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db.go:852 +0x5c
  19. sync_tree/data.Put(0xc00001c1a8, 0x3, 0x3, 0xc00001c1ab, 0x5, 0x5)
  20. 	/Users/danilafominyh/Documents/sync_tree_server/data/database.go:31 +0x58
  21. sync_tree/data.TestPutValue(0xc000001380)
  22. 	/Users/danilafominyh/Documents/sync_tree_server/data/database_test.go:11 +0xb3
  23. testing.tRunner(0xc000001380, 0x11c4b68)
  24. 	/usr/local/go/src/testing/testing.go:1193 +0xef
  25. created by testing.(*T).Run
  26. 	/usr/local/go/src/testing/testing.go:1238 +0x2b3
  27. FAIL	sync_tree/data	0.167s
  28. FAIL
  29. Error: Tests failed.

答案1

得分: 1

很可能在db, _ := leveldb.OpenFile(dbPath, nil)这一行中，应用程序出现了错误，但被忽略了。要么在openDB函数中，当leveldb.OpenFile返回错误时引发panic，要么返回错误信息。

  1. func openDB() *leveldb.DB {
  2. 	_, filename, _, _ := runtime.Caller(0)
  3. 	dbPath := path.Dir(filename) + "/leveldb"
  4. 	db, err := leveldb.OpenFile(dbPath, nil)
  5. 	if err != nil {
  6. 		panic(fmt.Errorf("无法打开leveldb数据库：%w", err))
  7. 	}
  8. 	return db
  9. }

免责声明：这并不能解决问题，但它会给你一个线索，指出问题出在哪里。

英文:

Most likely the app gets an error in db, _ := leveldb.OpenFile(dbPath, nil) line which is ignored. Make openDB either panic when leveldb.OpenFile returns an error or return an error

  1. func openDB() *leveldb.DB {
  2. 	_, filename, _, _ := runtime.Caller(0)
  3. 	dbPath := path.Dir(filename) + "/leveldb"
  4. 	db, err := leveldb.OpenFile(dbPath, nil)
  5. 	if err != nil {
  6. 		panic(fmt.Errorf("could not open leveldb database: %w", err))
  7. 	}
  8. 	return db
  9. }

Disclaimer: It's not going to solve the issue, but it will give you a clue what is wrong.

huangapple
  • 本文由 发表于 2021年9月17日 01:16:03
  • 转载请务必保留本文链接：https://go.coder-hub.com/69212620.html
  • go
  • leveldb
  • testing
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定