etcd3 Go客户端 – 如何对大量的键进行分页处理?

huangapple go评论199阅读模式
英文:

etcd3 Go Client - How to paginate large sets of keys?

问题

似乎在处理大量键的分页时,需要使用WithFromKey()和WithLimit()选项来进行Get()操作。例如,如果我想获取2页共10个项目,可以这样做:

  1. opts := []clientv3.OpOption {
  2. clientv3.WithPrefix(),
  3. clientv3.WithSort(clientv3.SortByKey, clientv3.SortAscend),
  4. clientv3.WithLimit(10),
  5. }
  6. gr, err := kv.Get(ctx, "key", opts...)
  7. if err != nil {
  8. log.Fatal(err)
  9. }
  10. fmt.Println("--- 第一页 ---")
  11. for _, item := range gr.Kvs {
  12. fmt.Println(string(item.Key), string(item.Key))
  13. }
  14. lastKey := string(gr.Kvs[len(gr.Kvs)-1].Value)
  15. fmt.Println("--- 第二页 ---")
  16. opts = append(opts, clientv3.WithFromKey())
  17. gr, _ = kv.Get(ctx, lastKey, opts...)
  18. // 跳过第一个项目,即前一个Get的最后一个项目
  19. for _, item := range gr.Kvs[1:] {
  20. fmt.Println(string(item.Key), string(item.Value))
  21. }

问题在于最后一个键再次作为第二页的第一个项目被获取到,我需要跳过它,只获取9个新项目。

这是正确的分页方式吗?还是我漏掉了什么?

英文:

It seems that pagination through a large set of keys involve using WithFromKey() and WithLimit() options to Get(). For example if I want to fetch 2 pages of 10 items I would do something like:

  1. opts := []clientv3.OpOption {
  2. clientv3.WithPrefix(),
  3. clientv3.WithSort(clientv3.SortByKey, clientv3.SortAscend),
  4. clientv3.WithLimit(10),
  5. }
  6. gr, err := kv.Get(ctx, "key", opts...)
  7. if err != nil {
  8. log.Fatal(err)
  9. }
  10. fmt.Println("--- First page ---")
  11. for _, item := range gr.Kvs {
  12. fmt.Println(string(item.Key), string(item.Key))
  13. }
  14. lastKey := string(gr.Kvs[len(gr.Kvs)-1].Value)
  15. fmt.Println("--- Second page ---")
  16. opts = append(opts, clientv3.WithFromKey())
  17. gr, _ = kv.Get(ctx, lastKey, opts...)
  18. // Skipping the first item, which the last item from from the previous Get
  19. for _, item := range gr.Kvs[1:] {
  20. fmt.Println(string(item.Key), string(item.Value))
  21. }

The problem is that the last key is fetched agains as the first item of the second page, which I need to skip and only 9 new items.

Is that the proper way to paginate or am I missing something?

答案1

得分: 1

在查看以下的clientv3代码时,下一个键可以通过在最后一个键后面添加0x00来计算。

https://github.com/coreos/etcd/blob/88acced1cd7ad670001d1280b97de4fe7b647687/clientv3/op.go#L353

话虽如此,我更喜欢忽略第一个键,从第二个和后续页面开始。

英文:

Looking at the following clientv3 code, the next key can be computed by appending 0x00 to the last key.

https://github.com/coreos/etcd/blob/88acced1cd7ad670001d1280b97de4fe7b647687/clientv3/op.go#L353

That said, I prefer ignoring the first key from the 2nd and following pages.

huangapple
  • 本文由 发表于 2017年7月3日 01:38:13
  • 转载请务必保留本文链接:https://go.coder-hub.com/44873514.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定