循环遍历 MongoDB 中的集合,并解码为不同的 Go 结构体。

huangapple go评论87阅读模式
英文:

Loop over collections in mongodb and decode to various go structs

问题

我有一系列的集合在一个mongodb实例中,我想要获取所有的集合并解码集合中的所有文档,以便在处理过程中对它们进行修改。

我正在使用go语言列出集合的名称:

...
collections, err := db.ListCollectionNames(context.TODO(), bson.M{})
if err != nil {
	log.Fatalf("Failed to get coll names: %v", err)
}

for _, coll := range collections {
	collection := db.Collection(coll)
...

这段代码运行得很顺利,下一步是打开每个集合的游标,并循环遍历游标,对每个对象调用解码操作,将其解码为相应的类型。

我已经在一个types包中定义了所有的类型:

type Announcements struct {
	Id            string    `bson:"_id"`
	Content       string    `bson:"content"`
	Title         string    `bson:"title"`
	DateModified  time.Time `bson:"dateModified,omitempty"`
	ModifiedBy    string    `bson:"modifiedBy,omitempty"`
}

问题是我无法动态定义用于解码的变量。

这段代码需要为每个集合重复执行:

var obj Announcements // 这需要是动态的

for cursor.Next(context.Background()) {
	err := cursor.Decode(&obj)
	if err != nil {
		log.Fatal(err)
	}

	log.Printf("%#v\n", obj)
}

有没有办法在不重复多次的情况下实现这个功能?我意识到使用动态类型的语言可能更适合这个任务。在将工作迁移到脚本语言之前,我想先询问一下。

我尝试使用reflect包和switch语句来动态实例化变量,并使用interface{}类型,但这会强制解码使用bson类型而不是我定义的结构体。

-- 编辑 --

我尝试使用一个映射将集合名称与类型关联起来,但没有成功。

var Collections = map[string]interface{}{
	"announcements":          Announcements{},
	"badges":                 Badges{},
	"betaUser":               BetaUser{},
	"bonusLearningItems":     BonusLearningItems{},
	"books":                  Books{},
...
}

这样当我使用obj变量时,我尝试进行如下赋值:

var obj types.Collections[coll]

希望如果我循环到了announcements集合,这将给我一个Announcement类型的变量,但是当我调用decode时,它返回的是bson类型。

我需要动态定义obj变量的类型。

英文:

I have a series of collections in a mongodb instance where I want to pull all the collections and decode all documents in the collections so I can mutate them as I go.

I am using go to list the collection names:

...
collections, err := db.ListCollectionNames(context.TODO(), bson.M{})
if err != nil {
	log.Fatalf("Failed to get coll names: %v", err)
}

for _, coll := range collections {
	collection := db.Collection(coll)
...

This works flawlessly and the next step would be to open a cursor to each collection and loop over the cursor and call decode on each object to its corresponding type.

I have defined all the types in a types package:

type Announcements struct {
	Id            string    `bson:"_id"`
	Content       string    `bson:"content"`
	Title         string    `bson:"title"`
	DateModified  time.Time `bson:"dateModified,omitempty"`
	ModifiedBy    string    `bson:"modifiedBy,omitempty"`
}

The issue is that I cant define the variable to be used for decoding dynamically.

This code would need to be repeated for every single collection

var obj Announcements // This would need to be dynamic

for cursor.Next(context.Background()) {
	err := cursor.Decode(&obj)
	if err != nil {
		log.Fatal(err)
	}

	log.Printf("%#v\n", obj)
}

Is there a way to do this without repeating this many times? I realize a dynamically typed language would be better for this. Wanted to ask before I migrate the work to a scripting language.

I have tried using the reflect package and switch statements to instantiate the variable dynamnically and using the interface{} type but that force the decode to use bson types and not my defined structs.

-- edit --

I have tried to use a map to link collection names to types but to no avail.

var Collections = map[string]interface{}{
	"announcements":          Announcements{},
	"badges":                 Badges{},
	"betaUser":               BetaUser{},
	"bonusLearningItems":     BonusLearningItems{},
	"books":                  Books{},
...
}

So that when I use the obj var I tried an assignment like:

var obj types.Collections[coll]

hoping that would give me a variable of type Announcement if I was had looped to the announcements collection. but when I call decode it returns bson types.

I need to dynamically define the obj variable type.

答案1

得分: 0

有很多方法可以做到这一点。其中之一是使用回调函数:

func Process(cursor *mongo.Cursor, cb func(), out interface{}) {
  for cursor.Next(context.Background()) {
     err := cursor.Decode(out)
     if err != nil {
         log.Fatal(err)
     }
     cb()
     log.Printf("%#v\n", obj)
  }
}

然后定义一个映射:

type collHandler struct {
   New func() interface{}
   Process func(interface{})
}

var collections = map[string]func() interface{} {
   "announcements":   collHandler{
     New: func() interface {} {return &Announcement{}},
     Process: func(data interface{}) {
        processAnnouncements(data.(*Announcement))
     },
   },
   ...
}

然后你可以这样做:

handler := collections[collectionName]
obj := handler.New()
process(cursor, func() {
   handler.Process(obj)
  },
  &obj)

这样,你可以将迭代放入一个公共函数中,并使用闭包来处理特定类型的逻辑。

英文:

There are many ways you can do this. One of them is by using a callback function:

func Process(cursor *mongo.Cursor, cb func(), out interface{}) {
  for cursor.Next(context.Background()) {
     err := cursor.Decode(out)
     if err != nil {
         log.Fatal(err)
     }
     cb()
     log.Printf("%#v\n", obj)
  }
}

Then define a map:

type collHandler struct {
   New func() interface{}
   Process func(interface{})
}


var collections=map[string]func() interface{} {
   "announcements":   collHandler{
     New: func() interface {} {return &Announcement{}},
     Process: func(data interface{}) {
        processAnnouncements(data.(*Announcement))
     },
   },
   ...
}

And then you can:

handler:=collections[collectionName]
obj:=handler.New()
process(cursor,func() {
   handler.Process(obj)
  },
  &obj)

This way, you can place the iteration into a common function, and use closures to deal with type specific logic.

huangapple
  • 本文由 发表于 2021年9月1日 01:34:42
  • 转载请务必保留本文链接:https://go.coder-hub.com/69003108.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定