在Golang中使用MongoDB,如何在指定的秒数后使文档过期?

huangapple go评论71阅读模式
英文:

Using mongo in golang, Expire Documents after a Specified Number of Seconds?

问题

我正在尝试使用mongo-go-driver做一些简单的事情。
我在一个集合中插入了一些数据,并希望它们在一定时间后自动删除。

我已经阅读了以下文档:https://docs.mongodb.com/manual/tutorial/expire-data/#expire-documents-after-a-specified-number-of-seconds

然后我用GO写了一些代码,但它似乎不像我预期的那样工作。也许有些地方我没有理解对,或者我做错了。

package main

import (
	"bytes"
	"context"
	"fmt"
	"log"
	"text/tabwriter"
	"time"

	"github.com/Pallinder/go-randomdata"
	"go.mongodb.org/mongo-driver/bson"
	"go.mongodb.org/mongo-driver/bson/primitive"
	"go.mongodb.org/mongo-driver/mongo"
	"go.mongodb.org/mongo-driver/mongo/options"
)

func main() {
	ctx := context.TODO()

	client, err := mongo.NewClient(options.Client().ApplyURI("mongodb://localhost:27017"))
	if err != nil {
		log.Fatal(err)
	}
	err = client.Connect(ctx)
	if err != nil {
		log.Fatal(err)
	}

	db := client.Database("LADB")
	col := db.Collection("LACOLL")

	// 为col添加索引
	// 目标是设置数据的TTL为1秒(测试目的)
	model := mongo.IndexModel{
		Keys:    bson.M{"createdAt": 1},
		Options: options.Index().SetExpireAfterSeconds(1),
	}
	ind, err := col.Indexes().CreateOne(ctx, model)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(ind)

	// 每秒插入一些数据
	for i := 0; i < 5; i++ {
		name := randomdata.SillyName()
		res, err := col.InsertOne(ctx, NFT{Timestamp: time.Now(), CreatedAt: time.Now(), Name: name})
		if err != nil {
			log.Fatal(err)
		}
		fmt.Println("Inserted", name, "with id", res.InsertedID)
		time.Sleep(1 * time.Second)
	}

	// 显示所有数据
	cursor, err := col.Find(ctx, bson.M{}, nil)
	if err != nil {
		log.Fatal(err)
	}
	var datas []NFT
	if err = cursor.All(ctx, &datas); err != nil {
		log.Fatal(err)
	}

	// 我期望有些数据不在那里(少于5个)
	fmt.Println(datas)
}

type NFT struct {
	ID        primitive.ObjectID `bson:"_id,omitempty"`
	CreatedAt time.Time          `bson:"createdAt,omitempty"`
	Timestamp time.Time          `bson:"timestamp,omitempty"`
	Name      string             `bson:"name,omitempty"`
}
英文:

I am trying something simple using the mongo-go-driver.
I insert some datas in a collection, and I want them to be automaticaly deleted after a number of seconds.

I have read the following documentation : https://docs.mongodb.com/manual/tutorial/expire-data/#expire-documents-after-a-specified-number-of-seconds

Then I have wrote something in GO, but it does not seems to work as I expected. Maybe there is something I did not get, or I am doing the wrong way.

package main

import (
	&quot;bytes&quot;
	&quot;context&quot;
	&quot;fmt&quot;
	&quot;log&quot;
	&quot;text/tabwriter&quot;
	&quot;time&quot;

	&quot;github.com/Pallinder/go-randomdata&quot;
	&quot;go.mongodb.org/mongo-driver/bson&quot;
	&quot;go.mongodb.org/mongo-driver/bson/primitive&quot;
	&quot;go.mongodb.org/mongo-driver/mongo&quot;
	&quot;go.mongodb.org/mongo-driver/mongo/options&quot;
)

func main() {
	ctx := context.TODO()

	client, err := mongo.NewClient(options.Client().ApplyURI(&quot;mongodb://localhost:27017&quot;))
	if err != nil {
		log.Fatal(err)
	}
	err = client.Connect(ctx)
	if err != nil {
		log.Fatal(err)
	}

	db := client.Database(&quot;LADB&quot;)
	col := db.Collection(&quot;LACOLL&quot;)

	// add index to col
    // the goal is to set a TTL for datas to only 1 secondes (test purpose)
	model := mongo.IndexModel{
		Keys:    bson.M{&quot;createdAt&quot;: 1},
		Options: options.Index().SetExpireAfterSeconds(1),
	}
	ind, err := col.Indexes().CreateOne(ctx, model)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(ind)

	// insert some datas each seconds
	for i := 0; i &lt; 5; i++ {
		name := randomdata.SillyName()
		res, err := col.InsertOne(ctx, NFT{Timestamp: time.Now(), CreatedAt: time.Now(), Name: name})
		if err != nil {
			log.Fatal(err)
		}
		fmt.Println(&quot;Inserted&quot;, name, &quot;with id&quot;, res.InsertedID)
		time.Sleep(1 * time.Second)
	}

	// display all
	cursor, err := col.Find(ctx, bson.M{}, nil)
	if err != nil {
		log.Fatal(err)
	}
	var datas []NFT
	if err = cursor.All(ctx, &amp;datas); err != nil {
		log.Fatal(err)
	}

	// I expect some datas not to be there (less than 5)
	fmt.Println(datas)
}

type NFT struct {
	ID        primitive.ObjectID `bson:&quot;_id,omitempty&quot;`
	CreatedAt time.Time          `bson:&quot;createdAt,omitempty&quot;`
	Timestamp time.Time          `bson:&quot;timestamp,omitempty&quot;`
	Name      string             `bson:&quot;name,omitempty&quot;`
}

答案1

得分: 5

你的示例没有问题,它可以正常工作。

请注意,你指定的expireAfterSeconds是文档在createdAt后过期的持续时间,而该时刻是文档可能被删除的最早时间,但不能保证删除会在那个时间点“立即”发生。

引用自MongoDB文档:TTL索引:删除操作的时间

TTL索引不能保证在过期后立即删除过期数据。在文档过期和MongoDB从数据库中删除文档之间可能会有延迟。

删除过期文档的后台任务每60秒运行一次。因此,在文档过期和后台任务运行之间的时间段内,文档可能仍然存在于集合中。

由于删除操作的持续时间取决于mongod实例的工作负载,过期数据可能会在后台任务运行之间的60秒期间之后的一段时间内存在。

正如你所看到的,如果文档过期,最坏的情况下可能需要60秒才能启动后台任务并开始删除过期文档,如果有很多文档(或者数据库负载很重),删除所有过期文档可能需要一些时间。

英文:

There's nothing wrong with your example, it works.

Please note that the expireAfterSeconds you specify is the duration after createdAt when the document expires, and that instant is the earliest time at which the document may be deleted, but there is no guarantee that the deletion will happen "immediately", exactly at that time.

Quoting from MongoDB docs: TTL indexes: Timing of the Delete Operation:

> The TTL index does not guarantee that expired data will be deleted immediately upon expiration. There may be a delay between the time a document expires and the time that MongoDB removes the document from the database.
>
> The background task that removes expired documents runs every 60 seconds. As a result, documents may remain in a collection during the period between the expiration of the document and the running of the background task.
>
> Because the duration of the removal operation depends on the workload of your mongod instance, expired data may exist for some time beyond the 60 second period between runs of the background task.

As you can see, if a document expires, at worst case it may take 60 seconds for the background task to kick in and start removing expired documents, and if there are many (or the database is under heavy load), it may take some time to delete all expired documents.

huangapple
  • 本文由 发表于 2021年10月20日 19:41:45
  • 转载请务必保留本文链接:https://go.coder-hub.com/69645345.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定