英文:
Should I copy session for each operation in mgo?
问题
我想要upsert
一组记录,所以我有两个选择,一个是只使用一个会话,另一个是为每个记录复制一个会话。根据我的观点,第一种方法可能比第二种方法慢,但是第一种方法会导致创建太多的会话吗?
- 使用一个会话
func (this *CvStoreServiceImpl) SetCvJobItemMeasureList(accessToken *base_datatype.ServiceAccessToken, versionPolicy string, jobItemList []*cv_common_type.CvJobItemMeasure) (err error) {
session := this.session.Clone()
defer session.Close()
for _, jobItem := range jobItemList {
objKey := &orm.ItemIdKey{
VersionName: versionPolicy, //XXX
ItemId: jobItem.ItemId,
}
obj := orm.ConvertToCvJobItemMeasureObj(versionPolicy, jobItem)
_, err2 := this.jobMeasureCollection.With(session).Upsert(objKey, obj)
if nil != err2 {
err = &common_error.NamedError{err2.Error()}
this.logger.Println(err2.Error())
}
}
return
}
- 为每个记录复制一个会话
func (this *CvStoreServiceImpl) SetCvJobItemMeasure(accessToken *base_datatype.ServiceAccessToken, versionPolicy string, jobItem *cv_common_type.CvJobItemMeasure) (err error) {
session := this.session.Clone()
defer session.Close()
objKey := &orm.ItemIdKey{
VersionName: versionPolicy, //XXX
ItemId: jobItem.ItemId,
}
obj := orm.ConvertToCvJobItemMeasureObj(versionPolicy, jobItem)
_, err2 := this.jobMeasureCollection.With(session).Upsert(objKey, obj)
if nil != err2 {
err = &common_error.NamedError{err2.Error()}
return
}
return
}
然后在循环中调用这个方法:
for _, item := range cvMeasure.GetJobList() {
err = this.SetCvJobItemMeasure(accessToken, versionPolicy, item)
if nil != err {
return
}
}
英文:
I want to upsert
a list of record, so I have two choice, one just use one session, another copy a session for every record. So, as my opinion, first method may slower than the second, but will the first one cause too many session created?
1.use one session
func (this *CvStoreServiceImpl) SetCvJobItemMeasureList(accessToken *base_datatype.ServiceAccessToken, versionPolicy string, jobItemList []*cv_common_type.CvJobItemMeasure) (err error) {
session := this.session.Clone()
defer session.Close()
for _, jobItem := range jobItemList {
objKey := &orm.ItemIdKey{
VersionName: versionPolicy, //XXX
ItemId: jobItem.ItemId,
}
obj := orm.ConvertToCvJobItemMeasureObj(versionPolicy, jobItem)
_, err2 := this.jobMeasureCollection.With(session).Upsert(objKey, obj)
if nil != err2 {
err = &common_error.NamedError{err2.Error()}
this.logger.Println(err2.Error())
}
}
return
}
2.copy session for every record
func (this *CvStoreServiceImpl) SetCvJobItemMeasure(accessToken *base_datatype.ServiceAccessToken, versionPolicy string, jobItem *cv_common_type.CvJobItemMeasure) (err error) {
session := this.session.Clone()
defer session.Close()
objKey := &orm.ItemIdKey{
VersionName: versionPolicy, //XXX
ItemId: jobItem.ItemId,
}
obj := orm.ConvertToCvJobItemMeasureObj(versionPolicy, jobItem)
_, err2 := this.jobMeasureCollection.With(session).Upsert(objKey, obj)
if nil != err2 {
err = &common_error.NamedError{err2.Error()}
return
}
return
}
then call this method in forloop:
for _, item := range cvMeasure.GetJobList() {
err = this.SetCvJobItemMeasure(accessToken, versionPolicy, item)
if nil != err {
return
}
}
答案1
得分: 14
首先,我们需要看一下mgo.Session.Copy()和mgo.Session.Clone()之间的区别。go.Session.Clone()
返回一个新的会话,但该会话使用相同的套接字连接。这并不一定是件坏事,但请记住,在服务器端,每个连接都会分配一个堆栈。因此,这些会话将共享相同的堆栈。根据您的用例,这可能会产生很大的差异。
这里的问题是,如果您为每个记录打开一个新的套接字连接,这将导致三次握手,速度较慢。重用相同的套接字可以减少这种开销,但仍然存在一些问题,如上所述。
我倾向于在每个长时间运行的工作单元中建立一个新的连接。一个简单的示例说明了这一点:
package main
import (
"fmt"
mgo "gopkg.in/mgo.v2"
bson "gopkg.in/mgo.v2/bson"
"net/http"
)
var (
Database *mgo.Database
)
// listPosts列出所有帖子
func listPosts(w http.ResponseWriter, r *http.Request) {
// 我们有一个相当长时间运行的工作单元
// (读取和列出所有帖子)
// 因此值得复制会话
collection := Database.C("posts").With(Database.Session.Copy())
post := bson.D{}
posts := collection.Find(bson.M{}).Iter()
for posts.Next(&post) {
// 处理帖子并将其发送到w
}
}
func main() {
session, _ := mgo.Dial("mongodb://localhost:27017")
Database := session.DB("myDb")
// Count是一个相当快的操作
// 这里不需要复制会话
count, _ := Database.C("posts").Count()
fmt.Printf("数据库中当前有%d个帖子", count)
http.HandleFunc("/posts", listPosts)
http.ListenAndServe(":8080", nil)
}
英文:
First of all, we need to see the difference between mgo.Session.Copy() and mgo.Session.Clone(). While go.Session.Clone()
returns a new session, the session uses the same socket connection. That isn't necessarily a bad thing, but keep in mind that on the server side, a stack is allocated per connection. So the sessions would share the same stack. Depending on your use cases, that may make a big difference.
And here is the problem – if you open a new socket connect for each record, this leads to a three way handshake, which is slowish. Reusing the same socket reduces this overhead, but there still is some and has the drawback described above.
What I tend to do is to establish a new connection per long(er) running unit of work. A simple example illustrates this:
package main
import (
"fmt"
mgo "gopkg.in/mgo.v2"
bson "gopkg.in/mgo.v2/bson"
"net/http"
)
var (
Database *mgo.Database
)
// The listEntries lists all posts
func listPosts(w http.ResponseWriter, r *http.Request) {
// We have a rather long running unit of work
// (reading and listing all posts)
// So it is worth copying the session
collection := Database.C("posts").With( Database.Session.Copy() )
post := bson.D{}
posts := collection.Find(bson.M{}).Iter()
for posts.Next(&post) {
// Process posts and send it to w
}
}
func main() {
session, _ := mgo.Dial("mongodb://localhost:27017")
Database := session.DB("myDb")
// Count is a rather fast operation
// No need to copy the session here
count, _ := Database.C( "posts" ).Count()
fmt.Printf("Currently %d posts in the database", count )
http.HandleFunc("/posts", listPosts)
http.ListenAndServe(":8080", nil)
}
答案2
得分: -1
是的,将会话复制以执行一个或多个操作是很好的,这样可以让mgo中的连接池提高性能。默认情况下,一个Mongo服务器的连接限制是4096,以防止连接过多。
func newSession(consistency Mode, cluster *mongoCluster, timeout time.Duration) (session *Session) {
cluster.Acquire()
session = &Session{
cluster_: cluster,
syncTimeout: timeout,
sockTimeout: timeout,
poolLimit: 4096,
}
debugf("New session %p on cluster %p", session, cluster)
session.SetMode(consistency, true)
session.SetSafe(&Safe{})
session.queryConfig.prefetch = defaultPrefetch
return session
}
英文:
yes, it is good to copy a session to perform one or several operations, letting the connection pool in mgo to improve the performance. the default limit is 4096 for one mongo server, to prevent too much connection.
func newSession(consistency Mode, cluster *mongoCluster, timeout time.Duration) (session *Session) {
cluster.Acquire()
session = &Session{
cluster_: cluster,
syncTimeout: timeout,
sockTimeout: timeout,
poolLimit: 4096,
}
debugf("New session %p on cluster %p", session, cluster)
session.SetMode(consistency, true)
session.SetSafe(&Safe{})
session.queryConfig.prefetch = defaultPrefetch
return session
}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论