To avoid multiple database calls blocking each other in a Go web app handler, are goroutines + syncGroup the only way?

huangapple go评论83阅读模式
英文:

To avoid multiple database calls blocking each other in a Go web app handler, are goroutines + syncGroup the only way?

问题

在查看了几个网络应用程序示例和样板文件之后,它们通常采用以下形式的方法(这里以Gin处理程序为例,假设有一个名为User和Billing的“存储库”结构体,它们从数据库或外部API获取数据。为了简化示例,我省略了错误处理):

func GetUserDetailsHandler(c *gin.Context) {
    // 这个结果可能来自应用程序的数据库
    var userResult = UserRepository.FindById(c.GetInt("user_id"))

    // 假设这个结果来自另一个数据源(例如:另一个数据库),所以我们不能只使用“User”进行连接查询
    var billingInfo = BillingRepository.FindById(c.GetInt("user_id"))

    c.JSON(http.StatusOK, gin.H{
        "user_data":    userResult,
        "billing_data": billingInfo,
    })

    return
}

在上述情况中,对User.FindById的调用可能使用某种数据库驱动程序,但据我所知,所有可用的Golang数据库/ORM库都以“同步”的方式返回数据(例如:作为返回值,而不是通过通道)。因此,在我可以继续执行BillingInfo.FindById之前,对User.FindById的调用将会阻塞,这显然不是理想的情况,因为它们可以并行工作。

因此,我认为最好的方法是使用Go协程+sync.WaitGroup来解决这个问题。类似这样:

func GetUserDetailsHandler(c *gin.Context) {
    var waitGroup sync.WaitGroup

    userChannel := make(chan User)
    billingChannel := make(chan Billing)

    waitGroup.Add(1)
    go func() {
        defer waitGroup.Done()
        userChannel <- UserRepository.FindById(c.GetInt("user_id"))
    }()

    waitGroup.Add(1)
    go func() {
        defer waitGroup.Done()
        billingChannel <- BillingRepository.FindById(c.GetInt("user_id"))
    }()

    waitGroup.Wait()

    userInfo := <-userChannel
    billingInfo := <-billingChannel

    c.JSON(http.StatusOK, gin.H{
        "user_data":    userResult,
        "billing_data": billingInfo,
    })

    return
}

现在,这个方法应该能够完成任务。但是对我来说,它似乎过于冗长,而且可能容易出错(如果我在任何go协程之前忘记“Add”到waitGroup中,或者如果我忘记“Wait”,那么整个过程都会失败)。这是唯一的方法吗?还是有更简单的方法我没有注意到的?

英文:

Having taken a look at several web application examples and boilerplates, the approach they take tends to be in the form of this (I'm using a Gin handler here as an example, and imaginary User and Billing "repository" structs that fetch data from either a database or an external API. I omitted error handling to make the example shorter) :

func GetUserDetailsHandler(c *gin.Context) {
    //this result presumably comes from the app&#39;s database
    var userResult = UserRepository.FindById( c.getInt(&quot;user_id&quot;) )
    
    //assume that this result comes from a different data source (e.g: a different database) all together, hence why we&#39;re not just doing a join query with &quot;User&quot;
    var billingInfo = BillingRepository.FindById(  c.getInt(&quot;user_id&quot;)  )
    
    c.JSON(http.StatusOK, gin.H {
        user_data : userResult,
        billing_data : billingInfo,
    })

    return
}

In the above scenario, the call to "User.FindById" might use some kind of database driver, but as far as I'm aware, all available Golang database/ORM libraries return data in a "synchronous" fashion (e.g: as return values, not via channels). As such, the call to "User.FindById" will block until it's complete, before I can move on to executing "BillingInfo.FindById", which is not at all ideal since they can both work in parallel.

So I figured that the best idea was to make use of go routines + syncGroup to solve the problem. Something like this:

func GetUserDetailsHandler(c *gin.Context) {
    var waitGroup sync.WaitGroup

    userChannel := make(chan User);
    billingChannel := make(chan Billing)
    
    waitGroup.Add(1)
    go func() {
            defer waitGroup.Done()
            userChannel &lt;- UserRepository.FindById( c.getInt(&quot;user_id&quot;) )               
    }()

    waitGroup.Add(1)
    go func(){
            defer waitGroup.Done()
            billingChannel &lt;- BillingRepository.FindById(  c.getInt(&quot;user_id&quot;) )
    }()

    waitGroup.Wait()
    
    userInfo := &lt;- userChannel
    billingInfo = &lt;- billingChannel
    
    c.JSON(http.StatusOK, gin.H {
        user_data : userResult,
        billing_data : billingInfo,
    })

    return
}

Now, this presumably does the job. But it seems unnecessarily verbose to me, and potentially error prone (if I forget to "Add" to the waitGroup before any go routine, or if I forget to "Wait", then it all falls apart). Is this the only way to do this? Or is there something simpler that I'm missing out?

答案1

得分: 2

也许可以像这样:


package main

import (
	"fmt"
)

func GetUserDetailsHander(c *gin.Context) {
	var userInfo USERINlFO
	var billingInfo BILLL

	err := parallel(
		func() (e error) {
			userInfo, e = UserRepository.FindById(c.getInt("user_id"))
			return
		},
		func() (e error) {
			billingInfo, e = BillingRepository.FindById(c.getInt("user_id"))
			return
		},
	)
	fmt.Println(err)

	c.JSON(http.StatusOK, gin.H{
		"user_data":    userInfo,
		"billing_data": billingInfo,
	})

	return
}
func parallel(do ...func() error) error {
	var err error
	rcverr := make(chan error)
	var wg sync.WaitGroup
	for _, d := range do {
		wg.Add(1)
		go func(do func() error) {
			rcverr <- do()
			wg.Done()
		}(d)
	}
	go func() {
		wg.Wait()
		close(rcverr)
	}()
	for range do {
		e := <-rcverr
		if e != nil {
			err = e // return here for fast path
		}
	}
	return err
}

英文:

maybe something like this


package main

import (
	&quot;fmt&quot;
)

func GetUserDetailsHander(c *gin.Context) {
	var userInfo USERINlFO
	var billingInfo BILLL

	err := parallel(
		func() (e error) {
			userInfo, e = UserRepository.FindById(c.getInt(&quot;user_id&quot;))
			return
		},
		func() (e error) {
			billingInfo, e = BillingRepository.FindById(c.getInt(&quot;user_id&quot;))
			return
		},
	)
	fmt.Println(err)

	c.JSON(http.StatusOK, gin.H{
		user_data:    userResult,
		billing_data: billingInfo,
	})

	return
}
func parallel(do ...func() error) error {
	var err error
	rcverr := make(chan error)
	var wg sync.WaitGroup
	for _, d := range do {
		wg.Add(1)
		go func(do func() error) {
			rcverr &lt;- do()
			wg.Done()
		}(d)
	}
	go func() {
		wg.Wait()
		close(rcverr)
	}()
	for range do {
		e := &lt;-rcverr
		if e != nil {
			err = e // return here for fast path
		}
	}
	return err
}

huangapple
  • 本文由 发表于 2017年9月8日 21:41:10
  • 转载请务必保留本文链接:https://go.coder-hub.com/46117887.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定