在Go语言中以一种易于访问的方式表示不同结构的集合

huangapple go评论88阅读模式
英文:

Representing a collection of different structs in an easy to access way in Go

问题

首先,我要说明一下,我被限制在使用 Go 1.16 版本,所以不幸的是,1.18 版本中的泛型功能不可用。(不过如果将来有人遇到这个问题,可以将其作为答案添加进来)

我有一堆类似这样的函数:

func GetPerson(id int) chan Person { // 一些操作 }
func GetPet(id int) chan Pet { // 一些操作 }
// 其他获取不同结构体的类似方法

简单的目标是调用所有这些方法,并从通道中获取数据以供访问。将对象加载到通道中的原因是为了能够并发调用这些函数,因为通常它们会执行 HTTP 请求等操作。

现在我们想要从这些通道中获取所有数据,并使其一次性可用。最简单的方法是这样的:

person := <- GetPerson(id)
pet := <- GetPet(id)
...

这样做就无法充分利用上述函数的并发性质,所以我的下一个解决方案是创建一个类型,在该类型上定义一个函数,该函数基本上循环遍历返回的通道并将其加载到字段中,如下所示:

type OverallData struct {
    field1 struct{}
    field2 struct{}
    // 其他字段
}

func (data OverallData) RetrieveData(personId int, petId int) {
    for x := 0; x < chanLen; x++ {
        select {
        case data.field1 = <- GetPerson(personId):
        case data.field2 = <- GetPet(petId):
        // 其他情况
        }
    }  
}

func GetOverallData(personId int, petId int) OverallData {
    data := OverallData{}
    data.RetrieveData(personId, petId)
    return data
}

但我对此并不是特别喜欢,它需要一个类型来聚合特定组的响应,而且这些类型在性质上非常相似。此外,GetOverallData 函数将根据需要返回的数据量越来越多而需要越来越多的参数,变得难以管理(虽然你可以为此添加另一个类型,但我认为这只是把问题转移到其他地方)。

因此,我开始思考一个更通用的解决方案,并想出了一个更糟糕的方法:

func ProcessResults(results []chan struct{}) map[reflect.Type]struct{} {
    respMap := map[reflect.Type]struct{}
    for _, ch := range results {
        go func(c chan struct{}) {
            val := <- ch
            typ := reflect.TypeOf(val)
            respMap[typ] = val
        }(ch)
    }
    return respMap
}

上述代码的目标是,你只需要调用一次类似下面的代码。这将是你所需的代码量,无需添加任何类型或传递大量参数。这种写法简单易读(大部分情况下,最后一行有点乱)。你还可以确保在进行类型转换时,类型一定匹配。

responses := ProcessResults([]chan struct{} {
    GetPerson(personId),
    GetPet(petId),
})

person := responses[reflect.TypeOf(Person)].(Person)

问题在于它不起作用,你不能将我的方法返回的类型直接映射到数组中的 struct{},而且尽管我可以将其全部转换为 struct,但我怀疑之前使用的 reflect.TypeOf() 在转换为 struct 后将无法工作(因为类型将变为 struct{},我的映射键将无法工作),还有其他问题,比如说如果有两个返回相同类型的通道,但可以解决这个问题。

我还在考虑是否添加一个嵌入式结构体,所有从通道返回的类型都将具有该结构体,强制它们符合像这个问题中的接口一样的接口:
https://stackoverflow.com/questions/26027350/go-interface-fields

我可以使用该结构体为映射指定一个键,这样就不必处理类型检查,而且可以将所有数据作为该接口类型传递,而不必将其强制转换回结构体类型,因此不会丢失类型信息,并且可以将其转换回通道返回的原始类型。

当我达到这一点时,我意识到我可能在尝试做一些不应该在 Go 中做的事情,而且可能有更好的方法来实现它。有人有更好的想法吗?否则,我可能会尝试上述方法,如果感觉不对,就放弃它,采用我的第二个解决方案。

英文:

First of all, just to preface this, I'm stuck with Go version 1.16, so the generics stuff in 1.18 is not an option unfortunately. (However feel free to add it as an answer in case someone stumbles across this in the future)

So I've got a bunch of functions that look like this:

func GetPerson(id int) chan Person { // stuff }
func GetPet(id int) chan Pet { // stuff }
// similar methods that get different structs

And the simple goal is to call all these methods and get the data out of the channels and provide them for access. The reason why the objects are being loaded into channels is so that we can call these functions concurrently, as often they'll go ahead and make http requests and so on.

Now we want to pull all the data off these channels and make it accessible to use all at once, the easiest way to do this would be something like

person := &lt;- GetPerson(id)
pet := &lt;- GetPet(id)
...

This wouldn't then take advantage of the concurrent nature of the functions above, so my next solution would be to have a type that has a function on it that instead basically loops over the channels that are returned and loads them into the fields, like so.

type OverallData struct {
    field1 struct{}
    field2 struct{}
    // etc.
}

func (data OverallData) RetrieveData(personId int, petId int) {
    for x:= 0, x &lt; chanLen, x++ {
        select {
        case data.field1 = &lt;- GetPerson(personId):
        case data.field2 = &lt;- GetPet(petId):
        // etc
        }
    }  

}

func GetOverallData(personId int, petId int) OverallData {
    data := OverallData{}
    data.RetrieveData(personId, petId)
    return data
}

But I'm not a massive fan of this, it requires a type to aggregate the responses for any particular group, and these types would be very similar in nature. Also the GetOverallData function will take more and more arguments depending on the amount of data you need back, and it becomes unruly (you can just add another type in for this, but I think that's just a way to move the problem somewhere else)

So I started thinking of a more generic solution, and came up with something a bit worse

func ProcessResults(results []chan struct{}) map[reflect.Type]struct{} {
    respMap := map[reflect.Type]struct{}
    for _, ch := range results {
        go func(c chan struct{}) {
            val &lt;- ch
            typ := reflect.TypeOf(val)
            respMap[typ] = val
        }(ch)
    }
    return respMap
}

The goal with the above was that you would then just have one call that looks something like. This would be just as much code as you can need, no need to add any types or pass tons of stuff as paramaeters. And this is easy to write and read (mostly, last line is a bit messy). You also would know for a fact your type would match when you cast it.

responses := ProcessResults([]chan struct{} {
    GetPerson(personId),
    GetPet(petId),
})


person := responses[reflect.TypeOf(Person)].(Person)

The problem with it, is that it doesn't work, you can't just map the types returned by my methods into struct{} within an array, and while I could just cast it all to structs I suspect the reflect.TypeOf() I was using before will not work after converting it to a struct (as the type with then by struct{} and my map key will not work), there are also other issues like say I had two channels that returned the same type, but could get around this.

I'm also thinking about adding a embedded struct that all the types returned from the channel will have, forcing them to confine to an interface like in this question:
https://stackoverflow.com/questions/26027350/go-interface-fields

I can use that struct to specify a key for the map so I don't have to mess around with the type checking, and also I can pass all that data as that interface type rather than type casting it back to a struct and therefore wouldn't lose the typing and it can cast it back to the original type returned by the channel.

When I got to this point I realise I'm probably trying to do something that I shouldn't be doing with Go, and there is a better way to accomplish it. Does anyone have any better ideas? Else I'll probably have a crack at the above, and if it feels wrong, will leave it and go with my second solution.

答案1

得分: 2

在调用函数而不是被调用函数中管理并发通常是一种典型的做法。使用错误组来管理一组并发调用。

声明函数如下:

func GetPerson(id int) (*Person, error)
func GetPet(id int) (*Pet, error)

像这样并发调用函数:

var g errgroup.Group

var person *Person
g.Go(func() error {
    var err error
    person, err = GetPerson(personId)
    return err
})

var pet *Pet
g.Go(func() error {
    var err error
    pet, err = GetPet(petId)
    return err
})

if err := g.Wait(); err != nil {
   // 处理错误
}

// 在这里访问 person 和 pet 变量。

使用reflect包来消除样板代码:

// ego 在组 g 中使用参数 args 调用函数 fn。fn 的第二个返回值
// 假设为错误,并将其返回给错误组。
// 如果错误为 nil,则将 fn 的第一个返回值赋值给 presult 指向的值。
func ego(g *errgroup.Group, presult any, fn any, args ...any) {
    g.Go(func() error {
        var rargs []reflect.Value
        for _, arg := range args {
            rargs = append(rargs, reflect.ValueOf(arg))
        }
        result := reflect.ValueOf(fn).Call(rargs)
        if err, ok := result[1].Interface().(error); ok && err != nil {
            return err
        }
        reflect.ValueOf(presult).Elem().Set(result[0])
        return nil
    })
}

像这样使用它:

var g &errgroup.Group{}
var person *Person
var pet *Pet
ego(g, &person, GetPerson, personId)
ego(g, &pet, GetPet, petId)
if err := g.Wait(); err != nil {
   // 处理错误
}
// 在这里访问 person 和 pet 变量。
英文:

It is typical to manage concurrency in the caller instead of the called function. Use an error group to manage a group of concurrent calls.

Declare the functions like this:

func GetPerson(id int) (*Person, error)
func GetPet(id int) (*Pet, error)

Call the functions concurrently like this:

var g errgroup.Group

var person *Person
g.Go(func() error {
    var err error
    person, err = GetPerson(personId)
    return err
})

var pet *Pet
g.Go(func() error {
    var err error
    pet, err = GetPet(petId)
    return err
})

if err := g.Wait(); err != nil {
   // handle error
}

// access person and pet variables here.

Use the reflect package to eliminate the boilerplate:

// ego calls fn with args in group g. fn&#39;s second return value
// is assumed to be an error and is returned to the error group.
// If the error is nil, then fn&#39;s first return value is assigned
// to the value that presult points to.
func ego(g *errgroup.Group, presult any, fn any, args ...any) {
	g.Go(func() error {
		var rargs []reflect.Value
		for _, arg := range args {
			rargs = append(rargs, reflect.ValueOf(arg))
		}
		result := reflect.ValueOf(fn).Call(rargs)
		if err, ok := result[1].Interface().(error); ok &amp;&amp; err != nil {
			return err
		}
		reflect.ValueOf(presult).Elem().Set(result[0])
		return nil
	})
}

Use it like this:

var g &amp;errgroup.Group{}
var person *Person
var pet *Pet
ego(g, &amp;person, GetPerson, personId)
ego(g, &amp;pet, GetPet, petId)
if err := g.Wait(); err != nil {
   // handle error
}
// access person and pet variables here.

huangapple
  • 本文由 发表于 2023年2月22日 22:08:59
  • 转载请务必保留本文链接:https://go.coder-hub.com/75533878.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定