Go | efficient and readable way to append a slice and send to variadic function

huangapple go评论86阅读模式
英文:

Go | efficient and readable way to append a slice and send to variadic function

问题

假设我有以下函数管道:

func func3(opts ...FunctionObject) {
    for _, opt := range opts {
        opt()
    }
}

func func2(opts ...FunctionObject) {
   var functions []FunctionObject
   functions = append(functions, someFunction3)
   functions = append(functions, someFunction4)
   // ...
   // ...
   func3(append(functions, opts...)...)
}


func func1(opts ...FunctionObject) {
   var functions []FunctionObject
   functions = append(functions, someFunction)
   functions = append(functions, someFunction2)
   // ...
   // ...
   func2(append(functions, opts...)...)
}

由于继承问题,我想要在调用opts中的函数之前调用functions中的函数,所以我不能简单地将functions追加到opts中,而是必须将functions插入到opts之前(通过append(functions, opts...)),然后再次使用...将其发送到管道中的下一个函数,所以我得到了这个奇怪的表达式:

func2(append(functions, opts...)...)

我不知道这样做的效率如何,但我确定它看起来很奇怪,肯定有更好的方法来实现这个目的,这就是我要找的。

同时,我会感激有关效率的解释 Go | efficient and readable way to append a slice and send to variadic function

编辑:
我不能将参数类型从opts ...FunctionObject更改为opts []FunctionObject(如@dev.bmax在评论中建议的)因为我正在修改现有的代码库,所以无法更改调用func{1,2,3}的函数。

另外,当我说"看起来很奇怪"时,我不仅指的是"外观",而是这个操作(省略号)看起来很奇怪,而且似乎效率低下(我错了吗?)。

英文:

Let's say I have the following pipeline of functions:

func func3(opts ...FunctionObject) {
    for _, opt := range opts {
        opt()
    }
}

func func2(opts ...FunctionObject) {
   var functions []FunctionObject
   functions = append(functions, someFunction3)
   functions = append(functions, someFunction4)
...
...
...
    func3(append(functions, opts...)...)
}


func func1(opts ...FunctionObject) {
   var functions []FunctionObject
   functions = append(functions, someFunction)
   functions = append(functions, someFunction2)
...
...
...
    func2(append(functions, opts...)...)
}

For reasons inherited in the problem I want to solve, the functions in functions should be called before the functions in opts , so i can't just append to opts but I have to prepend functions to opts (by append(functions, opts...) ) and then using ... again to send it to next function in the pipeline, so im getting the weird expression:

func2(append(functions, opts...)...)

I don't know how efficient it is, but Im sure it looks weird,
There must be a better way of doing it, and that's what Im looking for.

yet i'd be grateful for accompanying explanation about efficiency Go | efficient and readable way to append a slice and send to variadic function

Edit:
I can't change the the argument type from opts ...FunctionObject to opts []FunctionObject (as @dev.bmax suggested in comments) since im making changes in an existing codebase so i can't change the functions that call func{1,2,3}

  1. by saying that "it looks weird" i don't mean only of the "look" but it looks weird to do this operation (ellipsis) twice, and it seems to be inefficient (am i wrong?)

答案1

得分: 1

将一个切片的元素前置是基本上是低效的,因为它将需要以下操作之一:

  • 分配一个更大的后备数组
  • 将元素移动到切片的末尾
  • ...或者两者都需要。

如果你能够在函数之间改变调用约定,只追加选项然后以相反顺序处理它们,那将更高效。这样可以避免重复将元素移动到切片的末尾,并且在预先分配足够空间的情况下,可能避免除第一个之外的所有分配。

func func3(opts ...FunctionObject) {
    for i := len(opts) - 1; i >= 0; i-- {
        opts[i]()
    }
}

注意:func3(opts ...FunctionObject) / func3(opts...)func3(opts []FunctionObject) / func3(opts) 在性能上是等效的。前者实际上是传递切片的语法糖。

然而,你提到你需要保持你的调用约定...

你的示例代码将导致在每个函数内部的第1、2、3、5次追加时进行分配 - 需要为后备数组的大小加倍进行分配(对于小切片)。如果之前的追加没有创建足够的备用容量,append(functions, opts...) 也可能进行分配。

一个辅助函数可以使代码更易读。它还可以重用 opts 后备数组中的备用容量:

func func2(opts ...FunctionObject) {
    // 1-2次分配。始终分配包含前置项的可变切片。如果需要,Prepend 会重新分配 `opts` 的后备数组。
    opts = Prepend(opts, someFunction3, someFunction4)
    func3(opts...)
}

// 泛型需要 Go1.18+。否则将 T 更改为 FunctionObject。
func Prepend[T any](base []T, items ...T) []T {
    if size := len(items) + len(base); size <= cap(base) {
        // 使用备用切片容量扩展 base。
        out := base[:size]
        // 将元素从切片的开头移动到末尾(处理重叠)。
        copy(out[len(items):], base)
        // 复制前置元素。
        copy(out, items)
        return out
    }
    return append(items, base...) // 总是重新分配。
}

一些没有辅助函数的备选方案,更详细地描述了分配情况:

// 直接分配要前置的项(2次分配)。
func func1(opts ...FunctionObject) {
    // 分配没有备用容量的切片进行前置,然后 append 重新分配后备数组,因为它对于额外的 `opts` 来说不够大。
    // 在将来,Go 可能会在初始时分配足够的空间以避免重新分配,但目前还没有这样做(截至 Go1.20rc1)。
    functions := append([]FunctionObject{
        someFunction,
        someFunction2,
        ...
    }, opts...)
    // 不分配 - 切片只是传递给下一个函数。
    func2(functions...)
}

// 最小化分配(1次分配)。
func func2(opts ...FunctionObject) {
   // 预先分配所需的空间,以避免在此函数内进行任何进一步的 append 分配。
   functions := make([]FunctionObject, 0, 2 + len(opts))
   functions = append(functions, someFunction3)
   functions = append(functions, someFunction4)
   functions = append(functions, opts...)
   func3(functions...)
}

你还可以进一步重用 opts 中的备用容量,而无需分配包含要前置项的切片(每个函数 0-1 次分配)。然而,这很复杂且容易出错 - 我不建议这样做。

英文:

Prepending to a slice is fundamentally inefficient since it will require some combination of:

  • allocating a larger backing array
  • moving items to the end of the slice
  • ...or both.

It would be more efficient if you could change the calling convention between functions to only append options and then process them in reverse. This could avoid repeatedly moving items to the end of the slice and potentially all allocations beyond the first (if enough space is allocated in advance).

func func3(opts ...FunctionObject) {
    for i := len(opts) - 1; i &gt;= 0; i-- {
        opts[i]()
    }
}

Note: func3(opts ...FunctionObject) / func3(opts...) and func3(opts []FunctionObject) / func3(opts) are equivalent for performance. The former is effectively syntactic sugar for passing the slice.

However, you've mentioned you need to keep your calling conventions...

Your example code will cause allocations for the 1st, 2nd, 3rd, 5th,.. append within each function - allocations are needed to double the size of the backing array (for small slices). append(functions, opts...) will likely also allocate if the earlier appends didn't create enough spare capacity.

A helper function could make the code more readable. It could also reuse spare capacity in the opts backing array:

func func2(opts ...FunctionObject) {
    // 1-2 allocations. Always allocate the variadic slice containings 
    // prepend items. Prepend reallocates the backing array for `opts`
    // if needed.
    opts = Prepend(opts, someFunction3, someFunction4)
    func3(opts...)
}

// Generics requires Go1.18+. Otherwise change T to FunctionObject.
func Prepend[T any](base []T, items ...T) []T {
    if size := len(items) + len(base); size &lt;= cap(base) {
        // Extend base using spare slice capacity.
        out := base[:size]
        // Move elements from the start to the end of the slice (handles overlaps).
        copy(out[len(items):], base)
        // Copy prepended elements.
        copy(out, items)
        return out
    }
    return append(items, base...) // Always re-allocate.
}

Some alternate options without the helper function that describe the allocations in more detail:

// Directly allocate the items to prepend (2 allocations).
func func1(opts ...FunctionObject) {
    // Allocate slice to prepend with no spare capacity, then append re-allocates the backing array
    // since it is not large enough for the additional `opts`.
    // In future, Go could allocate enough space initially to avoid the
    // reallocation, but it doesn&#39;t do it yet (as of Go1.20rc1).
    functions := append([]FunctionObject{
        someFunction,
        someFunction2,
        ...
    }, opts...)
    // Does not allocate -- the slice is simply passed to the next function.
    func2(functions...)
}

// Minimise allocations (1 allocation).
func func2(opts ...FunctionObject) {
   // Pre-allocate the required space to avoid any further append
   // allocations within this function.
   functions := make([]FunctionObject, 0, 2 + len(opts))
   functions = append(functions, someFunction3)
   functions = append(functions, someFunction4)
   functions = append(functions, opts...)
   func3(functions...)
}

You could go further and reuse with spare capacity in opts without needing to allocate a slice containing the items to prepend (0-1 allocations per function). However this is complex and error prone -- I wouldn't recommend it.

huangapple
  • 本文由 发表于 2022年12月25日 19:21:07
  • 转载请务必保留本文链接:https://go.coder-hub.com/74912982.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定