从Go中的通道中读取多个元素

huangapple go评论73阅读模式
英文:

Reading multiple elements from a channel in Go

问题

我正在从一个通道中循环读取值,代码如下:

for {
    capturedFrame := <-capturedFrameChan
    remoteCopy(capturedFrame)
}

为了提高效率,我想要批量读取这些值,类似于以下伪代码:

for {
    capturedFrames := <-capturedFrameChan
    multipleRemoteCopy(capturedFrames)
}

但是我不确定如何实现这一点。如果我多次调用capturedFrames := <-capturedFrameChan,它会阻塞。

基本上,我想要的是读取captureFrameChan中所有可用的值,如果没有可用值,则像通常一样阻塞。

在Go语言中,如何实现这个目标呢?

英文:

I'm reading values from a channel in a loop like this:

for {
	capturedFrame := &lt;-capturedFrameChan
	remoteCopy(capturedFrame)
}

To make it more efficient, I would like to read these values in a batch, with something like this (pseudo-code):

for {
	capturedFrames := &lt;-capturedFrameChan
	multipleRemoteCopy(capturedFrames)
}

But I'm not sure how to do that. If I call capturedFrames := &lt;-capturedFrameChan multiple times it's going to block.

Basically, what I would like is to read all the available values in captureFrameChan and, if none is available, it blocks as usual.

What would be the way to accomplish this in Go?

答案1

得分: 6

应该可以这样实现:

for {
    // 初始化切片。你可能想要增加一个更大的容量,以避免在`append`时进行多次内存分配
    capturedFrames := make([]Frame, 1)
    // 阻塞等待第一帧
    capturedFrames[0] = <-capturedFrameChan

forLoop:
    for {
        select {
        case buf := <-capturedFrameChan:
            // 如果有更多的帧立即可用,我们将它们添加到切片中
            capturedFrames = append(capturedFrames, buf)
        default:
            // 否则,我们继续而不阻塞
            break forLoop
        }
    }

    multipleRemoteCopy(capturedFrames)
}

请注意,这只是一个代码片段,可能需要根据你的实际需求进行适当的修改。

英文:

Something like this should work:

for {
	// we initialize our slice. You may want to add a larger cap to avoid multiple memory allocations on `append`
	capturedFrames := make([]Frame, 1)
	// We block waiting for a first frame
	capturedFrames[0] = &lt;-capturedFrameChan

forLoop:
	for {
		select {
		case buf := &lt;-capturedFrameChan:
			// if there is more frame immediately available, we add them to our slice
			capturedFrames = append(capturedFrames, buf)
		default:
			// else we move on without blocking
			break forLoop
		}
	}

	multipleRemoteCopy(capturedFrames)
}

答案2

得分: 2

尝试这个(对于类型为T的通道ch):

for firstItem := range ch {                      // 为了确保任何批次都不会为空
	var itemsBatch []T
	itemsBatch = append(itemsBatch, firstItem)
Remaining: 
	for len(itemsBatch) < BATCHSIZE {            // 控制批次的最大大小
		select {
		case item := <-ch:
			itemsBatch = append(itemsBatch, item)
		default:
			break Remaining
		}
	}
	// 在这里处理itemsBatch...
}

但是,如果BATCHSIZE是常量,这段代码将更高效:

var i int
itemsBatch := [BATCHSIZE]T{}
for firstItem := range ch {                      // 为了确保任何批次都不会为空
	itemsBatch[0] = firstItem
Remaining:
	for i = 1; i < BATCHSIZE; i++ {              // 控制批次的最大大小
		select {
		case itemsBatch[i] = <-ch:
		default:
			break Remaining
		}
	}
	// 现在你有长度小于等于BATCHSIZE的itemsBatch;
	// 在这里处理它...
}
英文:

Try this (for channel ch with type T):

for firstItem := range ch {                      // For ensure that any batch could not be empty
	var itemsBatch []T
	itemsBatch = append(itemsBatch, firstItem)
Remaining: 
	for len(itemsBatch) &lt; BATCHSIZE {            // For control maximum size of batch
		select {
		case item := &lt;-ch:
			itemsBatch = append(itemsBatch, item)
		default:
			break Remaining
		}
	}
	// Consume itemsBatch here...
}

But, if BATCHSIZE is constant, this code would be more efficient:

var i int
itemsBatch := [BATCHSIZE]T{}
for firstItem := range ch {                      // For ensure that any batch could not be empty
	itemsBatch[0] = firstItem
Remaining:
	for i = 1; i &lt; BATCHSIZE; i++ {              // For control maximum size of batch
		select {
		case itemsBatch[i] = &lt;-ch:
		default:
			break Remaining
		}
	}
	// Now you have itemsBatch with length i &lt;= BATCHSIZE;
	// Consume that here...
}

答案3

得分: 0

通过使用len(capturedFrames),你可以像下面这样做:

for {
    select {
    case frame := <-capturedFrames:
        frames := []Frame{frame}
        for i := 0; i < len(capturedFrames); i++ {
            frames = append(frames, <-capturedFrames)
        }
        multipleRemoteCopy(frames)
    }
}
英文:

By using len(capturedFrames), you can do it like below:

for {
    select {
    case frame := &lt;-capturedFrames:
	    frames := []Frame{frame}
	    for i := 0; i &lt; len(capturedFrames); i++ {
		   frames = append(frames, &lt;-capturedFrames)
	    }
	    multipleRemoteCopy(frames)
    }
}

答案4

得分: -1

似乎你也可以仅仅对以下代码进行基准测试,而无需对代码库进行重构,以查看是否提高了效率。

for {
    capturedFrame := <-capturedFrameChan
    go remoteCopy(capturedFrame)
}
英文:

Seems you can also benchmark just

for {
    capturedFrame := &lt;-capturedFrameChan
    go remoteCopy(capturedFrame)
}

without any codebase refactoring to see if it increase efficiency.

答案5

得分: -1

我已经按照以下方式完成了它。基本上,我使用了len(capturedFrames)来知道有多少帧可用,然后在循环中检索它们:

for {
    var paths []string
    itemCount := len(capturedFrames)
    if itemCount <= 0 {
        time.Sleep(50 * time.Millisecond)
        continue
    }
    
    for i := 0; i < itemCount; i++ {
        f := <-capturedFrames
        paths = append(paths, f)
    }

    err := multipleRemoteCopy(paths, opts)
    if err != nil {
        fmt.Printf("错误:无法远程复制\"%s\":%s", paths, err)
    }
}
英文:

I've ended up doing it as below. Basically I've used len(capturedFrames) to know how many frames are available, then retrieved them in a loop:

for {
	var paths []string
	itemCount := len(capturedFrames)
	if itemCount &lt;= 0 {
		time.Sleep(50 * time.Millisecond)
		continue
	}
	
	for i := 0; i &lt; itemCount; i++ {
		f := &lt;-capturedFrames
		paths = append(paths, f)
	}

	err := multipleRemoteCopy(paths, opts)
	if err != nil {
		fmt.Printf(&quot;Error: could not remote copy \&quot;%s\&quot;: %s&quot;, paths, err)
	}
}

huangapple
  • 本文由 发表于 2015年10月25日 17:10:59
  • 转载请务必保留本文链接:https://go.coder-hub.com/33328000.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定