Is there a way to get transfer speed from io.Copy?

huangapple go评论69阅读模式
英文:

Is there a way to get transfer speed from io.Copy?

问题

我正在使用io.Copy将网络流复制到文件中。我想提取当前的传输速度,最好是以每秒字节数为单位。

res, err := http.Get(url)

if err != nil {
    panic(err)
}

// 打开输出文件
out, err := os.OpenFile("output", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
if err != nil {
    panic(err)
}

// 关闭输出文件和响应体
defer out.Close()

defer func(Body io.ReadCloser) {
    err := Body.Close()
    if err != nil {
        panic(err)
    }
}(res.Body)

_, err := io.Copy(out, res.Body)

以上是代码部分,它将网络流复制到名为"output"的文件中。你可以使用io.Copy函数来执行复制操作。要提取当前的传输速度,你需要使用其他方法来测量。

英文:

I am copying a network stream to a file using io.Copy. I would like to extract the current speed, preferably in bytes per second, that the transfer is operating at.

res, err := http.Get(url)

if err != nil {
    panic(err)
}

// Open output file
out, err := os.OpenFile("output", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
if err != nil {
    panic(err)
}

// Close output file as well as body    

defer out.Close()

defer func(Body io.ReadCloser) {
    err := Body.Close()
    if err != nil {
        panic(err)
    }
}(res.Body)

_, err := io.Copy(out, res.Body) 

答案1

得分: 2

如评论中所述,整个传输速率可以在事后轻松计算,特别是在使用io.Copy时。如果您想要实时跟踪传输速率,并在长时间文件传输过程中轮询结果,则需要进行一些额外的工作。

下面我概述了一个简单的io.Reader包装器来跟踪整体传输速率。为了简洁起见,它不是goroutine安全的,但是可以很容易地使其安全。然后,您可以从另一个goroutine中轮询进度,而主要的goroutine负责读取。


您可以创建一个io.Reader包装器,并使用它来跟踪第一次读取的时刻,然后跟踪未来的读取字节数。最终的结果可能如下所示:

r := NewRater(resp.Body) // io.Reader包装器

n, err := io.Copy(out, r)

log.Print(r) // stringer方法显示人类可读的“b/s”输出

要实现这个,可以采用以下方法:

type rate struct {
    r          io.Reader
    count      int64 // 可能有大文件(2GB+),所以不要使用int
    start, end time.Time
}

func NewRater(r io.Reader) *rate { return &rate{r: r} }

然后我们需要包装器Read来跟踪底层io.Reader的进度:

func (r *rate) Read(b []byte) (n int, err error) {

    if r.start.IsZero() {
        r.start = time.Now()
    }

    n, err = r.r.Read(b) // 底层io.Reader读取

    r.count += int64(n)

    if err == io.EOF {
        r.end = time.Now()
    }

    return
}

可以随时轮询当前速率,即使在EOF之前也可以:

func (r *rate) Rate() (n int64, d time.Duration) {
    end := r.rend
    if end.IsZero() {
        end = time.Now()
    }
    return r.count, end.Sub(r.start)
}

还有一个简单的Stringer方法来显示b/s

func (r *rate) String() string {
    n, d := r.Rate()
    return fmt.Sprintf("%.0f b/s", float64(n)/(d.Seconds()))
}

注意:上述的io.Reader包装器没有进行锁定操作,因此操作必须在同一个goroutine中进行。由于问题涉及到io.Copy,所以可以做出这个安全的假设。

英文:

As noted in the comments - the entire transfer rate is easily computed after the fact - especially when using io.Copy. If you want to track "live" transfer rates - and poll the results over a long file transfer - then a little more work is involved.

Below I've outlined a simple io.Reader wrapper to track the overall transfer rate. For brevity, it is not goroutine safe, but would be trivial do make it so. And then one could poll from another goroutine the progress, while the main goroutine does the reading.


You can create a io.Reader wrapper - and use that to track the moment of first read - and then track future read byte counts. The final result may look like this:

r := NewRater(resp.Body) // io.Reader wrapper

n, err := io.Copy(out, r)

log.Print(r) // stringer method shows human readable "b/s" output

To implement this, one approach:

type rate struct {
	r          io.Reader
	count      int64 // may have large (2GB+) files - so don't use int
	start, end time.Time
}

func NewRater(r io.Reader) *rate { return &rate{r: r} }

then we need the wrapper Read to track the underlying io.Readers progress:

func (r *rate) Read(b []byte) (n int, err error) {

	if r.start.IsZero() {
		r.start = time.Now()
	}

	n, err = r.r.Read(b) // underlying io.Reader read

	r.count += int64(n)

	if err == io.EOF {
		r.end = time.Now()
	}

	return
}

the rate at any time can be polled like so - even before EOF:

func (r *rate) Rate() (n int64, d time.Duration) {
    end := r.rend
    if end.IsZero() {
        end = time.Now()
    }
    return r.count, end.Sub(r.start)
}

and a simple Stringer method to show b/s:

func (r *rate) String() string {
    n, d := r.Rate()
    return fmt.Sprintf("%.0f b/s", float64(n)/(d.Seconds()))
}

Note: the above io.Reader wrapper has no locking in place, so operations must be from the same goroutine. Since the question relates to io.Copy - then this is a safe assumption to make.

huangapple
  • 本文由 发表于 2022年6月30日 05:21:03
  • 转载请务必保留本文链接:https://go.coder-hub.com/72808002.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定