在Go中使用缓冲区进行读取和写入操作的示例

huangapple go评论72阅读模式
英文:

Using buffer for read and read operations examples in Go

问题

我的操作系统背景不够强,能否有人提供一些例子(如果可能的话,请用Go语言),说明为什么使用缓冲区很重要?

英文:

my os background is not strong, could someone provide some examples (in Go please if possible), why using buffer is important?

答案1

得分: 6

假设你在谈论IO:

假设你有一个var fin *os.File,并且该文件被包装在一个缓冲区中,var instrm *bufio.Reader。现在想象一下,你正在编写一种解析器,它以每次一个字符(比如字节)的方式读取输入。bufio包实现了缓冲I/O。

如果你调用myParser.Parse(fin),你将调用.Read 4,194,304次来读取每个字节,这将导致系统调用4,194,304次,从而引起4,194,304次上下文切换上下文切换是控制从用户空间程序转移到操作系统的过程,是最慢的(非IO)操作之一。在操作系统没有合并/预取IO请求的情况下,还有一种可怕的可能性,即IO设备正在逐字节地寻找和读取,但大多数操作系统的IO电梯、预取和设备端缓冲区现在都可以防止这种情况发生(但最好还是以大的顺序批量读取)。

如果你使用默认的bufio.Reader缓冲区大小为4K来调用myParser.Parse(instrm),你将引起1,024次上下文切换(每个系统调用读取4K而不是1个字节)。由于每个系统调用都有一些开销,这意味着花费在系统调用上的时间较少,程序运行的时间较多。还值得指出的是,以这种方式运行(没有额外的上下文切换)通常会增加CPU指令缓存命中率,因为会花费更多的时间在内存的较小区域内进行分支

缓冲区在网络IO等领域也很重要,因为它可以让你以最大的MTU大小发送一连串的数据包,而不是发送一小部分的小数据包。

只是不要忘记刷新写缓冲区。

英文:

Assuming you're talking about IO:

Imagine you have a var fin *os.File and that file wrapped in a buffer, var instrm *bufio.Reader. Now imagine you are writing some kind of parser that reads the input one character (lets say byte) at a time. Package bufio implements buffered I/O.

If you call myParser.Parse(fin) you will call .Read 4,194,304 times to read each byte, which will make a system call 4,194,304 times, which will cause 4,194,304 context switches. context switches are when control transfers from the userspace program to the OS and are one of the slowest (non-IO) operations. In situations where the OS is not coalescing/prefetching IO requests there is also the horrible possibility your IO device is seeking and reading one byte at a time, but most operating system's IO elevator, prefetching and device-side buffers prevent this nowadays (but it is always better to read in large sequential batches).

If you call myParser.Parse(instrm) with the default bufio.Reader buffer being 4K you will cause 1,024 context switches (each system call reads 4K rather than 1 byte). Since each system call has some overhead, this will mean less time is spent making system calls and more time for your program to run. It is also worth pointing out that running this way (without the extra context switches) will often increase the CPU instruction cache hit rate since more time will be spent branching within a smaller region of memory.

Buffers are even important in areas like network IO as it will allow you to send bursts of packets at the maximum MTU size rather than sending a trickle of tiny packets.

Just don't forget to flush your write buffers.

huangapple
  • 本文由 发表于 2013年7月8日 17:03:34
  • 转载请务必保留本文链接:https://go.coder-hub.com/17522677.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定