英文:
Does it make sense to make expensive syscalls from different goroutines?
问题
如果应用程序在多个文件描述符上执行一些繁重的操作（例如，打开-写入数据-同步-关闭），Go运行时会发生什么？当昂贵的系统调用发生时（例如syscall.Fsync），它会阻塞所有的goroutine吗？还是只有调用的goroutine被阻塞，而其他goroutine仍在运行？
因此，编写使用多个工作线程进行大量用户空间-内核空间上下文切换的程序是否有意义？在磁盘输入方面使用多线程模式是否有意义？
package main
import (
"log"
"os"
"sync"
)
var data = []byte("some big data")
func worker(filenamechan chan string, wg *sync.WaitGroup) {
defer wg.Done()
for {
filename, ok := <-filenamechan
if !ok {
return
}
// 打开文件是一个相当昂贵的操作，因为它涉及到打开新的文件描述符
f, err := os.OpenFile(filename, os.O_CREATE|os.O_WRONLY, os.FileMode(0644))
if err != nil {
log.Fatal(err)
continue
}
// 写入是一个廉价的操作，因为它只是将数据从用户空间移动到内核空间
if _, err := f.Write(data); err != nil {
log.Fatal(err)
continue
}
// syscall.Fsync 是一个与磁盘相关的昂贵操作
if err := f.Sync(); err != nil {
log.Fatal(err)
continue
}
if err := f.Close(); err != nil {
log.Fatal(err)
}
}
}
func main() {
// 启动工作线程
filenamechan := make(chan string)
wg := &sync.WaitGroup{}
for i := 0; i < 2; i++ {
wg.Add(1)
go worker(filenamechan, wg)
}
// 将任务发送给工作线程
filenames := []string{
"1.txt",
"2.txt",
"3.txt",
"4.txt",
"5.txt",
}
for i := range filenames {
filenamechan <- filenames[i]
}
close(filenamechan)
wg.Wait()
}
你可以在这里查看代码：https://play.golang.org/p/O0omcPBMAJ
英文:
If application does some heavy lifting with multiple file descriptors (e.g., opening - writing data - syncing - closing), what actually happens to Go runtime? Does it block all the goroutines at the time when expensive syscall occures (like syscall.Fsync)? Or only the calling goroutine is blocked while the others are still operating?
So does it make sense to write programs with multiple workers that do a lot of user space - kernel space context switching? Does it make sense to use multithreading patterns for disk input?
package main
import (
"log"
"os"
"sync"
)
var data = []byte("some big data")
func worker(filenamechan chan string, wg *sync.waitgroup) {
defer wg.done()
for {
filename, ok := <-filenamechan
if !ok {
return
}
// open file is a quite expensive operation due to
// the opening new descriptor
f, err := os.openfile(filename, os.o_create|os.o_wronly, os.filemode(0644))
if err != nil {
log.fatal(err)
continue
}
// write is a cheap operation,
// because it just moves data from user space to the kernel space
if _, err := f.write(data); err != nil {
log.fatal(err)
continue
}
// syscall.fsync is a disk-bound expensive operation
if err := f.sync(); err != nil {
log.fatal(err)
continue
}
if err := f.close(); err != nil {
log.fatal(err)
}
}
}
func main() {
// launch workers
filenamechan := make(chan string)
wg := &sync.waitgroup{}
for i := 0; i < 2; i++ {
wg.add(1)
go worker(filenamechan, wg)
}
// send tasks to workers
filenames := []string{
"1.txt",
"2.txt",
"3.txt",
"4.txt",
"5.txt",
}
for i := range filenames {
filenamechan <- filenames[i]
}
close(filenamechan)
wg.wait()
}
答案1
得分: 4
如果系统调用阻塞，Go运行时将启动一个新线程，以保持可用于运行goroutine的线程数量不变。
可以在这里找到更详细的解释：https://morsmachine.dk/go-scheduler
英文:
If a syscall blocks, the Go runtime will launch a new thread so that the number of threads available to run goroutines remains the same.
A fuller explanation can be found here: https://morsmachine.dk/go-scheduler
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论