并发写入文件

huangapple go评论106阅读模式
英文:

Concurrent writing to a file

问题

在Go语言中,你可以使用互斥锁(Mutex)来控制并发写入文本文件。

互斥锁是一种同步原语,用于保护共享资源,确保在同一时间只有一个goroutine可以访问该资源。在你的代码中,你可以在写入文件之前使用互斥锁来保护文件的访问。

首先,你需要在包级别声明一个互斥锁:

var mutex = &sync.Mutex{}

然后,在WriteToFile函数中,在写入文件之前获取互斥锁,并在写入完成后释放互斥锁:

func WriteToFile(i int, f *os.File, w *sync.WaitGroup) {
    // 获取互斥锁
    mutex.Lock()
    defer mutex.Unlock()

    // 省略部分代码...

    // 写入文件
    fmt.Fprintf(f, "Printing out: %d\n", i)
    // 写入stdout
    fmt.Printf("Printing out: %d\n", i)
    w.Done()
}

通过这样的方式,每个goroutine在写入文件之前都会获取互斥锁,确保同一时间只有一个goroutine可以写入文件。这样可以避免并发写入导致的混乱输出。

希望对你有所帮助!如果有任何其他问题,请随时问我。

英文:

In go, how can I control the concurrent writing to a text file?

I ask this because I will have multiple goroutines writing to a text file using the same file handler.

I wrote this bit of code to try and see what happens but I'm not sure if I did it "right":

package main

import (
    "os"
    "sync"
    "fmt"
    "time"
    "math/rand"
    "math"
)


func WriteToFile( i int, f *os.File, w *sync.WaitGroup ){
    //sleep for either 200 or 201 milliseconds
    randSleep := int( math.Floor( 200 + ( 2 * rand.Float64() ) ) )
    fmt.Printf( "Thread %d waiting %d\n", i, randSleep )
    time.Sleep( time.Duration(randSleep) * time.Millisecond )

    //write to the file
    fmt.Fprintf( f, "Printing out: %d\n", i )
    //write to stdout
    fmt.Printf( "Printing out: %d\n", i )
    w.Done()
}

func main() {
    rand.Seed( time.Now().UnixNano() )

    d, err := os.Getwd()
    if err != nil {
        fmt.Println( err )
    }
    filename := d + "/log.txt"

    f, err := os.OpenFile( filename, os.O_CREATE | os.O_WRONLY | os.O_TRUNC, 0666 )

    if err != nil {
        fmt.Println( err )
    }
    var w *sync.WaitGroup = new(sync.WaitGroup)
    w.Add( 10 )

    //start 10 writers to the file
    for i:=1; i <= 10; i++ {
        go WriteToFile( i, f, w )
    }

    //wait for writers to finish
    w.Wait()

}

I half expected that the output would show something like this in the file instead of the coherent output I got:

Printing Printing out: 2
out: 5
Poriuntitng: 6

Essentially, I expected the characters to come out incoherently and interweaved due to a lack of synchronization. Did I not write code that would coax this behavior out? Or is some mechanism during calls to fmt.Fprintf synchronizing the writing?

答案1

得分: 46

一种简单的控制并发访问的方法是通过一个服务goroutine来实现,该goroutine从一个通道接收消息。这个goroutine将独占地访问文件,因此访问将是顺序的,没有任何竞争问题。

通道很好地交错处理请求。客户端将消息写入通道而不是直接写入文件。通道上的消息会自动交错处理。

这种方法相对于简单地使用互斥锁的好处在于,你开始将程序视为一组微服务。这是CSP的方式,可以轻松地将大型系统从较小的组件组合起来。

英文:

A simple approach to controlling concurrent access is via a service goroutine, receiving messages from a channel. This goroutine would have sole access to the file. Access would therefore be sequential, without any race problems.

Channels do a good job of interleaving requests. The clients write to the channel instead of directly to the file. Messages on the channel are automatically interleaved for you.

The benefit of this approach over simply using a Mutex is that you start viewing your program as a collection of microservices. This is the CSP way and leads to easy composition of large systems from smaller components.

答案2

得分: 25

有很多方法可以控制并发访问。最简单的方法是使用互斥锁(Mutex):

var mu sync.Mutex

func WriteToFile(i int, f *os.File, w *sync.WaitGroup) {
    mu.Lock()
    defer mu.Unlock()
    // 其他操作...
}

至于为什么你没有看到问题,Go使用操作系统调用来实现文件访问,而这些系统调用是线程安全的(强调部分):

根据POSIX.1-2008/SUSv4第XSI 2.9.7节("线程与常规文件操作的交互")的规定:

以下所有函数在它们在常规文件或符号链接上操作时,对于指定的效果都是原子的:...

其中列出的API包括write()和writev(2)。在线程(和进程)之间应该是原子的效果之一是文件偏移的更新。然而,在Linux 3.14版本之前,情况并非如此:如果共享一个打开文件描述符的两个进程同时执行write()(或writev(2))操作,那么I/O操作在更新文件偏移时不是原子的,结果是两个进程输出的数据块可能(错误地)重叠。这个问题在Linux 3.14中得到了修复。

尽管如此,我仍然建议使用锁,因为Go代码不会自动保证线程安全(两个goroutine修改同一个变量会导致奇怪的行为)。

英文:

There are many ways to control concurrent access. The easiest is to use a Mutex:

var mu sync.Mutex

func WriteToFile( i int, f *os.File, w *sync.WaitGroup ){
    mu.Lock()
    defer mu.Unlock()
    // etc...
}

As to why you're not seeing problems, Go uses operating system calls to implement file access, and those system calls are thread safe (emphasis added):

> According to POSIX.1-2008/SUSv4 Section XSI 2.9.7 ("Thread Interactions with Regular File Operations"):

>> All of the following functions shall be atomic with respect to
each other in the effects specified in POSIX.1-2008 when they
operate on regular files or symbolic links: ...

> Among the APIs subsequently listed are write() and writev(2). And
among the effects that should be atomic across threads (and
processes) are updates of the file offset. However, on Linux before
version 3.14, this was not the case: if two processes that share an
open file description (see open(2)) perform a write() (or writev(2))
at the same time, then the I/O operations were not atomic with
respect updating the file offset, with the result that the blocks of
data output by the two processes might (incorrectly) overlap. This
problem was fixed in Linux 3.14.

I would still use a lock though, since Go code is not automatically thread safe. (two goroutines modifying the same variable will result in strange behavior)

huangapple
  • 本文由 发表于 2015年5月1日 10:59:44
  • 转载请务必保留本文链接:https://go.coder-hub.com/29981050.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定