英文:
`os.OpenFile` with O_RDONLY hangs on named pipe with no writers
问题
我正在编写一个守护进程,该进程应该通过Unix命名管道接收来自临时CLI命令的通知。我选择使用Unix命名管道来实现。我编写了一个简短的包,一方面生成一个单独的goroutine来从节点读取数据,并将接收到的通知发送到一个通道中(playground w/ unit test):
type Writer struct {
f *os.File
}
func NewWriter(ipc string) (*Writer, error) {
f, err := os.OpenFile(ipc, os.O_WRONLY, 0600)
if err != nil {
return nil, fmt.Errorf("writer: open file: %w", err)
}
return &Writer{f: f}, nil
}
func (w *Writer) WriteString(str string) (int, error) {
return w.f.WriteString(fmt.Sprint(str, "\n"))
}
func (w *Writer) Close() error {
return w.f.Close()
}
type Reader struct {
f *os.File
rmFn func() error
quit chan struct{}
done *sync.WaitGroup
}
func NewReader(ipc string) (*Reader, error) {
err := syscall.Mkfifo(ipc, 0640)
if err != nil {
return nil, fmt.Errorf("reader: create fifo: %w", err)
}
f, err := os.OpenFile(ipc, os.O_RDONLY, 0640)
if err != nil {
return nil, fmt.Errorf("reader: open fifo: %w", err)
}
return &Reader{
f: f,
quit: make(chan struct{}),
done: &sync.WaitGroup{},
rmFn: func() error {
return os.Remove(ipc)
},
}, nil
}
func (r *Reader) PollRead() <-chan string {
reader := bufio.NewReader(r.f)
out := make(chan string)
r.done.Add(1)
go func() {
defer r.done.Done()
for {
line, err := reader.ReadBytes('\n')
if err != nil {
fmt.Printf("error reading from named pipe: %v\n", err)
return
}
nline := string(line)
nline = strings.TrimRight(nline, "\n")
select {
case out <- nline:
case <-r.quit:
close(out)
return
}
}
}()
return out
}
func (r *Reader) Close() error {
close(r.quit)
r.done.Wait()
err := r.f.Close()
if err != nil {
return fmt.Errorf("error closing named pipe: %v", err)
}
err = r.rmFn()
if err != nil {
return fmt.Errorf("error removing named pipe: %v", err)
}
return nil
}
这似乎是有效的,但它存在一个奇怪的行为,即在任何写入者打开文件之前,没有读取者可以打开文件,这与我在其他地方阅读到的相反行为相符;通常的抱怨是写入者挂起,因为没有读取者,但在这里,读取者根本无法实例化。
英文:
I am writing a daemon that is supposed to receive notifications from ephemeral CLI commands and have chosen to do so through a unix named pipe. I have written a short package that on the one hand spawns a separate goroutine to read from the node and send the received notifications down a channel (playground w/ unit test):
type Writer struct {
f *os.File
}
func NewWriter(ipc string) (*Writer, error) {
f, err := os.OpenFile(ipc, os.O_WRONLY, 0600)
if err != nil {
return nil, fmt.Errorf("writer: open file: %w", err)
}
return &Writer{f: f}, nil
}
func (w *Writer) WriteString(str string) (int, error) {
return w.f.WriteString(fmt.Sprint(str, "\n"))
}
func (w *Writer) Close() error {
return w.f.Close()
}
type Reader struct {
f *os.File
rmFn func() error
quit chan struct{}
done *sync.WaitGroup
}
func NewReader(ipc string) (*Reader, error) {
err := syscall.Mkfifo(ipc, 0640)
if err != nil {
return nil, fmt.Errorf("reader: create fifo: %w", err)
}
f, err := os.OpenFile(ipc, os.O_RDONLY, 0640)
if err != nil {
return nil, fmt.Errorf("reader: open fifo: %w", err)
}
return &Reader{
f: f,
quit: make(chan struct{}),
done: &sync.WaitGroup{},
rmFn: func() error {
return os.Remove(ipc)
},
}, nil
}
func (r *Reader) PollRead() <-chan string {
reader := bufio.NewReader(r.f)
out := make(chan string)
r.done.Add(1)
go func() {
defer r.done.Done()
for {
line, err := reader.ReadBytes('\n')
if err != nil {
fmt.Printf("error reading from named pipe: %v\n", err)
return
}
nline := string(line)
nline = strings.TrimRight(nline, "\n")
select {
case out <- nline:
case <-r.quit:
close(out)
return
}
}
}()
return out
}
func (r *Reader) Close() error {
close(r.quit)
r.done.Wait()
err := r.f.Close()
if err != nil {
return fmt.Errorf("error closing named pipe: %v", err)
}
err = r.rmFn()
if err != nil {
return fmt.Errorf("error removing named pipe: %v", err)
}
return nil
}
This does appear to be working, however it suffers from the peculiar behavior that no readers can open the file until any writers open the file, which appears to be the reverse behavior according to other things I've read on the topic; the usual complaint is that the writers are hanging because there aren't any readers, however, here a reader cannot be instantiated in the first place.
答案1
得分: 1
这是在POSIX系统接口文档中记录的默认行为:
> O_NONBLOCK 当使用O_RDONLY或O_WRONLY打开FIFO时:如果设置了O_NONBLOCK,读取时的open()将立即返回,不会延迟。写入时的open()将返回错误,如果没有进程当前打开该文件进行读取。
>
> 如果清除了O_NONBLOCK,读取时的open()将阻塞调用线程,直到有线程打开该文件进行写入。写入时的open()将阻塞调用线程,直到有线程打开该文件进行读取。
>
> 当打开支持非阻塞打开的块设备或字符设备文件时:
>
> 如果设置了O_NONBLOCK,open()函数将立即返回,而不会阻塞等待设备准备就绪或可用。设备的后续行为是特定于设备的。
>
> 如果清除了O_NONBLOCK,open()函数将阻塞调用线程,直到设备准备就绪或可用后再返回。
>
> 否则,O_NONBLOCK标志不会引发错误,但文件状态标志是否包括O_NONBLOCK标志是未指定的。
因此,解决方法是在OpenFile
调用中添加syscall.O_NONBLOCK
标志:
f, err := os.OpenFile(ipc, os.O_RDONLY|syscall.O_NONBLOCK, 0640)
编辑:如评论中所讨论的,这个解决方法在darwin
环境中不可移植。一个更可移植的解决方法是在读取端使用O_RDWR
打开文件。
英文:
This is the default behavior as documented in the POSIX system interfaces:
> O_NONBLOCK When opening a FIFO with O_RDONLY or O_WRONLY set: If
> O_NONBLOCK is set, an open() for reading-only shall return without
> delay. An open() for writing-only shall return an error if no process
> currently has the file open for reading.
>
> If O_NONBLOCK is clear, an open() for reading-only shall block the
> calling thread until a thread opens the file for writing. An open()
> for writing-only shall block the calling thread until a thread opens
> the file for reading.
>
> When opening a block special or character special file that supports
> non-blocking opens:
>
> If O_NONBLOCK is set, the open() function shall return without
> blocking for the device to be ready or available. Subsequent behavior
> of the device is device-specific.
>
> If O_NONBLOCK is clear, the open() function shall block the calling
l> thread until the device is ready or available before returning.
>
> Otherwise, the O_NONBLOCK flag shall not cause an error, but it is
> unspecified whether the file status flags will include the O_NONBLOCK
> flag.
The solution is, therefore, to add the syscall.O_NONBLOCK
flag to the OpenFile
call:
f, err := os.OpenFile(ipc, os.O_RDONLY|syscall.O_NONBLOCK, 0640)
EDIT: As discussed in the comments, this solution is not portable to darwin
environments. A more portable solution would be to open the file with O_RDWR
on the reader side.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论