Golang 从文件中读取数据 – 是否安全,不会出现锁定问题?

huangapple go评论65阅读模式
英文:

Golang reading from a file - is it safe from locking?

问题

我有一个函数,它会在每个HTTP GET请求上被调用。该函数读取一个文件,对文件内容进行一些处理,并返回这些内容的字节切片。然后,将该字节切片作为HTTP响应的响应体写入。

在这个函数的任何步骤中,我是否需要使用互斥锁来防止在多个HTTP请求尝试读取同一个文件时出现锁定?如果需要,是否只使用一个简单的RWMutex来锁定文件的读取就足够了,因为我实际上并没有对文件进行写入,而是创建了其内容的副本?

以下是该函数的代码:

// prepareIndex函数将获取index.html并为CSP头部合规性的脚本标签添加一个nonce。
func prepareIndex(nonce string) []byte {
    // 加载index.html文件。
    file, err := os.Open("./client/dist/index.html")
    if err != nil {
        log.Fatal(err)
    }

    // 转换为goquery文档。
    doc, err := goquery.NewDocumentFromReader(file)
    if err != nil {
        fmt.Println(err)
    }

    // 查找所有的脚本标签并设置nonce。
    doc.Find("body > script").SetAttr("nonce", nonce)

    // 获取HTML字符串。
    html, err := doc.Html()
    if err != nil {
        fmt.Println(err)
    }

    return []byte(html)
}

我也考虑过在main函数启动时只加载文件一次,但是我遇到了一个问题,只有第一个请求能够看到数据,而后续的请求则看不到任何内容。可能是我读取文件的方式有误。但实际上,我更喜欢当前的方法,因为如果对index.html进行任何更改,我希望这些更改能够立即传送给用户,而无需重新启动可执行文件。

英文:

I have a function that will be called on every single HTTP GET request. The function reads a file, does some stuff to the contents of that file, and returns a slice of bytes of those contents. That slice of bytes of then written as the response body to the HTTP response writer.

Do I need to use a mutex for any of the steps in this function to prevent locking in the event of multiple HTTP requests trying to read the same file? And if so, would a simple RWMutex locking the reading of the file suffice, since I am not actually writing to it but am creating a copy of its contents?

Here is the function:

// prepareIndex will grab index.html and add a nonce to the script tags for the CSP header compliance.
func prepareIndex(nonce string) []byte {
	// Load index.html.
	file, err := os.Open("./client/dist/index.html")
	if err != nil {
		log.Fatal(err)
	}

	// Convert to goquery document.
	doc, err := goquery.NewDocumentFromReader(file)
	if err != nil {
		fmt.Println(err)
	}

	// Find all script tags and set nonce.
	doc.Find("body > script").SetAttr("nonce", nonce)

	// Grab the HTML string.
	html, err := doc.Html()
	if err != nil {
		fmt.Println(err)
	}

	return []byte(html)
}

I also thought about just loading the file once when main starts, but I was having a problem where only the first request could see the data and the subsequent requests saw nothing. Probably an error in the way I was reading the file. But I actually prefer my current approach because if there are any changes to index.html, I want them to be persisted to the user immediately without having to restart the executable.

答案1

得分: 2

使用RWMutex无法保护文件免受其他程序的修改。在这里,最好的选择是在启动时将文件加载到[]byte中,并在使用goquery.NewDocumentFromReader时实例化"bytes".Buffer。为了将更改传播给用户,您可以使用fsnotify1检测文件的更改,并在必要时更新缓存的文件([]byte)(此操作需要使用RWMutex)。

例如:

type CachedFile struct {
    sync.RWMutex
    FileName string
    Content  []byte
    watcher  *fsnotify.Watcher
}

func (c *CachedFile) Buffer() *bytes.Buffer {
    c.RLock()
    defer c.RUnlock()
    return bytes.NewBuffer(c.Content)
}

func (c *CachedFile) Load() error {
    c.Lock()
    content, err := ioutil.ReadAll(c.FileName)
    if err != nil {
        c.Unlock()
        return err
    }
    c.Content = content
    c.Unlock()
}

func (c *CachedFile) Watch() error {
    var err error

    c.watcher, err = fsnotify.NewWatcher()
    if err != nil {
        return err
    }

    go func() {
        for ev := range c.watcher.Events {
            if ev.Op != fsnotify.Write {
                continue
            }
            err := c.Load()
            if err != nil {
                log.Printf("loading %q: %s", c.FileName, err)
            }
        }
    }()

    err = c.watcher.Add(c.FileName)
    if err != nil {
        c.watcher.Close()
        return err
    }

    return nil
}

func (c *CachedFile) Close() error {
    return c.watcher.Close()
}

1 https://godoc.org/github.com/fsnotify/fsnotify

英文:

Using RWMutex won't protect you from the file being modified by another program. The best option here would be to load your file in a []byte at startup, and instantiate "bytes".Buffer whenever you use goquery.NewDocumentFromReader. In order for the changes to be propagated to the user, you can use fsnotify1 to detect changes to your file, and update your cached file ([]byte) when necessary (you will need RWMutex for that operation).

For example:

type CachedFile struct {
	sync.RWMutex
	FileName string
	Content  []byte
	watcher  *fsnotify.Watcher
}

func (c *CachedFile) Buffer() *bytes.Buffer {
	c.RLock()
	defer c.RUnlock()
	return bytes.NewBuffer(c.Content)
}

func (c *CachedFile) Load() error {
	c.Lock()
	content, err := ioutil.ReadAll(c.FileName)
	if err != nil {
		c.Unlock()
		return err
	}
	c.Content = content
	c.Unlock()
}

func (c *CachedFile) Watch() error {
	var err error

	c.watcher, err = fsnotify.NewWatcher()
	if err != nil {
		return err
	}

	go func() {
		for ev := range c.watcher.Events {
			if ev.Op != fsnotify.Write {
				continue
			}
			err := c.Load()
			if err != nil {
				log.Printf("loading %q: %s", c.FileName, err)
			}
		}
	}()

	err = c.watcher.Add(c.FileName)
	if err != nil {
		c.watcher.Close()
		return err
	}

	return nil
}

func (c *CachedFile) Close() error {
	return c.watcher.Close()
}

1 https://godoc.org/github.com/fsnotify/fsnotify

答案2

得分: 1

如果你正在修改文件,你需要一个互斥锁。RWMutex 应该可以很好地工作。看起来你只是在读取文件,在这种情况下,你不应该看到任何锁定行为或损坏。

你第二次从同一个文件句柄读取时没有获取到任何数据的原因是,当你第二次开始读取时,你已经在文件的末尾。如果你想再次读取内容,你需要将偏移量0重新定位到文件开头。

英文:

If you're modifying the file, you need a mutex. RWMutex should work fine. It looks like you're just reading it, and in that case you should not see any locking behavior or corruption.

The reason you didn't get any data the second time you read from the same file handle is that you're already at the end of the file when you start reading from it the second time. You need to seek back to offset 0 if you want to read the contents again.

huangapple
  • 本文由 发表于 2017年7月17日 20:50:46
  • 转载请务必保留本文链接:https://go.coder-hub.com/45144788.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定