英文:
Is there an os-independent way to atomically overwrite a file?
问题
如果文件已经存在，我想要覆盖它。如果文件不存在，我想要创建它并写入内容。我希望不必使用第三方库，比如lockfile（它似乎可以处理各种类型的锁定）。
我的初始想法是：
- 使用随机生成的大型ID将内容写入临时文件，以避免冲突。
- 重命名临时文件名为新的路径名。
英文:
If the file already exists, I want to overwrite it. If it doesn't exist, I want to create it and write to it. I'd prefer to not have to use a 3rd party library like lockfile (which seems to handle all types of locking.)
My initial idea was to:
- Write to a temporary file with a randomly generated large id to avoid conflict.
- Rename the temp filename -> new path name.
答案1
得分: 12
os.Rename 调用 syscall.Rename，对于 Linux/UNIX 系统，它使用重命名系统调用（原子操作*）。在 Windows 上，syscall.Rename 调用 MoveFileW，假设源文件和目标文件在同一设备上（可以安排），并且文件系统是 NTFS（通常情况下），这也是原子操作*。
我建议确保源文件和目标文件在同一设备上，这样 Linux 的重命名操作就不会失败，而且 Windows 的重命名操作也是原子的。正如 Dave C 在上面提到的，创建临时文件（通常使用 ioutil.TempFile）在与现有文件相同的目录中是一个好方法；这就是我进行原子重命名的方式。
在我的使用场景中，这对我很有效：
在上述使用场景中，对我来说，简单地使用 os.Rename 工作得非常好。
以下是一些进一步的阅读材料：
- rename() 是否是原子操作？ “是和否。假设操作系统不崩溃，rename() 是原子的……”
- 在 Windows 上是否可能进行原子文件重命名（覆盖）？
注意：我想指出，当人们谈论原子文件系统文件*操作时，从应用程序的角度来看，他们通常指的是操作发生或不发生（这可以通过日志记录来帮助）。如果你在使用原子操作的意义上是指原子内存操作，那么很少有文件系统操作（除了使用直接 I/O [O_DIRECT] 进行禁用磁盘缓冲的单块写入和读取）可以被认为是真正的原子操作。
英文:
os.Rename calls syscall.Rename which for Linux/UNIXs uses the rename syscall (which is atomic*). On Windows syscall.Rename calls MoveFileW which assuming the source and destination are on the same device (which can be arranged) and the filesystem is NTFS (which is often the case) is atomic*.
I would take care to make sure the source and destination are on the same device so the Linux rename does not fail, and the Windows rename is actually atomic. As Dave C mentions above creating your temporary file (usually using ioutil.TempFile) in the same directory as existing file is the way to go; this is how I do my atomic renames.
This works for me in my use case which is:
- One Go process gets updates and renames files to swap updates in.
- Another Go process is watching for file updates with fsnotify and re-mmaps the file when it is updated.
In the above use case simply using os.Rename has worked perfectly well for me.
Some further reading:
- Is rename() atomic? "Yes and no. rename() is atomic assuming the OS does not crash...."
- Is an atomic file rename (with overwrite) possible on Windows?
*Note: I do want to point out that when people talk about atomic filesystem file operations, from an application perspective, they usually mean the operation happens or does not happen (which journaling can help with) from the users perspective. If you are using atomic in the sense of an atomic memory operation, very few filesystem operations (outside of direct I/O [O_DIRECT] one block writes and reads with disk buffering disabled) can be considered truly atomic.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论