英文:
Writing a reliable data store in GoLang
问题
我一直在为我要为教育目的构建的文档数据库系统的模块之一构建一个简单的数据存储。为了可靠地存储数据，我必须遵守ACID属性。下面是我的保存方法。
func (document Document) Save() (hash string, err error) {
if err := os.MkdirAll(document.FileDirectory(), 0600); err != nil {
return "", err
}
file, err := os.Create(document.TmpFile())
if err != nil {
return "", err
}
file.Write(document.Data)
if err := file.Sync(); err != nil {
return "", err
}
file.Close()
if err := os.Rename(document.TmpFile(), document.File()); err != nil {
return "", err
}
return document.Hash(), nil
}
首先，将数据（以[]byte形式）保存到临时文件中。然后使用file.Sync()将文件与持久存储同步，以确保数据被写入持久存储。然后将临时文件重命名为新文件。
注意：
我选择的存储数据文件的方式是以spoolDir格式。这意味着从数据生成的哈希的前两个字符用作父目录名。哈希的后两个字符用作后续目录名。文件名将是剩下的36个字符。临时文件只有一个后缀.tmp，文件路径和文件名相同。这个设计受到了git存储数据的启发。
问题：
我实现的数据存储算法的方式足以确保数据可靠持久化吗？
迄今为止的答案：
关于目录同步以确保数据持久性的一些内容（我不确定）
提前致谢
英文:
I have been building a simple data store as part of my module for a document database system which I'm going to build for educational purposes.
In order to store data reliably, I have to abide the ACID property. Shown below is my save method.
func (document Document) Save() (hash string, err error) {
if err := os.MkdirAll(document.FileDirectory(), 0600); err != nil {
return "", err
}
file, err := os.Create(document.TmpFile())
if err != nil {
return "", err
}
file.Write(document.Data)
if err := file.Sync(); err != nil {
return "", err
}
file.Close()
if err := os.Rename(document.TmpFile(), document.File()); err != nil {
return "", err
}
return document.Hash(), nil
}
First the data (in []byte) is saved to a temporary file. The file is then synced with file.Sync() to ensure the data is written to the persistent storage. Then the temporary file is renamed into the new file.
Note:
The way i chose to store the data file is in spoolDir format. Meaning the first two character of the hash generated from the data is used as parent directory name. The following two character of the hash is used as the subsequent directory name. The filename will be the 36 character left over. The temporary file only has a suffix .tmp with the file path and file name the same. This design is inspired by how git store data.
Question:
Is the way I implement the data storing algorithm sufficient to ensure the data is reliably persisted.
Answer so far:
Something about directory syncing to ensure data durability (I'm not sure)
Thanks in advance
Updated code as per suggested by rightfold:
func (document Document) Save() (hash string, err error) {
if err := os.MkdirAll(document.FileDirectory(), 0600); err != nil {
return "", err
}
file, err := os.Create(document.TmpFile())
if err != nil {
return "", err
}
file.Write(document.Data)
if err := file.Sync(); err != nil {
return "", err
}
file.Close()
if err := os.Rename(document.TmpFile(), document.File()); err != nil {
os.Remove(document.TmpFile())
return "", err
}
return document.Hash(), nil
}
答案1
得分: 4
你所做的工作保证了持久性，程度上取决于操作系统和硬件的保证（这是你能得到的最好的保证）。
它还是原子性的；即使CPU着火，不完整的写入也不会留下不完整的数据。
当重命名失败时，你可能希望删除临时文件：
if err := os.Rename(document.TmpFile(), document.File()); err != nil {
os.Remove(document.TmpFile()) // 忽略错误
return "", err
}
英文:
What you are doing guarantees durability to the degree the OS and hardware guarantee it (which is the best you can get).
It is also atomic; incomplete writes don’t leave incomplete data, even when the CPU catches fire.
You may want to delete the temporary file when renaming fails:
if err := os.Rename(document.TmpFile(), document.File()); err != nil {
os.Remove(document.TmpFile()) // ignore errors
return "", err
}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论