英文:
Get file path in Golang
问题
我需要读取目录中的所有文件，结构类似于：
/data
/folder1
somefiles.txt
/folder2
/folder2.1
somefiles.txt
/folder3
somefiles.txt
...
我尝试了walk方法，但是walk方法读取文件后，在完成文件处理后会删除信息并继续处理下一个文件。我真的需要将所有数据保存在一个对象中（我正在使用一个映射的切片），因为我需要使用一个API端点，当我批量插入数据时比逐个插入更快（在这些目录中有成千上万个文件，逐个插入需要很长时间）。
我的想法是编写一个递归函数（类似于walk），但我不知道如何获取文件路径，或者不知道是否可以在walk函数内部使用指针访问（我对GO语言真的很陌生），并将文件数据保存在全局变量或类似的地方。
英文:
I need to read all files inside a directory, the structure is something like:
/data
/folder1
somefiles.txt
/folder2
/folder2.1
somefiles.txt
/folder3
somefiles.txt
...
I tried the walk method, but walk reads the file and when it's done with the file it deletes the information and moves on to the next one. I really need to save it in an object (I'm using a slice of map) with all the data because I need to use an api endpoint and it's faster when I insert a lot of data than one by one (Inside the directories there are thousands of files so one by one takes a lot of time)
My idea is do a recursive function(like walk) but I don't know how to know the file path or I don't know if I can access with pointers inside walk function (I'm really new with GO) and save the file data in a global variable or something like that
答案1
得分: 1
要使用fs.WalkDir()，你只需要在处理程序中引用你需要在fs.WalkDir()执行完成后填充的变量即可。
例如，下面的代码递归遍历指定的目录树。忽略目录。一个map[string][]byte被填充，以每个文件的完全限定路径为键，包含每个文件的内容。
package main
import (
"fmt"
"io/fs"
"os"
"path"
"path/filepath"
)
func main() {
root := "/path/to/root/directory"
files := map[string][]byte{}
handler := func(p string, d fs.DirEntry, err error) error {
switch {
case err != nil:
return err
case shouldSkipDir(p, d):
return fs.SkipDir
case d.IsDir():
return nil
case shouldSkipFile(p, d):
return nil
default:
if fqn, e := filepath.Abs(path.Join(root, p)); e != nil {
return e
}
buf, e := os.ReadFile(fqn)
if e != nil {
return e
}
files[fqn] = buf
}
return nil
}
err := fs.WalkDir(os.DirFS(root), ".", handler)
if err != nil {
panic(err)
}
for fqn, contents := range files {
fmt.Printf("%s: %d bytes\n", fqn, len(contents))
}
}
func shouldSkipDir(path string, d fs.DirEntry) (skip bool) {
// TODO: 设置skip为true，表示应该跳过此目录。
// 如果当前目录条目是一个文件，并且此函数返回true，则其所在目录的其余部分将被跳过。
return skip
}
func shouldSkipFile(path string, d fs.DirEntry) (skip bool) {
// TODO: 设置skip为true，以忽略此文件
return skip
}
英文:
To use fs.WalkDir(), you just need to have your handler close over the variable(s) that you need to have populated when the execution of fs.WalkDir() is complete.
This, for instance, recursively walks the specified directory tree. Directories are ignored. A map[string][]byte is populated, keyed by the fully-qualified path of each file, containing the contents of each file.
package main
import (
"fmt"
"io/fs"
"os"
"path"
"path/filepath"
// "path"
)
func main() {
root := "/path/to/root/directory"
files := map[string][]byte{}
handler := func(p string, d fs.DirEntry, err error) error {
switch {
case err != nil:
return err
case shouldSkipDir(p, d):
return fs.SkipDir
case d.IsDir():
return nil
case shouldSkipFile(p, d):
return nil
default:
if fqn, e := filepath.Abs(path.Join(root, p)); e != nil {
return e
}
buf, e := os.ReadFile(fqn)
if e != nil {
return e
}
files[fqn] = buf
}
return nil
}
err := fs.WalkDir(os.DirFS(root), ".", handler)
if err != nil {
panic(err)
}
for fqn, contents := range files {
fmt.Printf("%s: %d bytes", fqn, len(contents))
}
}
func shouldSkipDir(path string, d fs.DirEntry) (skip bool) {
// TODO: set skip to true to indicate that this directory should be skipped.
// If the current direcotry entry is a file and this returns true, then
// the remainder of its containing directory are skipped.
return skip
}
func shouldSkipFile(path string, d fs.DirEntry) (skip bool) {
// TODO: set skip to true to ignore this file
return skip
}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论