如何在Golang的filepath.walk()函数中设置递归迭代目录的深度?

huangapple go评论90阅读模式
英文:

How to set depth for recursive iteration of directories in filepath.walk() func in Golang?

问题

我想在一个包含多个子目录的目录中搜索特定类型的文件。我在Golang中使用filepath.walk()来实现这个功能。然而,我不想递归迭代到超过我知道文件不存在的最大深度。

在Golang中是否有这样的预构建函数/库?

英文:

I want to search for a specific type of file inside a directory which has various sub-directories in it. I'm using filepath.walk() in Golang for this. However, I don't want to iterate recursively beyond a max depth where I know that file can't exist.

Is there any such pre-built function/library in Golang?

答案1

得分: 4

首先,你应该使用Go 1.16中引入的filepath.WalkDir,它比filepath.Walk更高效。

> WalkDir比Walk更高效,它在访问每个文件或目录时避免了调用os.Lstat。

然后,没有直接的方法来指定最大深度作为参数。你需要在WalkDirFunc中计算递归深度。

显然,在文件路径中计算分隔符的数量是一种可接受的策略(可能比其他可能的技巧更简单),因此解决方案可能如下所示:

maxDepth := 2
rootDir := "root"
err := filepath.WalkDir(rootDir, func(path string, d fs.DirEntry, err error) error {
    if err != nil {
        // 处理可能的路径错误,以防万一...
        return err
    }
    if d.IsDir() && strings.Count(path, string(os.PathSeparator)) > maxDepth {
        fmt.Println("跳过", path)
        return fs.SkipDir
    }
    // ... 处理条目
    return nil
})

因此,假设root的深度为0,上述代码将打印:

跳过 root/root1/root2/root3
跳过 root/root1/root2/root4

请注意,这只是翻译的代码部分,不包括任何其他内容。

英文:

First, you should use filepath.WalkDir introduced in Go 1.16, which is more efficient than filepath.Walk.

> Walk is less efficient than WalkDir, introduced in Go 1.16, which avoids calling os.Lstat on every visited file or directory.

Then, there is no way to specify the max depth as a direct argument. You have to compute the recursion depth in the WalkDirFunc.

Apparently counting separators in the filepath is an acceptable strategy (and arguably simpler than other possible tricks), so the solution might look like:

	maxDepth := 2
    rootDir := "root"
	err := filepath.WalkDir(rootDir, func(path string, d fs.DirEntry, err error) error {
        if err != nil {
            // handle possible path err, just in case...
            return err
        }
		if d.IsDir() && strings.Count(path, string(os.PathSeparator)) > maxDepth {
			fmt.Println("skip", path)
			return fs.SkipDir
		}
		// ... process entry
        return nil
	})

So with dir structure as the following:

.
└── root
    ├── a.txt
    ├── b.txt
    └── root1
        ├── a.txt
        └── root2
            ├── a.txt
            ├── b.txt
            ├── root3
            │   └── a.txt
            └── root4

and assuming root is at depth 0, the above code the above code prints:

skip root/root1/root2/root3
skip root/root1/root2/root4

答案2

得分: -1

func ControlDeepWalk(basepath string, count int, hard, debug bool) error {
    var (
        stock       = make(map[string][]string)
        countBase   = 0
        correctdirs []string
    )
    base := filepath.Base(basepath)
    dirbase := filepath.Dir(basepath)
    countBase = len(strings.Split(filepath.Dir(basepath), "/"))
    if debug {
        log.Printf("countbase: %v  %v\n", strings.Split(filepath.Dir(basepath), "/"), countBase)
        log.Printf("base :%v : %v : %v\n", base, dirbase, strings.Split(basepath, "/"))
    }

    err := filepath.WalkDir(basepath, func(path string, d fs.DirEntry, err error) error {
        if err != nil {
            if debug {
                log.Printf("--error inf walkdir function, exit")
            }
            return err
        }
        if d.IsDir() {
            if filepath.Dir(path) == filepath.Dir(basepath) {
                if debug {
                    log.Printf("found root directory, skipping  %v :  %v\n", filepath.Dir(path), filepath.Dir(basepath))
                }
            } else {
                compare := false
                if hard {
                    compare = len(strings.Split(filepath.Dir(path), "/")) == countBase+count
                } else {
                    compare = len(strings.Split(filepath.Dir(path), "/")) <= countBase+count
                }
                if compare {
                    if debug {
                        log.Printf("-found dir: [%v] %v\n", path, d.Name())
                    }
                    stock[filepath.Dir(filepath.Join(path, d.Name()))] = []string{}
                    correctdirs = append(correctdirs, filepath.Dir(filepath.Join(path, d.Name())))
                }
            }
        } else {
            fdir, ffile := filepath.Split(path)
            for _, x := range correctdirs {
                if x == filepath.Dir(fdir) {
                    if debug {
                        log.Printf("-found file:%v  : %v   %v  %v \n", d.Name(), path, fdir, ffile)
                    }
                    stock[x] = append(stock[x], d.Name())
                }
            }
        }
        return nil
    })
    if debug {
        for k, v := range stock {
            log.Printf("%v : %v \n", k, v)
        }
        log.Printf("%v\n", stock)
    }
    return err
}

func main() {
    p := "/backup/backuper/test"
    count := 2
    _ = ControlDeepWalk(p, count, false, true)
}

enjoy, dude! :)
英文:
func ControlDeepWalk(basepath string, count int, hard, debug bool) error {
var (
stock       = make(map[string][]string)
countBase   = 0
correctdirs []string
)
base := filepath.Base(basepath)
dirbase := filepath.Dir(basepath)
countBase = len(strings.Split(filepath.Dir(basepath), &quot;/&quot;))
if debug {
log.Printf(&quot;countbase: %v  %v\n&quot;, strings.Split(filepath.Dir(basepath), &quot;/&quot;), countBase)
log.Printf(&quot;base :%v : %v : %v\n&quot;, base, dirbase, strings.Split(basepath, &quot;/&quot;))
}
err := filepath.WalkDir(basepath, func(path string, d fs.DirEntry, err error) error {
if err != nil {
if debug {
log.Printf(&quot;--error inf walkdir function, exit&quot;)
}
return err
}
if d.IsDir() {
if filepath.Dir(path) == filepath.Dir(basepath) {
if debug {
log.Printf(&quot;found root directory, skipping  %v :  %v\n&quot;, filepath.Dir(path), filepath.Dir(basepath))
}
} else {
compare := false
if hard {
compare = len(strings.Split(filepath.Dir(path), &quot;/&quot;)) == countBase+count
} else {
compare = len(strings.Split(filepath.Dir(path), &quot;/&quot;)) &lt;= countBase+count
}
if compare {
if debug {
log.Printf(&quot;-found dir: [%v] %v\n&quot;, path, d.Name())
}
stock[filepath.Dir(filepath.Join(path, d.Name()))] = []string{}
correctdirs = append(correctdirs, filepath.Dir(filepath.Join(path, d.Name())))
}
}
} else {
fdir, ffile := filepath.Split(path)
for _, x := range correctdirs {
if x == filepath.Dir(fdir) {
if debug {
log.Printf(&quot;-found file:%v  : %v   %v  %v \n&quot;, d.Name(), path, fdir, ffile)
}
stock[x] = append(stock[x], d.Name())
}
}
}
return nil
})
if debug {
for k, v := range stock {
log.Printf(&quot;%v : %v \n&quot;, k, v)
}
log.Printf(&quot;%v\n&quot;, stock)
}
return err
}
func main() {
p := &quot;/backup/backuper/test&quot;
count := 2
_ = ControlDeepWalk(p, count, false, true)
}

enjoy, dude! 如何在Golang的filepath.walk()函数中设置递归迭代目录的深度?

huangapple
  • 本文由 发表于 2022年2月17日 14:16:14
  • 转载请务必保留本文链接:https://go.coder-hub.com/71153302.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定