Most efficient way to convert a [][]byte to []string in golang

huangapple
go评论111阅读模式
英文:

Most efficient way to convert a [][]byte to []string in golang

问题

将[][]byte转换为[]string，我这样做：

  1. data, err := ioutil.ReadFile("test.txt")
  2. if err != nil {
  3.     return nil, err
  4. }
  6. db := bytes.Split(data, []byte("\n"))
  8. // 将 [][]byte 转换为 []string
  9. s := make([]string, len(db))
  10. for i, val := range db {
  11.     s[i] = string(val)
  12. }
  13. fmt.Printf("%v", s)

我是Go语言的新手，不确定这是否是最高效的方法。

英文:

To convert [][]byte to []string, I do this

  1. data, err := ioutil.ReadFile("test.txt")
  2. if err != nil {
  3. 	return nil, err
  4. }
  6. db := bytes.Split(data, []uint8("\n"))
  8. // Convert [][]byte to []string
  9. s := make([]string, len(db))
  10. for i, val := range db {
  11. 	s[i] = string(val)
  12. }
  13. fmt.Printf("%v", s)

I am new to golang, I'm not sure is most efficient way to do this.

答案1

得分: 2

最有效的方法是删除这一步骤：db := bytes.Split(data, []uint8("\n"))，而是像这样迭代data：

  1. func main() {
  2.     data, _ := ioutil.ReadFile("test.txt")
  4.     s := make([]string, 0)
  5.     start := 0
  6.     for i := range data {
  7.         if data[i] == '\n' {
  8.             elem := string(data[start : i-1])
  9.             s = append(s, elem)
  10.             start = i
  11.         }
  12.     }
  13.     fmt.Printf("%v", s)
  14. }

或者如果你想将[][]byte转换为[]string：

  1. func convert(data [][]byte) []string {
  2.     s := make([]string, len(data))
  3.     for row := range data {
  4.         s[row] = string(data[row])
  5.     }
  7.     return s
  8. }
英文:

The most effective way would be to remove this step: db := bytes.Split(data, []uint8("\n")) and instead iterate over data like that:

  1. func main() {
  2. 	data, _ := ioutil.ReadFile("test.txt")
  4. 	s := make([]string, 0)
  5. 	start := 0
  6. 	for i := range data {
  7. 		if data[i] == '\n' {
  8. 			elem := string(data[start : i-1])
  9. 			s = append(s, elem)
  10. 			start = i
  11. 		}
  12. 	}
  13. 	fmt.Printf("%v", s)
  14. }

Or if you want to convert [][]byte to []string:

  1. func convert(data [][]byte) []string {
  2. 	s := make([]string, len(data))
  3. 	for row := range data {
  4. 		s[row] = string(data[row])
  5. 	}
  7. 	return s
  8. }

答案2

得分: 1

如果你真的想将文件内容转换为[]string，你可以使用bufio.Scanner，它比你发布的代码更简洁（在我看来）和更高效：

  1. func readFile(filename string) ([]string, error) {
  2.     file, err := os.Open(filename)
  3.     if err != nil {
  4.         return nil, err
  5.     }
  6.     defer file.Close()
  8.     scanner := bufio.NewScanner(file)
  10.     var data []string
  12.     for scanner.Scan() {
  13.         line := scanner.Text()
  14.         data = append(data, line)
  15.     }
  16.     if err = scanner.Err(); err != nil {
  17.         return nil, err
  18.     }
  20.     return data, nil
  21. }

这里有一个基准测试*，比较了原始函数(readFile1)和我的函数(readFile2)：

  1. BenchmarkReadFile1-8   	     300	   4632189 ns/op	 3035552 B/op	   10570 allocs/op
  2. BenchmarkReadFile2-8   	    1000	   1695820 ns/op	 2169655 B/op	   10587 allocs/op

*该基准测试读取了一个大小为1.2 MiB、约10K行的示例文件。

新代码的运行时间是原始函数的36%，内存使用量是原始函数的71%。

英文:

If you actually want to convert a file content to a []string, you can use bufio.Scanner which is cleaner (IMO) and more efficient than the code you posted:

  1. func readFile(filename string) ([]string, error) {
  2.     file, err := os.Open(filename)
  3.     if err != nil {
  4.         return nil, err
  5.     }   
  6.     defer file.Close()
  8.     scanner := bufio.NewScanner(file)
  10.     var data []string
  12.     for scanner.Scan() {
  13.         line := scanner.Text()
  14.         data = append(data, line)
  15.     }   
  16.     if err = scanner.Err(); err != nil {
  17.         return nil, err
  18.     }   
  20.     return data, nil 
  21. }

Here's a benchmark* comparing the original function (readFile1) and my function (readFile2):

  1. BenchmarkReadFile1-8   	     300	   4632189 ns/op	 3035552 B/op	   10570 allocs/op
  2. BenchmarkReadFile2-8   	    1000	   1695820 ns/op	 2169655 B/op	   10587 allocs/op

*the benchmark read a sample file of 1.2 MiB and ~10K lines

The new code runs in 36% of the time and 71% of the memory used by the original function.

huangapple
  • 本文由 发表于 2017年6月1日 15:32:31
  • 转载请务必保留本文链接：https://go.coder-hub.com/44301262.html
  • go
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定