Most efficient way to convert a [][]byte to []string in golang

huangapple go评论69阅读模式
英文:

Most efficient way to convert a [][]byte to []string in golang

问题

[][]byte转换为[]string,我这样做:

data, err := ioutil.ReadFile("test.txt")
if err != nil {
    return nil, err
}

db := bytes.Split(data, []byte("\n"))

// 将 [][]byte 转换为 []string
s := make([]string, len(db))
for i, val := range db {
    s[i] = string(val)
}
fmt.Printf("%v", s)

我是Go语言的新手,不确定这是否是最高效的方法。

英文:

To convert [][]byte to []string, I do this

data, err := ioutil.ReadFile("test.txt")
if err != nil {
	return nil, err
}

db := bytes.Split(data, []uint8("\n"))

// Convert [][]byte to []string
s := make([]string, len(db))
for i, val := range db {
	s[i] = string(val)
}
fmt.Printf("%v", s)

I am new to golang, I'm not sure is most efficient way to do this.

答案1

得分: 2

最有效的方法是删除这一步骤:db := bytes.Split(data, []uint8("\n")),而是像这样迭代data

func main() {
    data, _ := ioutil.ReadFile("test.txt")

    s := make([]string, 0)
    start := 0
    for i := range data {
        if data[i] == '\n' {
            elem := string(data[start : i-1])
            s = append(s, elem)
            start = i
        }
    }
    fmt.Printf("%v", s)
}

或者如果你想将[][]byte转换为[]string

func convert(data [][]byte) []string {
    s := make([]string, len(data))
    for row := range data {
        s[row] = string(data[row])
    }

    return s
}
英文:

The most effective way would be to remove this step: db := bytes.Split(data, []uint8("\n")) and instead iterate over data like that:

func main() {
	data, _ := ioutil.ReadFile("test.txt")

	s := make([]string, 0)
	start := 0
	for i := range data {
		if data[i] == '\n' {
			elem := string(data[start : i-1])
			s = append(s, elem)
			start = i
		}
	}
	fmt.Printf("%v", s)
}

Or if you want to convert [][]byte to []string:

func convert(data [][]byte) []string {
	s := make([]string, len(data))
	for row := range data {
		s[row] = string(data[row])
	}

	return s
}

答案2

得分: 1

如果你真的想将文件内容转换为[]string,你可以使用bufio.Scanner,它比你发布的代码更简洁(在我看来)和更高效:

func readFile(filename string) ([]string, error) {
    file, err := os.Open(filename)
    if err != nil {
        return nil, err
    }
    defer file.Close()

    scanner := bufio.NewScanner(file)

    var data []string

    for scanner.Scan() {
        line := scanner.Text()
        data = append(data, line)
    }
    if err = scanner.Err(); err != nil {
        return nil, err
    }

    return data, nil
}

这里有一个基准测试*,比较了原始函数(readFile1)和我的函数(readFile2):

BenchmarkReadFile1-8   	     300	   4632189 ns/op	 3035552 B/op	   10570 allocs/op
BenchmarkReadFile2-8   	    1000	   1695820 ns/op	 2169655 B/op	   10587 allocs/op

*该基准测试读取了一个大小为1.2 MiB、约10K行的示例文件。

新代码的运行时间是原始函数的36%,内存使用量是原始函数的71%。

英文:

If you actually want to convert a file content to a []string, you can use bufio.Scanner which is cleaner (IMO) and more efficient than the code you posted:

func readFile(filename string) ([]string, error) {
    file, err := os.Open(filename)
    if err != nil {
        return nil, err
    }   
    defer file.Close()

    scanner := bufio.NewScanner(file)

    var data []string

    for scanner.Scan() {
        line := scanner.Text()
        data = append(data, line)
    }   
    if err = scanner.Err(); err != nil {
        return nil, err
    }   

    return data, nil 
}

Here's a benchmark* comparing the original function (readFile1) and my function (readFile2):

BenchmarkReadFile1-8   	     300	   4632189 ns/op	 3035552 B/op	   10570 allocs/op
BenchmarkReadFile2-8   	    1000	   1695820 ns/op	 2169655 B/op	   10587 allocs/op

*the benchmark read a sample file of 1.2 MiB and ~10K lines

The new code runs in 36% of the time and 71% of the memory used by the original function.

huangapple
  • 本文由 发表于 2017年6月1日 15:32:31
  • 转载请务必保留本文链接:https://go.coder-hub.com/44301262.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定