如何在Go中高效地拼接字符串

huangapple go评论99阅读模式
英文:

How to efficiently concatenate strings in go

问题

在Go语言中,string是一种原始类型,这意味着它是只读的,对它的任何操作都会创建一个新的字符串。

所以,如果我想要多次连接字符串而不知道结果字符串的长度,最好的方法是什么?

一种简单的方法是:

var s string
for i := 0; i < 1000; i++ {
    s += getShortStringFromSomewhere()
}
return s

但这种方法似乎不太高效。

英文:

In Go, a string is a primitive type, which means it is read-only, and every manipulation of it will create a new string.

So if I want to concatenate strings many times without knowing the length of the resulting string, what's the best way to do it?

The naive way would be:

var s string
for i := 0; i &lt; 1000; i++ {
    s += getShortStringFromSomewhere()
}
return s

but that does not seem very efficient.

答案1

得分: 1043

新的方法:

从Go 1.10开始,有一个strings.Builder类型,请查看这个答案以获取更多细节

旧的方法:

使用bytes包。它有一个Buffer类型,实现了io.Writer接口。

package main

import (
    "bytes"
    "fmt"
)

func main() {
    var buffer bytes.Buffer

    for i := 0; i < 1000; i++ {
        buffer.WriteString("a")
    }

    fmt.Println(buffer.String())
}

这样做的时间复杂度为O(n)。

英文:

New Way:

From Go 1.10 there is a strings.Builder type, please take a look at this answer for more detail.

Old Way:

Use the bytes package. It has a Buffer type which implements io.Writer.

package main

import (
    &quot;bytes&quot;
    &quot;fmt&quot;
)

func main() {
    var buffer bytes.Buffer

    for i := 0; i &lt; 1000; i++ {
        buffer.WriteString(&quot;a&quot;)
    }

    fmt.Println(buffer.String())
}

This does it in O(n) time.

答案2

得分: 542

在Go 1.10+中,有**strings.Builder**,这里

> Builder用于使用Write方法高效地构建字符串。它最小化了内存复制。零值可以直接使用。


示例

它与bytes.Buffer几乎相同。

package main

import (
    "strings"
    "fmt"
)

func main() {
    // 零值:
    //
    // 它可以直接使用。
    // 你不需要初始化它。
    var sb strings.Builder

    for i := 0; i < 1000; i++ {
        sb.WriteString("a")
    }

    fmt.Println(sb.String())
}

点击在playground上查看


支持的接口

strings.Builder的方法是根据现有接口实现的,因此您可以在代码中轻松切换到新的Builder类型。

方法签名 接口 描述
Grow(int) bytes.Buffer 增加缓冲区的容量。更多信息请参见bytes.Buffer#Grow
Len() int bytes.Buffer 返回缓冲区中的字节数。更多信息请参见bytes.Buffer#Len
Reset() bytes.Buffer 将缓冲区重置为空。更多信息请参见bytes.Buffer#Reset
String() string fmt.Stringer 将缓冲区的内容作为字符串返回。更多信息请参见fmt.Stringer
Write([]byte) (int, error) io.Writer 将给定的字节写入缓冲区。更多信息请参见io.Writer
WriteByte(byte) error io.ByteWriter 将给定的字节写入缓冲区。更多信息请参见io.ByteWriter
WriteRune(rune) (int, error) bufio.Writerbytes.Buffer 将给定的符文写入缓冲区。更多信息请参见bufio.Writer#WriteRunebytes.Buffer#WriteRune
WriteString(string) (int, error) io.stringWriter 将给定的字符串写入缓冲区。更多信息请参见io.stringWriter

与bytes.Buffer的区别

  • 它只能增长或重置。
  • 它内置了copyCheck机制,防止意外复制。在bytes.Buffer中,可以像这样访问底层字节:(*Buffer).Bytes()strings.Builder解决了这个问题。不过有时,这不是一个问题,反而是期望的。例如:当字节被传递给io.Reader等时的查看行为。
  • bytes.Buffer.Reset() 重新定位并重用底层缓冲区,而strings.Builder.Reset() 不会,它会分离缓冲区。

注意

  • 不要复制strings.Builder值,因为它会缓存底层数据。
  • 如果要共享strings.Builder值,请使用指针。
英文:

In Go 1.10+ there is strings.Builder, here.

> A Builder is used to efficiently build a string using Write methods. It minimizes memory copying. The zero value is ready to use.


Example

It's almost the same with bytes.Buffer.

package main

import (
    &quot;strings&quot;
    &quot;fmt&quot;
)

func main() {
    // ZERO-VALUE:
    //
    // It&#39;s ready to use from the get-go.
    // You don&#39;t need to initialize it.
    var sb strings.Builder

    for i := 0; i &lt; 1000; i++ {
        sb.WriteString(&quot;a&quot;)
    }

    fmt.Println(sb.String())
}

Click to see this on the playground.


Supported Interfaces

strings.Builder's methods are being implemented with the existing interfaces in mind so that you can switch to the new Builder type easily in your code.

Method Signature Interface Description
Grow(int) bytes.Buffer Grows the buffer's capacity by the specified amount. See bytes.Buffer#Grow for more information.
Len() int bytes.Buffer Returns the number of bytes in the buffer. See bytes.Buffer#Len for more information.
Reset() bytes.Buffer Resets the buffer to be empty. See bytes.Buffer#Reset for more information.
String() string fmt.Stringer Returns the contents of the buffer as a string. See fmt.Stringer for more information.
Write([]byte) (int, error) io.Writer Writes the given bytes to the buffer. See io.Writer for more information.
WriteByte(byte) error io.ByteWriter Writes the given byte to the buffer. See io.ByteWriter for more information.
WriteRune(rune) (int, error) bufio.Writer or bytes.Buffer Writes the given rune to the buffer. See bufio.Writer#WriteRune or bytes.Buffer#WriteRune for more information.
WriteString(string) (int, error) io.stringWriter Writes the given string to the buffer. See io.stringWriter for more information.

Differences from bytes.Buffer

  • It can only grow or reset.
  • It has a copyCheck mechanism built-in that prevents accidentally copying it. In bytes.Buffer, one can access the underlying bytes like this: (*Buffer).Bytes(). strings.Builder prevents this problem. Sometimes, this is not a problem, though, and is desired instead. For example: For the peeking behavior when the bytes are passed to an io.Reader etc.
  • bytes.Buffer.Reset() rewinds and reuses the underlying buffer whereas the strings.Builder.Reset() does not, it detaches the buffer.

Note

  • Do not copy a strings.Builder value as it caches the underlying data.
  • If you want to share a strings.Builder value, use a pointer to it.

Check out its source code for more details, here.

答案3

得分: 293

如果您知道要预分配的字符串的总长度,那么最高效的字符串连接方式可能是使用内置函数copy。如果您事先不知道总长度,请不要使用copy,而是阅读其他答案。

在我的测试中,这种方法比使用bytes.Buffer快约3倍,比使用运算符+快得多(约12,000倍)。而且,它使用的内存更少。

我创建了一个测试案例来证明这一点,以下是结果:

<!-- language: lang-none -->

BenchmarkConcat	 1000000	64497 ns/op   502018 B/op   0 allocs/op
BenchmarkBuffer	 100000000	15.5  ns/op	  2 B/op	    0 allocs/op
BenchmarkCopy	 500000000	5.39  ns/op	  0 B/op	    0 allocs/op

以下是用于测试的代码:

package main

import (
	&quot;bytes&quot;
	&quot;strings&quot;
	&quot;testing&quot;
)

func BenchmarkConcat(b *testing.B) {
	var str string
	for n := 0; n &lt; b.N; n++ {
		str += &quot;x&quot;
	}
	b.StopTimer()

	if s := strings.Repeat(&quot;x&quot;, b.N); str != s {
		b.Errorf(&quot;unexpected result; got=%s, want=%s&quot;, str, s)
	}
}

func BenchmarkBuffer(b *testing.B) {
	var buffer bytes.Buffer
	for n := 0; n &lt; b.N; n++ {
		buffer.WriteString(&quot;x&quot;)
	}
	b.StopTimer()

	if s := strings.Repeat(&quot;x&quot;, b.N); buffer.String() != s {
		b.Errorf(&quot;unexpected result; got=%s, want=%s&quot;, buffer.String(), s)
	}
}

func BenchmarkCopy(b *testing.B) {
	bs := make([]byte, b.N)
	bl := 0

	b.ResetTimer()
	for n := 0; n &lt; b.N; n++ {
		bl += copy(bs[bl:], &quot;x&quot;)
	}
	b.StopTimer()

	if s := strings.Repeat(&quot;x&quot;, b.N); string(bs) != s {
		b.Errorf(&quot;unexpected result; got=%s, want=%s&quot;, string(bs), s)
	}
}

// Go 1.10
func BenchmarkStringBuilder(b *testing.B) {
	var strBuilder strings.Builder

	b.ResetTimer()
	for n := 0; n &lt; b.N; n++ {
		strBuilder.WriteString(&quot;x&quot;)
	}
	b.StopTimer()

	if s := strings.Repeat(&quot;x&quot;, b.N); strBuilder.String() != s {
		b.Errorf(&quot;unexpected result; got=%s, want=%s&quot;, strBuilder.String(), s)
	}
}
英文:

If you know the total length of the string that you're going to preallocate then the most efficient way to concatenate strings may be using the builtin function copy. If you don't know the total length before hand, do not use copy, and read the other answers instead.

In my tests, that approach is ~3x faster than using bytes.Buffer and much much faster (~12,000x) than using the operator +. Also, it uses less memory.

I've created a test case to prove this and here are the results:

<!-- language: lang-none -->

BenchmarkConcat	 1000000	64497 ns/op   502018 B/op   0 allocs/op
BenchmarkBuffer	 100000000	15.5  ns/op	  2 B/op	    0 allocs/op
BenchmarkCopy	 500000000	5.39  ns/op	  0 B/op	    0 allocs/op

Below is code for testing:

package main

import (
	&quot;bytes&quot;
	&quot;strings&quot;
	&quot;testing&quot;
)

func BenchmarkConcat(b *testing.B) {
	var str string
	for n := 0; n &lt; b.N; n++ {
		str += &quot;x&quot;
	}
	b.StopTimer()

	if s := strings.Repeat(&quot;x&quot;, b.N); str != s {
		b.Errorf(&quot;unexpected result; got=%s, want=%s&quot;, str, s)
	}
}

func BenchmarkBuffer(b *testing.B) {
	var buffer bytes.Buffer
	for n := 0; n &lt; b.N; n++ {
		buffer.WriteString(&quot;x&quot;)
	}
	b.StopTimer()

	if s := strings.Repeat(&quot;x&quot;, b.N); buffer.String() != s {
		b.Errorf(&quot;unexpected result; got=%s, want=%s&quot;, buffer.String(), s)
	}
}

func BenchmarkCopy(b *testing.B) {
	bs := make([]byte, b.N)
	bl := 0

	b.ResetTimer()
	for n := 0; n &lt; b.N; n++ {
		bl += copy(bs[bl:], &quot;x&quot;)
	}
	b.StopTimer()

	if s := strings.Repeat(&quot;x&quot;, b.N); string(bs) != s {
		b.Errorf(&quot;unexpected result; got=%s, want=%s&quot;, string(bs), s)
	}
}

// Go 1.10
func BenchmarkStringBuilder(b *testing.B) {
	var strBuilder strings.Builder

	b.ResetTimer()
	for n := 0; n &lt; b.N; n++ {
		strBuilder.WriteString(&quot;x&quot;)
	}
	b.StopTimer()

	if s := strings.Repeat(&quot;x&quot;, b.N); strBuilder.String() != s {
		b.Errorf(&quot;unexpected result; got=%s, want=%s&quot;, strBuilder.String(), s)
	}
}

答案4

得分: 151

如果您有一个字符串切片,想要高效地将其转换为字符串,您可以使用以下方法。否则,请查看其他答案。

在strings包中有一个名为Join的库函数:
http://golang.org/pkg/strings/#Join

查看Join的代码可以看到与Kinopiko编写的Append函数类似的方法:https://golang.org/src/strings/strings.go#L420

用法:

import (
    "fmt";
    "strings";
)

func main() {
    s := []string{"this", "is", "a", "joined", "string\n"};
    fmt.Printf(strings.Join(s, " "));
}

$ ./test.bin
this is a joined string
英文:

If you have a string slice that you want to efficiently convert to a string then you can use this approach. Otherwise, take a look at the other answers.

There is a library function in the strings package called Join:
http://golang.org/pkg/strings/#Join

A look at the code of Join shows a similar approach to Append function Kinopiko wrote: https://golang.org/src/strings/strings.go#L420

Usage:

import (
    &quot;fmt&quot;;
    &quot;strings&quot;;
)

func main() {
    s := []string{&quot;this&quot;, &quot;is&quot;, &quot;a&quot;, &quot;joined&quot;, &quot;string\n&quot;};
    fmt.Printf(strings.Join(s, &quot; &quot;));
}

$ ./test.bin
this is a joined string

答案5

得分: 47

我刚刚在我的代码中对上面发布的最佳答案进行了基准测试(递归树遍历),结果发现简单的连接运算符比BufferString更快。

func (r *record) String() string {
    buffer := bytes.NewBufferString("")
    fmt.Fprint(buffer, "(", r.name, "[")
    for i := 0; i < len(r.subs); i++ {
        fmt.Fprint(buffer, "\t", r.subs[i])
    }
    fmt.Fprint(buffer, "]", r.size, ")\n")
    return buffer.String()
}

这个方法花费了0.81秒,而下面的代码:

func (r *record) String() string {
    s := "(" + r.name + " ["
    for i := 0; i < len(r.subs); i++ {
        s += r.subs[i].String()
    }
    s += "] " + strconv.FormatInt(r.size, 10) + ")\n"
    return s
}

只花费了0.61秒。这可能是因为创建新的BufferString的开销。

更新: 我还对join函数进行了基准测试,它运行时间为0.54秒。

func (r *record) String() string {
    var parts []string
    parts = append(parts, "(", r.name, " [")
    for i := 0; i < len(r.subs); i++ {
        parts = append(parts, r.subs[i].String())
    }
    parts = append(parts, strconv.FormatInt(r.size, 10), ")\n")
    return strings.Join(parts, "")
}
英文:

I just benchmarked the top answer posted above in my own code (a recursive tree walk) and the simple concat operator is actually faster than the BufferString.

func (r *record) String() string {
	buffer := bytes.NewBufferString(&quot;&quot;);
	fmt.Fprint(buffer,&quot;(&quot;,r.name,&quot;[&quot;)
	for i := 0; i &lt; len(r.subs); i++ {
		fmt.Fprint(buffer,&quot;\t&quot;,r.subs[i])
	}
	fmt.Fprint(buffer,&quot;]&quot;,r.size,&quot;)\n&quot;)
	return buffer.String()
}

This took 0.81 seconds, whereas the following code:

func (r *record) String() string {
	s := &quot;(\&quot;&quot; + r.name + &quot;\&quot; [&quot;
	for i := 0; i &lt; len(r.subs); i++ {
		s += r.subs[i].String()
	}
	s += &quot;] &quot; + strconv.FormatInt(r.size,10) + &quot;)\n&quot;
	return s
} 

only took 0.61 seconds. This is probably due to the overhead of creating the new BufferString.

Update: I also benchmarked the join function and it ran in 0.54 seconds.

func (r *record) String() string {
	var parts []string
	parts = append(parts, &quot;(\&quot;&quot;, r.name, &quot;\&quot; [&quot; )
	for i := 0; i &lt; len(r.subs); i++ {
		parts = append(parts, r.subs[i].String())
	}
	parts = append(parts, strconv.FormatInt(r.size,10), &quot;)\n&quot;)
	return strings.Join(parts,&quot;&quot;)
}

答案6

得分: 31

包主要

导入 (
"fmt"
)

功能 主要() {
var str1 = "字符串1"
var str2 = "字符串2"
out := fmt.Sprintf("%s %s ",str1, str2)
fmt.Println(out)
}

英文:
package main

import (
  &quot;fmt&quot;
)

func main() {
    var str1 = &quot;string1&quot;
    var str2 = &quot;string2&quot;
    out := fmt.Sprintf(&quot;%s %s &quot;,str1, str2)
    fmt.Println(out)
}

答案7

得分: 26

这是最快的解决方案,不需要你知道或计算整个缓冲区的大小:

var data []byte
for i := 0; i < 1000; i++ {
    data = append(data, getShortStringFromSomewhere()...)
}
return string(data)

根据我的基准测试,它比复制解决方案慢20%(每次追加需要8.1纳秒,而不是6.72纳秒),但仍然比使用bytes.Buffer快55%。

英文:

This is the fastest solution that does not require
you to know or calculate the overall buffer size first:

var data []byte
for i := 0; i &lt; 1000; i++ {
    data = append(data, getShortStringFromSomewhere()...)
}
return string(data)

By my benchmark, it's 20% slower than the copy solution (8.1ns per
append rather than 6.72ns) but still 55% faster than using bytes.Buffer.

答案8

得分: 25

你可以创建一个大的字节切片,并使用字符串切片将短字符串的字节复制到其中。在《Effective Go》中提供了一个函数:

func Append(slice, data[]byte) []byte {
    l := len(slice);
    if l + len(data) > cap(slice) {    // 重新分配内存
        // 分配比所需空间多一倍的内存,以供未来扩展。
        newSlice := make([]byte, (l+len(data))*2);
        // 复制数据(可以使用 bytes.Copy())。
        for i, c := range slice {
            newSlice[i] = c
        }
        slice = newSlice;
    }
    slice = slice[0:l+len(data)];
    for i, c := range data {
        slice[l+i] = c
    }
    return slice;
}

然后在操作完成后,使用 string() 将大的字节切片转换回字符串。

英文:

You could create a big slice of bytes and copy the bytes of the short strings into it using string slices. There is a function given in "Effective Go":

func Append(slice, data[]byte) []byte {
	l := len(slice);
	if l + len(data) &gt; cap(slice) {	// reallocate
		// Allocate double what&#39;s needed, for future growth.
		newSlice := make([]byte, (l+len(data))*2);
		// Copy data (could use bytes.Copy()).
		for i, c := range slice {
			newSlice[i] = c
		}
		slice = newSlice;
	}
	slice = slice[0:l+len(data)];
	for i, c := range data {
		slice[l+i] = c
	}
	return slice;
}

Then when the operations are finished, use string ( ) on the big slice of bytes to convert it into a string again.

答案9

得分: 24

2018年添加的注释

从Go 1.10开始,有一个strings.Builder类型,请参考此答案了解更多细节

201x年之前的答案

@cd1和其他答案的基准代码是错误的。基准函数中不应该设置b.N,它是由go test工具动态设置的,用于确定测试的执行时间是否稳定。

基准函数应该运行相同的测试b.N次,循环内部的测试对于每次迭代都应该是相同的。所以我通过添加一个内部循环来修复它。我还为其他一些解决方案添加了基准测试:

package main

import (
	"bytes"
	"strings"
	"testing"
)

const (
	sss = "xfoasneobfasieongasbg"
	cnt = 10000
)

var (
	bbb      = []byte(sss)
	expected = strings.Repeat(sss, cnt)
)

func BenchmarkCopyPreAllocate(b *testing.B) {
	var result string
	for n := 0; n < b.N; n++ {
		bs := make([]byte, cnt*len(sss))
		bl := 0
		for i := 0; i < cnt; i++ {
			bl += copy(bs[bl:], sss)
		}
		result = string(bs)
	}
	b.StopTimer()
	if result != expected {
		b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
	}
}

func BenchmarkAppendPreAllocate(b *testing.B) {
	var result string
	for n := 0; n < b.N; n++ {
		data := make([]byte, 0, cnt*len(sss))
		for i := 0; i < cnt; i++ {
			data = append(data, sss...)
		}
		result = string(data)
	}
	b.StopTimer()
	if result != expected {
		b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
	}
}

func BenchmarkBufferPreAllocate(b *testing.B) {
	var result string
	for n := 0; n < b.N; n++ {
		buf := bytes.NewBuffer(make([]byte, 0, cnt*len(sss)))
		for i := 0; i < cnt; i++ {
			buf.WriteString(sss)
		}
		result = buf.String()
	}
	b.StopTimer()
	if result != expected {
		b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
	}
}

func BenchmarkCopy(b *testing.B) {
	var result string
	for n := 0; n < b.N; n++ {
		data := make([]byte, 0, 64) // 与bytes.Buffer的引导数组大小相同
		for i := 0; i < cnt; i++ {
			off := len(data)
			if off+len(sss) > cap(data) {
				temp := make([]byte, 2*cap(data)+len(sss))
				copy(temp, data)
				data = temp
			}
			data = data[0 : off+len(sss)]
			copy(data[off:], sss)
		}
		result = string(data)
	}
	b.StopTimer()
	if result != expected {
		b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
	}
}

func BenchmarkAppend(b *testing.B) {
	var result string
	for n := 0; n < b.N; n++ {
		data := make([]byte, 0, 64)
		for i := 0; i < cnt; i++ {
			data = append(data, sss...)
		}
		result = string(data)
	}
	b.StopTimer()
	if result != expected {
		b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
	}
}

func BenchmarkBufferWrite(b *testing.B) {
	var result string
	for n := 0; n < b.N; n++ {
		var buf bytes.Buffer
		for i := 0; i < cnt; i++ {
			buf.Write(bbb)
		}
		result = buf.String()
	}
	b.StopTimer()
	if result != expected {
		b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
	}
}

func BenchmarkBufferWriteString(b *testing.B) {
	var result string
	for n := 0; n < b.N; n++ {
		var buf bytes.Buffer
		for i := 0; i < cnt; i++ {
			buf.WriteString(sss)
		}
		result = buf.String()
	}
	b.StopTimer()
	if result != expected {
		b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
	}
}

func BenchmarkConcat(b *testing.B) {
	var result string
	for n := 0; n < b.N; n++ {
		var str string
		for i := 0; i < cnt; i++ {
			str += sss
		}
		result = str
	}
	b.StopTimer()
	if result != expected {
		b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
	}
}

环境为OS X 10.11.6,2.2 GHz Intel Core i7

测试结果:

BenchmarkCopyPreAllocate-8         20000             84208 ns/op          425984 B/op          2 allocs/op
BenchmarkAppendPreAllocate-8       10000            102859 ns/op          425984 B/op          2 allocs/op
BenchmarkBufferPreAllocate-8       10000            166407 ns/op          426096 B/op          3 allocs/op
BenchmarkCopy-8                    10000            160923 ns/op          933152 B/op         13 allocs/op
BenchmarkAppend-8                  10000            175508 ns/op         1332096 B/op         24 allocs/op
BenchmarkBufferWrite-8             10000            239886 ns/op          933266 B/op         14 allocs/op
BenchmarkBufferWriteString-8       10000            236432 ns/op          933266 B/op         14 allocs/op
BenchmarkConcat-8                     10         105603419 ns/op        1086685168 B/op    10000 allocs/op

结论:

  1. CopyPreAllocate是最快的方法;AppendPreAllocate非常接近第一名,但编写代码更容易。
  2. Concat的性能非常差,无论是速度还是内存使用。不要使用它。
  3. Buffer#WriteBuffer#WriteString在速度上基本相同,与@Dani-Br在评论中所说的相反。考虑到在Go中string实际上是[]byte,这是有道理的。
  4. bytes.Buffer基本上使用与Copy相同的解决方案,还有额外的书籍管理和其他内容。
  5. CopyAppend使用与bytes.Buffer相同的引导大小为64。
  6. Append使用更多的内存和分配,我认为这与它使用的增长算法有关。它的内存增长速度不如bytes.Buffer快。

建议:

  1. 对于像OP所需的简单任务,我会使用AppendAppendPreAllocate。它足够快且易于使用。
  2. 如果需要同时读取和写入缓冲区,请使用bytes.Buffer。这就是它的设计目的。
英文:

Note added in 2018

From Go 1.10 there is a strings.Builder type, please take a look at this answer for more detail.

Pre-201x answer

The benchmark code of @cd1 and other answers are wrong. b.N is not supposed to be set in benchmark function. It's set by the go test tool dynamically to determine if the execution time of the test is stable.

A benchmark function should run the same test b.N times and the test inside the loop should be the same for each iteration. So I fix it by adding an inner loop. I also add benchmarks for some other solutions:

package main

import (
	&quot;bytes&quot;
	&quot;strings&quot;
	&quot;testing&quot;
)

const (
	sss = &quot;xfoasneobfasieongasbg&quot;
	cnt = 10000
)

var (
	bbb      = []byte(sss)
	expected = strings.Repeat(sss, cnt)
)

func BenchmarkCopyPreAllocate(b *testing.B) {
	var result string
	for n := 0; n &lt; b.N; n++ {
		bs := make([]byte, cnt*len(sss))
		bl := 0
		for i := 0; i &lt; cnt; i++ {
			bl += copy(bs[bl:], sss)
		}
		result = string(bs)
	}
	b.StopTimer()
	if result != expected {
		b.Errorf(&quot;unexpected result; got=%s, want=%s&quot;, string(result), expected)
	}
}

func BenchmarkAppendPreAllocate(b *testing.B) {
	var result string
	for n := 0; n &lt; b.N; n++ {
		data := make([]byte, 0, cnt*len(sss))
		for i := 0; i &lt; cnt; i++ {
			data = append(data, sss...)
		}
		result = string(data)
	}
	b.StopTimer()
	if result != expected {
		b.Errorf(&quot;unexpected result; got=%s, want=%s&quot;, string(result), expected)
	}
}

func BenchmarkBufferPreAllocate(b *testing.B) {
	var result string
	for n := 0; n &lt; b.N; n++ {
		buf := bytes.NewBuffer(make([]byte, 0, cnt*len(sss)))
		for i := 0; i &lt; cnt; i++ {
			buf.WriteString(sss)
		}
		result = buf.String()
	}
	b.StopTimer()
	if result != expected {
		b.Errorf(&quot;unexpected result; got=%s, want=%s&quot;, string(result), expected)
	}
}

func BenchmarkCopy(b *testing.B) {
	var result string
	for n := 0; n &lt; b.N; n++ {
		data := make([]byte, 0, 64) // same size as bootstrap array of bytes.Buffer
		for i := 0; i &lt; cnt; i++ {
			off := len(data)
			if off+len(sss) &gt; cap(data) {
				temp := make([]byte, 2*cap(data)+len(sss))
				copy(temp, data)
				data = temp
			}
			data = data[0 : off+len(sss)]
			copy(data[off:], sss)
		}
		result = string(data)
	}
	b.StopTimer()
	if result != expected {
		b.Errorf(&quot;unexpected result; got=%s, want=%s&quot;, string(result), expected)
	}
}

func BenchmarkAppend(b *testing.B) {
	var result string
	for n := 0; n &lt; b.N; n++ {
		data := make([]byte, 0, 64)
		for i := 0; i &lt; cnt; i++ {
			data = append(data, sss...)
		}
		result = string(data)
	}
	b.StopTimer()
	if result != expected {
		b.Errorf(&quot;unexpected result; got=%s, want=%s&quot;, string(result), expected)
	}
}

func BenchmarkBufferWrite(b *testing.B) {
	var result string
	for n := 0; n &lt; b.N; n++ {
		var buf bytes.Buffer
		for i := 0; i &lt; cnt; i++ {
			buf.Write(bbb)
		}
		result = buf.String()
	}
	b.StopTimer()
	if result != expected {
		b.Errorf(&quot;unexpected result; got=%s, want=%s&quot;, string(result), expected)
	}
}

func BenchmarkBufferWriteString(b *testing.B) {
	var result string
	for n := 0; n &lt; b.N; n++ {
		var buf bytes.Buffer
		for i := 0; i &lt; cnt; i++ {
			buf.WriteString(sss)
		}
		result = buf.String()
	}
	b.StopTimer()
	if result != expected {
		b.Errorf(&quot;unexpected result; got=%s, want=%s&quot;, string(result), expected)
	}
}

func BenchmarkConcat(b *testing.B) {
	var result string
	for n := 0; n &lt; b.N; n++ {
		var str string
		for i := 0; i &lt; cnt; i++ {
			str += sss
		}
		result = str
	}
	b.StopTimer()
	if result != expected {
		b.Errorf(&quot;unexpected result; got=%s, want=%s&quot;, string(result), expected)
	}
}

Environment is OS X 10.11.6, 2.2 GHz Intel Core i7

Test results:

BenchmarkCopyPreAllocate-8         20000             84208 ns/op          425984 B/op          2 allocs/op
BenchmarkAppendPreAllocate-8       10000            102859 ns/op          425984 B/op          2 allocs/op
BenchmarkBufferPreAllocate-8       10000            166407 ns/op          426096 B/op          3 allocs/op
BenchmarkCopy-8                    10000            160923 ns/op          933152 B/op         13 allocs/op
BenchmarkAppend-8                  10000            175508 ns/op         1332096 B/op         24 allocs/op
BenchmarkBufferWrite-8             10000            239886 ns/op          933266 B/op         14 allocs/op
BenchmarkBufferWriteString-8       10000            236432 ns/op          933266 B/op         14 allocs/op
BenchmarkConcat-8                     10         105603419 ns/op        1086685168 B/op    10000 allocs/op

Conclusion:

  1. CopyPreAllocate is the fastest way; AppendPreAllocate is pretty close to No.1, but it's easier to write the code.
  2. Concat has really bad performance both for speed and memory usage. Don't use it.
  3. Buffer#Write and Buffer#WriteString are basically the same in speed, contrary to what @Dani-Br said in the comment. Considering string is indeed []byte in Go, it makes sense.
  4. bytes.Buffer basically use the same solution as Copy with extra book keeping and other stuff.
  5. Copy and Append use a bootstrap size of 64, the same as bytes.Buffer
  6. Append use more memory and allocs, I think it's related to the grow algorithm it use. It's not growing memory as fast as bytes.Buffer

Suggestion:

  1. For simple task such as what OP wants, I would use Append or AppendPreAllocate . It's fast enough and easy to use.
  2. If need to read and write the buffer at the same time, use bytes.Buffer of course. That's what it's designed for.

答案10

得分: 14

我的原始建议是

s12 := fmt.Sprint(s1,s2)

但是上面的答案使用了<a href="http://golang.org/pkg/bytes/#Buffer.WriteString" >bytes.Buffer - WriteString()</a> 是最高效的方法。

我的初始建议使用了反射和类型切换。<a href="https://golang.org/src/pkg/fmt/print.go">查看 (p *pp) doPrint(p *pp) printArg</a><br>
对于基本类型,没有通用的Stringer()接口,这是我天真地认为的。

至少,Sprint() 内部 使用了一个bytes.Buffer。因此

`s12 := fmt.Sprint(s1,s2,s3,s4,...,s1000)`

在内存分配方面是可接受的。

=> Sprint() 连接可以用于快速调试输出。<br>
=> 否则使用 bytes.Buffer ... WriteString

英文:

My original suggestion was

s12 := fmt.Sprint(s1,s2)

But above answer using <a href="http://golang.org/pkg/bytes/#Buffer.WriteString" >bytes.Buffer - WriteString()</a> is the most efficient way.

My initial suggestion uses reflection and a type switch. <a href="https://golang.org/src/pkg/fmt/print.go">See (p *pp) doPrint and (p *pp) printArg</a><br>
There is no universal Stringer() interface for basic types, as I had naively thought.

At least though, Sprint() internally uses a bytes.Buffer. Thus

`s12 := fmt.Sprint(s1,s2,s3,s4,...,s1000)`

is acceptable in terms of memory allocations.

=> Sprint() concatenation can be used for quick debug output.<br>
=> Otherwise use bytes.Buffer ... WriteString

答案11

得分: 11

在cd1的答案上进行扩展:
你可以使用append()代替copy()。
append()提供了更大的预留空间,会消耗一些内存,但可以节省时间。
我在你的代码顶部添加了另外两个基准测试
在本地运行:

go test -bench=. -benchtime=100ms

在我的ThinkPad T400s上运行结果如下:

BenchmarkAppendEmpty    50000000         5.0 ns/op
BenchmarkAppendPrealloc 50000000         3.5 ns/op
BenchmarkCopy           20000000        10.2 ns/op
英文:

Expanding on cd1's answer:
You might use append() instead of copy().
append() makes ever bigger advance provisions, costing a little more memory, but saving time.
I added two more benchmarks at the top of yours.
Run locally with

go test -bench=. -benchtime=100ms

On my thinkpad T400s it yields:

BenchmarkAppendEmpty    50000000         5.0 ns/op
BenchmarkAppendPrealloc 50000000         3.5 ns/op
BenchmarkCopy           20000000        10.2 ns/op

答案12

得分: 4

这是由@cd1提供的实际版本的基准测试(Go 1.8, linux x86_64),修复了@icza和@PickBoy提到的错误。

Bytes.Buffer只比直接使用+运算符进行字符串连接快7倍。

package performance_test

import (
	"bytes"
	"fmt"
	"testing"
)

const (
	concatSteps = 100
)

func BenchmarkConcat(b *testing.B) {
	for n := 0; n < b.N; n++ {
		var str string
		for i := 0; i < concatSteps; i++ {
			str += "x"
		}
	}
}

func BenchmarkBuffer(b *testing.B) {
	for n := 0; n < b.N; n++ {
		var buffer bytes.Buffer
		for i := 0; i < concatSteps; i++ {
			buffer.WriteString("x")
		}
	}
}

时间:

BenchmarkConcat-4                       	  300000	      6869 ns/op
BenchmarkBuffer-4                       	 1000000	      1186 ns/op
英文:

This is actual version of benchmark provided by @cd1 (Go 1.8, linux x86_64) with the fixes of bugs mentioned by @icza and @PickBoy.

Bytes.Buffer is only 7 times faster than direct string concatenation via + operator.

package performance_test

import (
	&quot;bytes&quot;
	&quot;fmt&quot;
	&quot;testing&quot;
)

const (
	concatSteps = 100
)

func BenchmarkConcat(b *testing.B) {
	for n := 0; n &lt; b.N; n++ {
		var str string
		for i := 0; i &lt; concatSteps; i++ {
			str += &quot;x&quot;
		}
	}
}

func BenchmarkBuffer(b *testing.B) {
	for n := 0; n &lt; b.N; n++ {
		var buffer bytes.Buffer
		for i := 0; i &lt; concatSteps; i++ {
			buffer.WriteString(&quot;x&quot;)
		}
	}
}

Timings:

BenchmarkConcat-4                       	  300000	      6869 ns/op
BenchmarkBuffer-4                       	 1000000	      1186 ns/op

答案13

得分: 4

goutils.JoinBetween

 func JoinBetween(in []string, separator string, startIndex, endIndex int) string {
	if in == nil {
		return ""
	}

	noOfItems := endIndex - startIndex

	if noOfItems <= 0 {
		return EMPTY
	}

	var builder strings.Builder

	for i := startIndex; i < endIndex; i++ {
		if i > startIndex {
			builder.WriteString(separator)
		}
		builder.WriteString(in[i])
	}
	return builder.String()
}
英文:

goutils.JoinBetween

 func JoinBetween(in []string, separator string, startIndex, endIndex int) string {
	if in == nil {
		return &quot;&quot;
	}

	noOfItems := endIndex - startIndex

	if noOfItems &lt;= 0 {
		return EMPTY
	}

	var builder strings.Builder

	for i := startIndex; i &lt; endIndex; i++ {
		if i &gt; startIndex {
			builder.WriteString(separator)
		}
		builder.WriteString(in[i])
	}
	return builder.String()
}

答案14

得分: 1

我使用以下方式进行操作:

package main

import (
	"fmt"
	"strings"
)

func main (){
	concatenation:= strings.Join([]string{"a","b","c"},"") //第二个参数是分隔符。
	fmt.Println(concatenation) //abc
}
英文:

I do it using the following :-

package main

import (
	&quot;fmt&quot;
	&quot;strings&quot;
)

func main (){
	concatenation:= strings.Join([]string{&quot;a&quot;,&quot;b&quot;,&quot;c&quot;},&quot;&quot;) //where second parameter is a separator. 
	fmt.Println(concatenation) //abc
}

答案15

得分: 1

包主要

导入 (
"fmt"
)

功能 主要() {
var str1 = "string1"
var str2 = "string2"
result := make([]byte, 0)
result = append(result, []byte(str1)...)
result = append(result, []byte(str2)...)
result = append(result, []byte(str1)...)
result = append(result, []byte(str2)...)

fmt.Println(string(result))

}

英文:
package main

import (
&quot;fmt&quot;
)

func main() {
    var str1 = &quot;string1&quot;
    var str2 = &quot;string2&quot;
    result := make([]byte, 0)
    result = append(result, []byte(str1)...)
    result = append(result, []byte(str2)...)
    result = append(result, []byte(str1)...)
    result = append(result, []byte(str2)...)

    fmt.Println(string(result))
}

答案16

得分: 0

简单易懂的解决方案。详细信息请参见注释。
复制覆盖切片的元素。我们正在逐个元素切片并进行覆盖。

package main

import (
	"fmt"
)

var N int = 100000

func main() {
	slice1 := make([]rune, N, N)
	//高效且性能快,需要预先分配内存
    //我们可以添加一个检查,如果达到限制,则增加容量
    //使用append,但会对数据复制到新数组进行罚款。此外,append发生在当前切片的长度之后。
	for i := 0; i < N; i++ {
		copy(slice1[i:i+1], []rune{'N'})
	}
	fmt.Println(slice1)

	//简单但快速的解决方案,每次达到切片容量时,我们都会付出努力的罚款
	//将数据复制到新数组
	slice2 := []rune{}
	for i := 0; i <= N; i++ {
		slice2 = append(slice2, 'N')
	}
	fmt.Println(slice2)

}
英文:

Simple and easy to digest solution. Details in the comments.
Copy overwrites the elements of slice. We are slicing single-single element and overwriting it.

package main

import (
	&quot;fmt&quot;
)

var N int = 100000

func main() {
	slice1 := make([]rune, N, N)
	//Efficient with fast performance, Need pre-allocated memory
    //We can add a check if we reached the limit then increase capacity
    //using append, but would be fined for data copying to new array. Also append happens after the length of current slice.
	for i := 0; i &lt; N; i++ {
		copy(slice1[i:i+1], []rune{&#39;N&#39;})
	}
	fmt.Println(slice1)

	//Simple but fast solution, Every time the slice capacity is reached we get a fine of effort that goes
	//in copying data to new array
	slice2 := []rune{}
	for i := 0; i &lt;= N; i++ {
		slice2 = append(slice2, &#39;N&#39;)
	}
	fmt.Println(slice2)

}

答案17

得分: -1

使用内存分配统计进行基准测试结果。在github上查看基准测试代码。

使用strings.Builder来优化性能。

go test -bench . -benchmem
goos: darwin
goarch: amd64
pkg: github.com/hechen0/goexp/exps
BenchmarkConcat-8                1000000             60213 ns/op          503992 B/op          1 allocs/op
BenchmarkBuffer-8               100000000               11.3 ns/op             2 B/op          0 allocs/op
BenchmarkCopy-8                 300000000                4.76 ns/op            0 B/op          0 allocs/op
BenchmarkStringBuilder-8        1000000000               4.14 ns/op            6 B/op          0 allocs/op
PASS
ok      github.com/hechen0/goexp/exps   70.071s
英文:

benchmark result with memory allocation statistics. check benchmark code at github.

use strings.Builder to optimize performance.

go test -bench . -benchmem
goos: darwin
goarch: amd64
pkg: github.com/hechen0/goexp/exps
BenchmarkConcat-8                1000000             60213 ns/op          503992 B/op          1 allocs/op
BenchmarkBuffer-8               100000000               11.3 ns/op             2 B/op          0 allocs/op
BenchmarkCopy-8                 300000000                4.76 ns/op            0 B/op          0 allocs/op
BenchmarkStringBuilder-8        1000000000               4.14 ns/op            6 B/op          0 allocs/op
PASS
ok      github.com/hechen0/goexp/exps   70.071s

答案18

得分: -4

strings.Join()来自于"strings"包

如果你有类型不匹配的情况(比如你试图连接一个整数和一个字符串),你可以使用RANDOMTYPE(你想要改变的东西)

例子:

package main

import (
	"fmt"
	"strings"
)

var intEX = 0
var stringEX = "hello all you "
var stringEX2 = "people in here"


func main() {
	s := []string{stringEX, stringEX2}
	fmt.Println(strings.Join(s, ""))
}

输出:

hello all you people in here
英文:

strings.Join() from the "strings" package

If you have a type mismatch(like if you are trying to join an int and a string), you do RANDOMTYPE (thing you want to change)

EX:

package main

import (
	&quot;fmt&quot;
	&quot;strings&quot;
)

var intEX = 0
var stringEX = &quot;hello all you &quot;
var stringEX2 = &quot;people in here&quot;


func main() {
	s := []string{stringEX, stringEX2}
	fmt.Println(strings.Join(s, &quot;&quot;))
}

Output :

hello all you people in here

答案19

得分: -5

s := fmt.Sprintf("%s%s", []byte(s1), []byte(s2))

英文:
s := fmt.Sprintf(&quot;%s%s&quot;, []byte(s1), []byte(s2))

huangapple
  • 本文由 发表于 2009年11月19日 11:44:10
  • 转载请务必保留本文链接:https://go.coder-hub.com/1760757.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定