英文:
How to efficiently concatenate strings in go
问题
在Go语言中,string
是一种原始类型,这意味着它是只读的,对它的任何操作都会创建一个新的字符串。
所以,如果我想要多次连接字符串而不知道结果字符串的长度,最好的方法是什么?
一种简单的方法是:
var s string
for i := 0; i < 1000; i++ {
s += getShortStringFromSomewhere()
}
return s
但这种方法似乎不太高效。
英文:
In Go, a string
is a primitive type, which means it is read-only, and every manipulation of it will create a new string.
So if I want to concatenate strings many times without knowing the length of the resulting string, what's the best way to do it?
The naive way would be:
var s string
for i := 0; i < 1000; i++ {
s += getShortStringFromSomewhere()
}
return s
but that does not seem very efficient.
答案1
得分: 1043
新的方法:
从Go 1.10开始,有一个strings.Builder
类型,请查看这个答案以获取更多细节。
旧的方法:
使用bytes
包。它有一个Buffer
类型,实现了io.Writer
接口。
package main
import (
"bytes"
"fmt"
)
func main() {
var buffer bytes.Buffer
for i := 0; i < 1000; i++ {
buffer.WriteString("a")
}
fmt.Println(buffer.String())
}
这样做的时间复杂度为O(n)。
英文:
New Way:
From Go 1.10 there is a strings.Builder
type, please take a look at this answer for more detail.
Old Way:
Use the bytes
package. It has a Buffer
type which implements io.Writer
.
package main
import (
"bytes"
"fmt"
)
func main() {
var buffer bytes.Buffer
for i := 0; i < 1000; i++ {
buffer.WriteString("a")
}
fmt.Println(buffer.String())
}
This does it in O(n) time.
答案2
得分: 542
在Go 1.10+中,有**strings.Builder
**,这里。
> Builder用于使用Write方法高效地构建字符串。它最小化了内存复制。零值可以直接使用。
示例
它与bytes.Buffer
几乎相同。
package main
import (
"strings"
"fmt"
)
func main() {
// 零值:
//
// 它可以直接使用。
// 你不需要初始化它。
var sb strings.Builder
for i := 0; i < 1000; i++ {
sb.WriteString("a")
}
fmt.Println(sb.String())
}
支持的接口
strings.Builder
的方法是根据现有接口实现的,因此您可以在代码中轻松切换到新的Builder
类型。
方法签名 | 接口 | 描述 |
---|---|---|
Grow(int) |
bytes.Buffer |
增加缓冲区的容量。更多信息请参见bytes.Buffer#Grow。 |
Len() int |
bytes.Buffer |
返回缓冲区中的字节数。更多信息请参见bytes.Buffer#Len。 |
Reset() |
bytes.Buffer |
将缓冲区重置为空。更多信息请参见bytes.Buffer#Reset。 |
String() string |
fmt.Stringer |
将缓冲区的内容作为字符串返回。更多信息请参见fmt.Stringer。 |
Write([]byte) (int, error) |
io.Writer |
将给定的字节写入缓冲区。更多信息请参见io.Writer。 |
WriteByte(byte) error |
io.ByteWriter |
将给定的字节写入缓冲区。更多信息请参见io.ByteWriter。 |
WriteRune(rune) (int, error) |
bufio.Writer 或bytes.Buffer |
将给定的符文写入缓冲区。更多信息请参见bufio.Writer#WriteRune或bytes.Buffer#WriteRune。 |
WriteString(string) (int, error) |
io.stringWriter |
将给定的字符串写入缓冲区。更多信息请参见io.stringWriter。 |
与bytes.Buffer的区别
- 它只能增长或重置。
- 它内置了
copyCheck
机制,防止意外复制。在bytes.Buffer
中,可以像这样访问底层字节:(*Buffer).Bytes()
。strings.Builder
解决了这个问题。不过有时,这不是一个问题,反而是期望的。例如:当字节被传递给io.Reader
等时的查看行为。 bytes.Buffer.Reset()
重新定位并重用底层缓冲区,而strings.Builder.Reset()
不会,它会分离缓冲区。
注意
- 不要复制
strings.Builder
值,因为它会缓存底层数据。 - 如果要共享
strings.Builder
值,请使用指针。
英文:
In Go 1.10+ there is strings.Builder
, here.
> A Builder is used to efficiently build a string using Write methods. It minimizes memory copying. The zero value is ready to use.
Example
It's almost the same with bytes.Buffer
.
package main
import (
"strings"
"fmt"
)
func main() {
// ZERO-VALUE:
//
// It's ready to use from the get-go.
// You don't need to initialize it.
var sb strings.Builder
for i := 0; i < 1000; i++ {
sb.WriteString("a")
}
fmt.Println(sb.String())
}
Click to see this on the playground.
Supported Interfaces
strings.Builder
's methods are being implemented with the existing interfaces in mind so that you can switch to the new Builder
type easily in your code.
Method Signature | Interface | Description |
---|---|---|
Grow(int) |
bytes.Buffer |
Grows the buffer's capacity by the specified amount. See bytes.Buffer#Grow for more information. |
Len() int |
bytes.Buffer |
Returns the number of bytes in the buffer. See bytes.Buffer#Len for more information. |
Reset() |
bytes.Buffer |
Resets the buffer to be empty. See bytes.Buffer#Reset for more information. |
String() string |
fmt.Stringer |
Returns the contents of the buffer as a string. See fmt.Stringer for more information. |
Write([]byte) (int, error) |
io.Writer |
Writes the given bytes to the buffer. See io.Writer for more information. |
WriteByte(byte) error |
io.ByteWriter |
Writes the given byte to the buffer. See io.ByteWriter for more information. |
WriteRune(rune) (int, error) |
bufio.Writer or bytes.Buffer |
Writes the given rune to the buffer. See bufio.Writer#WriteRune or bytes.Buffer#WriteRune for more information. |
WriteString(string) (int, error) |
io.stringWriter |
Writes the given string to the buffer. See io.stringWriter for more information. |
Differences from bytes.Buffer
- It can only grow or reset.
- It has a
copyCheck
mechanism built-in that prevents accidentally copying it. Inbytes.Buffer
, one can access the underlying bytes like this:(*Buffer).Bytes()
.strings.Builder
prevents this problem. Sometimes, this is not a problem, though, and is desired instead. For example: For the peeking behavior when the bytes are passed to anio.Reader
etc. bytes.Buffer.Reset()
rewinds and reuses the underlying buffer whereas thestrings.Builder.Reset()
does not, it detaches the buffer.
Note
- Do not copy a
strings.Builder
value as it caches the underlying data. - If you want to share a
strings.Builder
value, use a pointer to it.
Check out its source code for more details, here.
答案3
得分: 293
如果您知道要预分配的字符串的总长度,那么最高效的字符串连接方式可能是使用内置函数copy
。如果您事先不知道总长度,请不要使用copy
,而是阅读其他答案。
在我的测试中,这种方法比使用bytes.Buffer
快约3倍,比使用运算符+
快得多(约12,000倍)。而且,它使用的内存更少。
我创建了一个测试案例来证明这一点,以下是结果:
<!-- language: lang-none -->
BenchmarkConcat 1000000 64497 ns/op 502018 B/op 0 allocs/op
BenchmarkBuffer 100000000 15.5 ns/op 2 B/op 0 allocs/op
BenchmarkCopy 500000000 5.39 ns/op 0 B/op 0 allocs/op
以下是用于测试的代码:
package main
import (
"bytes"
"strings"
"testing"
)
func BenchmarkConcat(b *testing.B) {
var str string
for n := 0; n < b.N; n++ {
str += "x"
}
b.StopTimer()
if s := strings.Repeat("x", b.N); str != s {
b.Errorf("unexpected result; got=%s, want=%s", str, s)
}
}
func BenchmarkBuffer(b *testing.B) {
var buffer bytes.Buffer
for n := 0; n < b.N; n++ {
buffer.WriteString("x")
}
b.StopTimer()
if s := strings.Repeat("x", b.N); buffer.String() != s {
b.Errorf("unexpected result; got=%s, want=%s", buffer.String(), s)
}
}
func BenchmarkCopy(b *testing.B) {
bs := make([]byte, b.N)
bl := 0
b.ResetTimer()
for n := 0; n < b.N; n++ {
bl += copy(bs[bl:], "x")
}
b.StopTimer()
if s := strings.Repeat("x", b.N); string(bs) != s {
b.Errorf("unexpected result; got=%s, want=%s", string(bs), s)
}
}
// Go 1.10
func BenchmarkStringBuilder(b *testing.B) {
var strBuilder strings.Builder
b.ResetTimer()
for n := 0; n < b.N; n++ {
strBuilder.WriteString("x")
}
b.StopTimer()
if s := strings.Repeat("x", b.N); strBuilder.String() != s {
b.Errorf("unexpected result; got=%s, want=%s", strBuilder.String(), s)
}
}
英文:
If you know the total length of the string that you're going to preallocate then the most efficient way to concatenate strings may be using the builtin function copy
. If you don't know the total length before hand, do not use copy
, and read the other answers instead.
In my tests, that approach is ~3x faster than using bytes.Buffer
and much much faster (~12,000x) than using the operator +
. Also, it uses less memory.
I've created a test case to prove this and here are the results:
<!-- language: lang-none -->
BenchmarkConcat 1000000 64497 ns/op 502018 B/op 0 allocs/op
BenchmarkBuffer 100000000 15.5 ns/op 2 B/op 0 allocs/op
BenchmarkCopy 500000000 5.39 ns/op 0 B/op 0 allocs/op
Below is code for testing:
package main
import (
"bytes"
"strings"
"testing"
)
func BenchmarkConcat(b *testing.B) {
var str string
for n := 0; n < b.N; n++ {
str += "x"
}
b.StopTimer()
if s := strings.Repeat("x", b.N); str != s {
b.Errorf("unexpected result; got=%s, want=%s", str, s)
}
}
func BenchmarkBuffer(b *testing.B) {
var buffer bytes.Buffer
for n := 0; n < b.N; n++ {
buffer.WriteString("x")
}
b.StopTimer()
if s := strings.Repeat("x", b.N); buffer.String() != s {
b.Errorf("unexpected result; got=%s, want=%s", buffer.String(), s)
}
}
func BenchmarkCopy(b *testing.B) {
bs := make([]byte, b.N)
bl := 0
b.ResetTimer()
for n := 0; n < b.N; n++ {
bl += copy(bs[bl:], "x")
}
b.StopTimer()
if s := strings.Repeat("x", b.N); string(bs) != s {
b.Errorf("unexpected result; got=%s, want=%s", string(bs), s)
}
}
// Go 1.10
func BenchmarkStringBuilder(b *testing.B) {
var strBuilder strings.Builder
b.ResetTimer()
for n := 0; n < b.N; n++ {
strBuilder.WriteString("x")
}
b.StopTimer()
if s := strings.Repeat("x", b.N); strBuilder.String() != s {
b.Errorf("unexpected result; got=%s, want=%s", strBuilder.String(), s)
}
}
答案4
得分: 151
如果您有一个字符串切片,想要高效地将其转换为字符串,您可以使用以下方法。否则,请查看其他答案。
在strings包中有一个名为Join
的库函数:
http://golang.org/pkg/strings/#Join
查看Join
的代码可以看到与Kinopiko编写的Append函数类似的方法:https://golang.org/src/strings/strings.go#L420
用法:
import (
"fmt";
"strings";
)
func main() {
s := []string{"this", "is", "a", "joined", "string\n"};
fmt.Printf(strings.Join(s, " "));
}
$ ./test.bin
this is a joined string
英文:
If you have a string slice that you want to efficiently convert to a string then you can use this approach. Otherwise, take a look at the other answers.
There is a library function in the strings package called Join
:
http://golang.org/pkg/strings/#Join
A look at the code of Join
shows a similar approach to Append function Kinopiko wrote: https://golang.org/src/strings/strings.go#L420
Usage:
import (
"fmt";
"strings";
)
func main() {
s := []string{"this", "is", "a", "joined", "string\n"};
fmt.Printf(strings.Join(s, " "));
}
$ ./test.bin
this is a joined string
答案5
得分: 47
我刚刚在我的代码中对上面发布的最佳答案进行了基准测试(递归树遍历),结果发现简单的连接运算符比BufferString
更快。
func (r *record) String() string {
buffer := bytes.NewBufferString("")
fmt.Fprint(buffer, "(", r.name, "[")
for i := 0; i < len(r.subs); i++ {
fmt.Fprint(buffer, "\t", r.subs[i])
}
fmt.Fprint(buffer, "]", r.size, ")\n")
return buffer.String()
}
这个方法花费了0.81秒,而下面的代码:
func (r *record) String() string {
s := "(" + r.name + " ["
for i := 0; i < len(r.subs); i++ {
s += r.subs[i].String()
}
s += "] " + strconv.FormatInt(r.size, 10) + ")\n"
return s
}
只花费了0.61秒。这可能是因为创建新的BufferString
的开销。
更新: 我还对join
函数进行了基准测试,它运行时间为0.54秒。
func (r *record) String() string {
var parts []string
parts = append(parts, "(", r.name, " [")
for i := 0; i < len(r.subs); i++ {
parts = append(parts, r.subs[i].String())
}
parts = append(parts, strconv.FormatInt(r.size, 10), ")\n")
return strings.Join(parts, "")
}
英文:
I just benchmarked the top answer posted above in my own code (a recursive tree walk) and the simple concat operator is actually faster than the BufferString
.
func (r *record) String() string {
buffer := bytes.NewBufferString("");
fmt.Fprint(buffer,"(",r.name,"[")
for i := 0; i < len(r.subs); i++ {
fmt.Fprint(buffer,"\t",r.subs[i])
}
fmt.Fprint(buffer,"]",r.size,")\n")
return buffer.String()
}
This took 0.81 seconds, whereas the following code:
func (r *record) String() string {
s := "(\"" + r.name + "\" ["
for i := 0; i < len(r.subs); i++ {
s += r.subs[i].String()
}
s += "] " + strconv.FormatInt(r.size,10) + ")\n"
return s
}
only took 0.61 seconds. This is probably due to the overhead of creating the new BufferString
.
Update: I also benchmarked the join
function and it ran in 0.54 seconds.
func (r *record) String() string {
var parts []string
parts = append(parts, "(\"", r.name, "\" [" )
for i := 0; i < len(r.subs); i++ {
parts = append(parts, r.subs[i].String())
}
parts = append(parts, strconv.FormatInt(r.size,10), ")\n")
return strings.Join(parts,"")
}
答案6
得分: 31
包主要
导入 (
"fmt"
)
功能 主要() {
var str1 = "字符串1"
var str2 = "字符串2"
out := fmt.Sprintf("%s %s ",str1, str2)
fmt.Println(out)
}
英文:
package main
import (
"fmt"
)
func main() {
var str1 = "string1"
var str2 = "string2"
out := fmt.Sprintf("%s %s ",str1, str2)
fmt.Println(out)
}
答案7
得分: 26
这是最快的解决方案,不需要你知道或计算整个缓冲区的大小:
var data []byte
for i := 0; i < 1000; i++ {
data = append(data, getShortStringFromSomewhere()...)
}
return string(data)
根据我的基准测试,它比复制解决方案慢20%(每次追加需要8.1纳秒,而不是6.72纳秒),但仍然比使用bytes.Buffer快55%。
英文:
This is the fastest solution that does not require
you to know or calculate the overall buffer size first:
var data []byte
for i := 0; i < 1000; i++ {
data = append(data, getShortStringFromSomewhere()...)
}
return string(data)
By my benchmark, it's 20% slower than the copy solution (8.1ns per
append rather than 6.72ns) but still 55% faster than using bytes.Buffer.
答案8
得分: 25
你可以创建一个大的字节切片,并使用字符串切片将短字符串的字节复制到其中。在《Effective Go》中提供了一个函数:
func Append(slice, data[]byte) []byte {
l := len(slice);
if l + len(data) > cap(slice) { // 重新分配内存
// 分配比所需空间多一倍的内存,以供未来扩展。
newSlice := make([]byte, (l+len(data))*2);
// 复制数据(可以使用 bytes.Copy())。
for i, c := range slice {
newSlice[i] = c
}
slice = newSlice;
}
slice = slice[0:l+len(data)];
for i, c := range data {
slice[l+i] = c
}
return slice;
}
然后在操作完成后,使用 string()
将大的字节切片转换回字符串。
英文:
You could create a big slice of bytes and copy the bytes of the short strings into it using string slices. There is a function given in "Effective Go":
func Append(slice, data[]byte) []byte {
l := len(slice);
if l + len(data) > cap(slice) { // reallocate
// Allocate double what's needed, for future growth.
newSlice := make([]byte, (l+len(data))*2);
// Copy data (could use bytes.Copy()).
for i, c := range slice {
newSlice[i] = c
}
slice = newSlice;
}
slice = slice[0:l+len(data)];
for i, c := range data {
slice[l+i] = c
}
return slice;
}
Then when the operations are finished, use string ( )
on the big slice of bytes to convert it into a string again.
答案9
得分: 24
2018年添加的注释
从Go 1.10开始,有一个strings.Builder
类型,请参考此答案了解更多细节。
201x年之前的答案
@cd1和其他答案的基准代码是错误的。基准函数中不应该设置b.N
,它是由go test工具动态设置的,用于确定测试的执行时间是否稳定。
基准函数应该运行相同的测试b.N
次,循环内部的测试对于每次迭代都应该是相同的。所以我通过添加一个内部循环来修复它。我还为其他一些解决方案添加了基准测试:
package main
import (
"bytes"
"strings"
"testing"
)
const (
sss = "xfoasneobfasieongasbg"
cnt = 10000
)
var (
bbb = []byte(sss)
expected = strings.Repeat(sss, cnt)
)
func BenchmarkCopyPreAllocate(b *testing.B) {
var result string
for n := 0; n < b.N; n++ {
bs := make([]byte, cnt*len(sss))
bl := 0
for i := 0; i < cnt; i++ {
bl += copy(bs[bl:], sss)
}
result = string(bs)
}
b.StopTimer()
if result != expected {
b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
}
}
func BenchmarkAppendPreAllocate(b *testing.B) {
var result string
for n := 0; n < b.N; n++ {
data := make([]byte, 0, cnt*len(sss))
for i := 0; i < cnt; i++ {
data = append(data, sss...)
}
result = string(data)
}
b.StopTimer()
if result != expected {
b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
}
}
func BenchmarkBufferPreAllocate(b *testing.B) {
var result string
for n := 0; n < b.N; n++ {
buf := bytes.NewBuffer(make([]byte, 0, cnt*len(sss)))
for i := 0; i < cnt; i++ {
buf.WriteString(sss)
}
result = buf.String()
}
b.StopTimer()
if result != expected {
b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
}
}
func BenchmarkCopy(b *testing.B) {
var result string
for n := 0; n < b.N; n++ {
data := make([]byte, 0, 64) // 与bytes.Buffer的引导数组大小相同
for i := 0; i < cnt; i++ {
off := len(data)
if off+len(sss) > cap(data) {
temp := make([]byte, 2*cap(data)+len(sss))
copy(temp, data)
data = temp
}
data = data[0 : off+len(sss)]
copy(data[off:], sss)
}
result = string(data)
}
b.StopTimer()
if result != expected {
b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
}
}
func BenchmarkAppend(b *testing.B) {
var result string
for n := 0; n < b.N; n++ {
data := make([]byte, 0, 64)
for i := 0; i < cnt; i++ {
data = append(data, sss...)
}
result = string(data)
}
b.StopTimer()
if result != expected {
b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
}
}
func BenchmarkBufferWrite(b *testing.B) {
var result string
for n := 0; n < b.N; n++ {
var buf bytes.Buffer
for i := 0; i < cnt; i++ {
buf.Write(bbb)
}
result = buf.String()
}
b.StopTimer()
if result != expected {
b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
}
}
func BenchmarkBufferWriteString(b *testing.B) {
var result string
for n := 0; n < b.N; n++ {
var buf bytes.Buffer
for i := 0; i < cnt; i++ {
buf.WriteString(sss)
}
result = buf.String()
}
b.StopTimer()
if result != expected {
b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
}
}
func BenchmarkConcat(b *testing.B) {
var result string
for n := 0; n < b.N; n++ {
var str string
for i := 0; i < cnt; i++ {
str += sss
}
result = str
}
b.StopTimer()
if result != expected {
b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
}
}
环境为OS X 10.11.6,2.2 GHz Intel Core i7
测试结果:
BenchmarkCopyPreAllocate-8 20000 84208 ns/op 425984 B/op 2 allocs/op
BenchmarkAppendPreAllocate-8 10000 102859 ns/op 425984 B/op 2 allocs/op
BenchmarkBufferPreAllocate-8 10000 166407 ns/op 426096 B/op 3 allocs/op
BenchmarkCopy-8 10000 160923 ns/op 933152 B/op 13 allocs/op
BenchmarkAppend-8 10000 175508 ns/op 1332096 B/op 24 allocs/op
BenchmarkBufferWrite-8 10000 239886 ns/op 933266 B/op 14 allocs/op
BenchmarkBufferWriteString-8 10000 236432 ns/op 933266 B/op 14 allocs/op
BenchmarkConcat-8 10 105603419 ns/op 1086685168 B/op 10000 allocs/op
结论:
CopyPreAllocate
是最快的方法;AppendPreAllocate
非常接近第一名,但编写代码更容易。Concat
的性能非常差,无论是速度还是内存使用。不要使用它。Buffer#Write
和Buffer#WriteString
在速度上基本相同,与@Dani-Br在评论中所说的相反。考虑到在Go中string
实际上是[]byte
,这是有道理的。- bytes.Buffer基本上使用与
Copy
相同的解决方案,还有额外的书籍管理和其他内容。 Copy
和Append
使用与bytes.Buffer相同的引导大小为64。Append
使用更多的内存和分配,我认为这与它使用的增长算法有关。它的内存增长速度不如bytes.Buffer快。
建议:
- 对于像OP所需的简单任务,我会使用
Append
或AppendPreAllocate
。它足够快且易于使用。 - 如果需要同时读取和写入缓冲区,请使用
bytes.Buffer
。这就是它的设计目的。
英文:
Note added in 2018
From Go 1.10 there is a strings.Builder
type, please take a look at this answer for more detail.
Pre-201x answer
The benchmark code of @cd1 and other answers are wrong. b.N
is not supposed to be set in benchmark function. It's set by the go test tool dynamically to determine if the execution time of the test is stable.
A benchmark function should run the same test b.N
times and the test inside the loop should be the same for each iteration. So I fix it by adding an inner loop. I also add benchmarks for some other solutions:
package main
import (
"bytes"
"strings"
"testing"
)
const (
sss = "xfoasneobfasieongasbg"
cnt = 10000
)
var (
bbb = []byte(sss)
expected = strings.Repeat(sss, cnt)
)
func BenchmarkCopyPreAllocate(b *testing.B) {
var result string
for n := 0; n < b.N; n++ {
bs := make([]byte, cnt*len(sss))
bl := 0
for i := 0; i < cnt; i++ {
bl += copy(bs[bl:], sss)
}
result = string(bs)
}
b.StopTimer()
if result != expected {
b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
}
}
func BenchmarkAppendPreAllocate(b *testing.B) {
var result string
for n := 0; n < b.N; n++ {
data := make([]byte, 0, cnt*len(sss))
for i := 0; i < cnt; i++ {
data = append(data, sss...)
}
result = string(data)
}
b.StopTimer()
if result != expected {
b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
}
}
func BenchmarkBufferPreAllocate(b *testing.B) {
var result string
for n := 0; n < b.N; n++ {
buf := bytes.NewBuffer(make([]byte, 0, cnt*len(sss)))
for i := 0; i < cnt; i++ {
buf.WriteString(sss)
}
result = buf.String()
}
b.StopTimer()
if result != expected {
b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
}
}
func BenchmarkCopy(b *testing.B) {
var result string
for n := 0; n < b.N; n++ {
data := make([]byte, 0, 64) // same size as bootstrap array of bytes.Buffer
for i := 0; i < cnt; i++ {
off := len(data)
if off+len(sss) > cap(data) {
temp := make([]byte, 2*cap(data)+len(sss))
copy(temp, data)
data = temp
}
data = data[0 : off+len(sss)]
copy(data[off:], sss)
}
result = string(data)
}
b.StopTimer()
if result != expected {
b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
}
}
func BenchmarkAppend(b *testing.B) {
var result string
for n := 0; n < b.N; n++ {
data := make([]byte, 0, 64)
for i := 0; i < cnt; i++ {
data = append(data, sss...)
}
result = string(data)
}
b.StopTimer()
if result != expected {
b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
}
}
func BenchmarkBufferWrite(b *testing.B) {
var result string
for n := 0; n < b.N; n++ {
var buf bytes.Buffer
for i := 0; i < cnt; i++ {
buf.Write(bbb)
}
result = buf.String()
}
b.StopTimer()
if result != expected {
b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
}
}
func BenchmarkBufferWriteString(b *testing.B) {
var result string
for n := 0; n < b.N; n++ {
var buf bytes.Buffer
for i := 0; i < cnt; i++ {
buf.WriteString(sss)
}
result = buf.String()
}
b.StopTimer()
if result != expected {
b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
}
}
func BenchmarkConcat(b *testing.B) {
var result string
for n := 0; n < b.N; n++ {
var str string
for i := 0; i < cnt; i++ {
str += sss
}
result = str
}
b.StopTimer()
if result != expected {
b.Errorf("unexpected result; got=%s, want=%s", string(result), expected)
}
}
Environment is OS X 10.11.6, 2.2 GHz Intel Core i7
Test results:
BenchmarkCopyPreAllocate-8 20000 84208 ns/op 425984 B/op 2 allocs/op
BenchmarkAppendPreAllocate-8 10000 102859 ns/op 425984 B/op 2 allocs/op
BenchmarkBufferPreAllocate-8 10000 166407 ns/op 426096 B/op 3 allocs/op
BenchmarkCopy-8 10000 160923 ns/op 933152 B/op 13 allocs/op
BenchmarkAppend-8 10000 175508 ns/op 1332096 B/op 24 allocs/op
BenchmarkBufferWrite-8 10000 239886 ns/op 933266 B/op 14 allocs/op
BenchmarkBufferWriteString-8 10000 236432 ns/op 933266 B/op 14 allocs/op
BenchmarkConcat-8 10 105603419 ns/op 1086685168 B/op 10000 allocs/op
Conclusion:
CopyPreAllocate
is the fastest way;AppendPreAllocate
is pretty close to No.1, but it's easier to write the code.Concat
has really bad performance both for speed and memory usage. Don't use it.Buffer#Write
andBuffer#WriteString
are basically the same in speed, contrary to what @Dani-Br said in the comment. Consideringstring
is indeed[]byte
in Go, it makes sense.- bytes.Buffer basically use the same solution as
Copy
with extra book keeping and other stuff. Copy
andAppend
use a bootstrap size of 64, the same as bytes.BufferAppend
use more memory and allocs, I think it's related to the grow algorithm it use. It's not growing memory as fast as bytes.Buffer
Suggestion:
- For simple task such as what OP wants, I would use
Append
orAppendPreAllocate
. It's fast enough and easy to use. - If need to read and write the buffer at the same time, use
bytes.Buffer
of course. That's what it's designed for.
答案10
得分: 14
我的原始建议是
s12 := fmt.Sprint(s1,s2)
但是上面的答案使用了<a href="http://golang.org/pkg/bytes/#Buffer.WriteString" >bytes.Buffer - WriteString()</a> 是最高效的方法。
我的初始建议使用了反射和类型切换。<a href="https://golang.org/src/pkg/fmt/print.go">查看 (p *pp) doPrint
和 (p *pp) printArg
</a><br>
对于基本类型,没有通用的Stringer()接口,这是我天真地认为的。
至少,Sprint() 内部 使用了一个bytes.Buffer。因此
`s12 := fmt.Sprint(s1,s2,s3,s4,...,s1000)`
在内存分配方面是可接受的。
=> Sprint() 连接可以用于快速调试输出。<br>
=> 否则使用 bytes.Buffer ... WriteString
英文:
My original suggestion was
s12 := fmt.Sprint(s1,s2)
But above answer using <a href="http://golang.org/pkg/bytes/#Buffer.WriteString" >bytes.Buffer - WriteString()</a> is the most efficient way.
My initial suggestion uses reflection and a type switch. <a href="https://golang.org/src/pkg/fmt/print.go">See (p *pp) doPrint
and (p *pp) printArg
</a><br>
There is no universal Stringer() interface for basic types, as I had naively thought.
At least though, Sprint() internally uses a bytes.Buffer. Thus
`s12 := fmt.Sprint(s1,s2,s3,s4,...,s1000)`
is acceptable in terms of memory allocations.
=> Sprint() concatenation can be used for quick debug output.<br>
=> Otherwise use bytes.Buffer ... WriteString
答案11
得分: 11
在cd1的答案上进行扩展:
你可以使用append()代替copy()。
append()提供了更大的预留空间,会消耗一些内存,但可以节省时间。
我在你的代码顶部添加了另外两个基准测试。
在本地运行:
go test -bench=. -benchtime=100ms
在我的ThinkPad T400s上运行结果如下:
BenchmarkAppendEmpty 50000000 5.0 ns/op
BenchmarkAppendPrealloc 50000000 3.5 ns/op
BenchmarkCopy 20000000 10.2 ns/op
英文:
Expanding on cd1's answer:
You might use append() instead of copy().
append() makes ever bigger advance provisions, costing a little more memory, but saving time.
I added two more benchmarks at the top of yours.
Run locally with
go test -bench=. -benchtime=100ms
On my thinkpad T400s it yields:
BenchmarkAppendEmpty 50000000 5.0 ns/op
BenchmarkAppendPrealloc 50000000 3.5 ns/op
BenchmarkCopy 20000000 10.2 ns/op
答案12
得分: 4
这是由@cd1提供的实际版本的基准测试(Go 1.8
, linux x86_64
),修复了@icza和@PickBoy提到的错误。
Bytes.Buffer
只比直接使用+
运算符进行字符串连接快7
倍。
package performance_test
import (
"bytes"
"fmt"
"testing"
)
const (
concatSteps = 100
)
func BenchmarkConcat(b *testing.B) {
for n := 0; n < b.N; n++ {
var str string
for i := 0; i < concatSteps; i++ {
str += "x"
}
}
}
func BenchmarkBuffer(b *testing.B) {
for n := 0; n < b.N; n++ {
var buffer bytes.Buffer
for i := 0; i < concatSteps; i++ {
buffer.WriteString("x")
}
}
}
时间:
BenchmarkConcat-4 300000 6869 ns/op
BenchmarkBuffer-4 1000000 1186 ns/op
英文:
This is actual version of benchmark provided by @cd1 (Go 1.8
, linux x86_64
) with the fixes of bugs mentioned by @icza and @PickBoy.
Bytes.Buffer
is only 7
times faster than direct string concatenation via +
operator.
package performance_test
import (
"bytes"
"fmt"
"testing"
)
const (
concatSteps = 100
)
func BenchmarkConcat(b *testing.B) {
for n := 0; n < b.N; n++ {
var str string
for i := 0; i < concatSteps; i++ {
str += "x"
}
}
}
func BenchmarkBuffer(b *testing.B) {
for n := 0; n < b.N; n++ {
var buffer bytes.Buffer
for i := 0; i < concatSteps; i++ {
buffer.WriteString("x")
}
}
}
Timings:
BenchmarkConcat-4 300000 6869 ns/op
BenchmarkBuffer-4 1000000 1186 ns/op
答案13
得分: 4
func JoinBetween(in []string, separator string, startIndex, endIndex int) string {
if in == nil {
return ""
}
noOfItems := endIndex - startIndex
if noOfItems <= 0 {
return EMPTY
}
var builder strings.Builder
for i := startIndex; i < endIndex; i++ {
if i > startIndex {
builder.WriteString(separator)
}
builder.WriteString(in[i])
}
return builder.String()
}
英文:
func JoinBetween(in []string, separator string, startIndex, endIndex int) string {
if in == nil {
return ""
}
noOfItems := endIndex - startIndex
if noOfItems <= 0 {
return EMPTY
}
var builder strings.Builder
for i := startIndex; i < endIndex; i++ {
if i > startIndex {
builder.WriteString(separator)
}
builder.WriteString(in[i])
}
return builder.String()
}
答案14
得分: 1
我使用以下方式进行操作:
package main
import (
"fmt"
"strings"
)
func main (){
concatenation:= strings.Join([]string{"a","b","c"},"") //第二个参数是分隔符。
fmt.Println(concatenation) //abc
}
英文:
I do it using the following :-
package main
import (
"fmt"
"strings"
)
func main (){
concatenation:= strings.Join([]string{"a","b","c"},"") //where second parameter is a separator.
fmt.Println(concatenation) //abc
}
答案15
得分: 1
包主要
导入 (
"fmt"
)
功能 主要() {
var str1 = "string1"
var str2 = "string2"
result := make([]byte, 0)
result = append(result, []byte(str1)...)
result = append(result, []byte(str2)...)
result = append(result, []byte(str1)...)
result = append(result, []byte(str2)...)
fmt.Println(string(result))
}
英文:
package main
import (
"fmt"
)
func main() {
var str1 = "string1"
var str2 = "string2"
result := make([]byte, 0)
result = append(result, []byte(str1)...)
result = append(result, []byte(str2)...)
result = append(result, []byte(str1)...)
result = append(result, []byte(str2)...)
fmt.Println(string(result))
}
答案16
得分: 0
简单易懂的解决方案。详细信息请参见注释。
复制覆盖切片的元素。我们正在逐个元素切片并进行覆盖。
package main
import (
"fmt"
)
var N int = 100000
func main() {
slice1 := make([]rune, N, N)
//高效且性能快,需要预先分配内存
//我们可以添加一个检查,如果达到限制,则增加容量
//使用append,但会对数据复制到新数组进行罚款。此外,append发生在当前切片的长度之后。
for i := 0; i < N; i++ {
copy(slice1[i:i+1], []rune{'N'})
}
fmt.Println(slice1)
//简单但快速的解决方案,每次达到切片容量时,我们都会付出努力的罚款
//将数据复制到新数组
slice2 := []rune{}
for i := 0; i <= N; i++ {
slice2 = append(slice2, 'N')
}
fmt.Println(slice2)
}
英文:
Simple and easy to digest solution. Details in the comments.
Copy overwrites the elements of slice. We are slicing single-single element and overwriting it.
package main
import (
"fmt"
)
var N int = 100000
func main() {
slice1 := make([]rune, N, N)
//Efficient with fast performance, Need pre-allocated memory
//We can add a check if we reached the limit then increase capacity
//using append, but would be fined for data copying to new array. Also append happens after the length of current slice.
for i := 0; i < N; i++ {
copy(slice1[i:i+1], []rune{'N'})
}
fmt.Println(slice1)
//Simple but fast solution, Every time the slice capacity is reached we get a fine of effort that goes
//in copying data to new array
slice2 := []rune{}
for i := 0; i <= N; i++ {
slice2 = append(slice2, 'N')
}
fmt.Println(slice2)
}
答案17
得分: -1
使用内存分配统计进行基准测试结果。在github上查看基准测试代码。
使用strings.Builder来优化性能。
go test -bench . -benchmem
goos: darwin
goarch: amd64
pkg: github.com/hechen0/goexp/exps
BenchmarkConcat-8 1000000 60213 ns/op 503992 B/op 1 allocs/op
BenchmarkBuffer-8 100000000 11.3 ns/op 2 B/op 0 allocs/op
BenchmarkCopy-8 300000000 4.76 ns/op 0 B/op 0 allocs/op
BenchmarkStringBuilder-8 1000000000 4.14 ns/op 6 B/op 0 allocs/op
PASS
ok github.com/hechen0/goexp/exps 70.071s
英文:
benchmark result with memory allocation statistics. check benchmark code at github.
use strings.Builder to optimize performance.
go test -bench . -benchmem
goos: darwin
goarch: amd64
pkg: github.com/hechen0/goexp/exps
BenchmarkConcat-8 1000000 60213 ns/op 503992 B/op 1 allocs/op
BenchmarkBuffer-8 100000000 11.3 ns/op 2 B/op 0 allocs/op
BenchmarkCopy-8 300000000 4.76 ns/op 0 B/op 0 allocs/op
BenchmarkStringBuilder-8 1000000000 4.14 ns/op 6 B/op 0 allocs/op
PASS
ok github.com/hechen0/goexp/exps 70.071s
答案18
得分: -4
strings.Join()
来自于"strings"包
如果你有类型不匹配的情况(比如你试图连接一个整数和一个字符串),你可以使用RANDOMTYPE(你想要改变的东西)
例子:
package main
import (
"fmt"
"strings"
)
var intEX = 0
var stringEX = "hello all you "
var stringEX2 = "people in here"
func main() {
s := []string{stringEX, stringEX2}
fmt.Println(strings.Join(s, ""))
}
输出:
hello all you people in here
英文:
strings.Join()
from the "strings" package
If you have a type mismatch(like if you are trying to join an int and a string), you do RANDOMTYPE (thing you want to change)
EX:
package main
import (
"fmt"
"strings"
)
var intEX = 0
var stringEX = "hello all you "
var stringEX2 = "people in here"
func main() {
s := []string{stringEX, stringEX2}
fmt.Println(strings.Join(s, ""))
}
Output :
hello all you people in here
答案19
得分: -5
s := fmt.Sprintf("%s%s", []byte(s1), []byte(s2))
英文:
s := fmt.Sprintf("%s%s", []byte(s1), []byte(s2))
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论