为什么我的Go程序在以下场景中表现得比预期的要差很多?

huangapple go评论102阅读模式
英文:

Why is my Go program performing much worse than expected in the following scenario?

问题

我在解决一个简单的问题,使用C、Go和Python进行比较结果。解决方案只需要在一个if-else块中有两个方程。以下是我在这三种语言中解决方案的代码:

C

python

go

我比较了这三种语言处理浮点数结果的方式,因此编写了这个脚本来生成测试用例,并使用这个脚本进行结果比较,每次比较两个。

奇怪的是前三个脚本运行所需的时间。C程序显然在几秒钟内运行。Python大约需要2.5-3秒。然而,Go在由generate_test_cases.py脚本生成的测试用例上运行程序需要大约24-25秒。

我原以为Go在运行时间方面应该介于C和Python之间。我在Go代码中是否做了一些低效的操作,如果是,是什么?

附注:我还在没有文件处理操作的情况下运行了上述三个程序,结果仍然相同,即Go比其他两种语言花费的时间要长得多。

英文:

I was solving a simple question in C, Go, and Python and comparing the results. The solution is simply supposed to have two equations inside an if-else block. The following are the code for my solution in:

C

python

go

I was comparing the way the three languages deal with floating point results and hence made this script to generate the test cases and this to compare the results, two at a time.

The weird thing is the time it takes to run the first 3 scripts. The C program obviously runs within a couple of seconds. Python takes around 2.5-3 seconds. However, Go is taking around 24-25 seconds to run the program on the test cases generated by the generate_test_cases.py script.

I thought Go would be somewhere between C and Python as far as running time is concerned. Am I doing something inefficiently with my Go code, and if so, what?

P.S. I also ran the above three programs without the file handling operations, and still the same result, i.e., Go takes an extraordinarily long time compared to the other two.

答案1

得分: 4

像Max一样,我强烈怀疑Go语言的慢速是与I/O性能差有关的。我测试了这个假设:

package main

import "fmt"
import "os"
import "time"

func main(){
    now := time.Now()
    input,_ := os.Open("testing/test_cases.txt")
    defer input.Close()
    output,_ := os.Create("testing/Goutput.txt")
    defer output.Close()

    var ncases int
    var p float64
    fmt.Fscanf(input,"%d",&ncases)

    fmt.Println("Opened files in ", time.Since(now), "seconds")
    now = time.Now()

    cases := make([]float64, ncases)
    fmt.Println("Made array in ", time.Since(now), "seconds")
    now = time.Now()

    for i := 0; i < ncases; i++ {
        fmt.Fscanf(input,"%f",&cases[i])
    }

    fmt.Println("Read data in ", time.Since(now), "seconds")
    now = time.Now()

    for i := 0; i < ncases; i++ {
        p = cases[i]
        if p >= 0.5 {
            cases[i] = 10000 * (1-p) * (2*p-1) + 10000
        } else {
            cases[i] = p*(1-2*p)*10000 + 10000
        }
    }

    fmt.Println("Processed data in ", time.Since(now), "seconds")
    now = time.Now()

    for i := 0; i < ncases; i++ {
        fmt.Fprintln(output, cases[i])
    }

    fmt.Println("Output processed data in ", time.Since(now), "seconds")
}

运行后产生以下输出:

Opened files in  2.011228ms seconds
Made array in  109.904us seconds
Read data in  4.524544608s seconds
Processed data in  10.083329ms seconds
Output processed data in  1.703542918s seconds

所以在我的机器上,所有的数学计算大约需要10毫秒,但是I/O很慢,证实了这个假设。正如评论中Janne指出的,可能有比fmt更快的选项。

更新: 例如,使用bufio的读写器包装inputoutput

binput := bufio.NewReader(input)
boutput := bufio.NewWriter(output)

并且使用binputboutput进行缓冲I/O,你的原始版本在我的机器上运行时间为2.1秒,比Python 2.7要快一些。

更新2: 我注意到只是切换到缓冲I/O就得到了不同的结果。

  1. 原来你还需要调整格式字符串以包含\n,就像在C版本中一样。我认为这样做实际上更正确,但看起来在非缓冲的情况下可以忽略。

  2. 还需要Flush()你的缓冲输出,我之前没有提到。

这是我的完整缓冲解决方案:

package main

import "fmt"
import "os"
import "bufio"
import "time"

func main(){
    now := time.Now()

    nbinput, _ := os.Open("testing/test_cases.txt")
    defer nbinput.Close()

    nboutput, _ := os.Create("testing/Goutput.txt")
    defer nboutput.Close()

    binput := bufio.NewReader(nbinput)
    boutput := bufio.NewWriter(nboutput)

    var ncases int
    var gain, p float64
    fmt.Fscanf(binput,"%d\n",&ncases)

    for i := 0; i < ncases; i++ {
        fmt.Fscanf(binput, "%f\n", &p)
        if p >= 0.5 {
            gain = 10000 * (1-p) * (2*p -1)
        } else {
            gain = p*(1-2*p)*10000
        }
        fmt.Fprintln(boutput, gain+10000)
    }
    boutput.Flush()
    fmt.Println("Took ", time.Since(now), "seconds")
}
英文:

Like Max, my strong suspicion was that the slowness in Go was related to poor I/O performance. I tested this hypothesis:

package main
import &quot;fmt&quot;
import &quot;os&quot;
import &quot;time&quot;
func main(){
now := time.Now()
input,_ := os.Open(&quot;testing/test_cases.txt&quot;)
defer input.Close()
output,_ := os.Create(&quot;testing/Goutput.txt&quot;)
defer output.Close()
var ncases int
var p float64
fmt.Fscanf(input,&quot;%d&quot;,&amp;ncases)
fmt.Println(&quot;Opened files in &quot;, time.Since(now), &quot;seconds&quot;)
now = time.Now()
cases := make([]float64, ncases)
fmt.Println(&quot;Made array in &quot;, time.Since(now), &quot;seconds&quot;)
now = time.Now()
for i := 0; i &lt; ncases; i++ {
fmt.Fscanf(input,&quot;%f&quot;,&amp;cases[i])
}
fmt.Println(&quot;Read data in &quot;, time.Since(now), &quot;seconds&quot;)
now = time.Now()
for i := 0; i &lt; ncases; i++ {
p = cases[i]
if p &gt;= 0.5 {
cases[i] = 10000 * (1-p) * (2*p-1) + 10000
} else {
cases[i] = p*(1-2*p)*10000 + 10000
}
}
fmt.Println(&quot;Processed data in &quot;, time.Since(now), &quot;seconds&quot;)
now = time.Now()
for i := 0; i &lt; ncases; i++ {
fmt.Fprintln(output, cases[i])
}
fmt.Println(&quot;Output processed data in &quot;, time.Since(now), &quot;seconds&quot;)
}

Running it produced this output:

<pre>
Opened files in 2.011228ms seconds
Made array in 109.904us seconds
Read data in 4.524544608s seconds
Processed data in 10.083329ms seconds
Output processed data in 1.703542918s seconds
</pre>

So it appears that on my machine, all of the math occurs in about 10ms, but the I/O is slow, confirming the hypothesis. As pointed out by Janne in the comments, there are likely faster options than fmt.

Update: For example, wrapping input and output with bufio's Readers and Writers:

binput := bufio.NewReader(input)
boutput := bufio.NewWriter(output)

and using binput and boutput for buffered I/O, your original version runs in 2.1 seconds on my machine, somewhat faster than Python's 2.7.

Update 2: I noticed that I was getting different results by just switching to buffered I/O.

  1. It turns out that you also need to adjust your format strings to include the \n, as you do in the C version. I think this is actually more correct either way, but it looks like you can get away with it while unbuffered.

  2. It's also important to Flush() your buffered output, which I did but didn't mention before.

Here's my complete buffered solution:

package main
import &quot;fmt&quot;
import &quot;os&quot;
import &quot;bufio&quot;
import &quot;time&quot;
func main(){
now := time.Now()
nbinput, _ := os.Open(&quot;testing/test_cases.txt&quot;)
defer nbinput.Close()
nboutput, _ := os.Create(&quot;testing/Goutput.txt&quot;)
defer nboutput.Close()
binput := bufio.NewReader(nbinput)
boutput := bufio.NewWriter(nboutput)
var ncases int
var gain, p float64
fmt.Fscanf(binput,&quot;%d\n&quot;,&amp;ncases)
for i := 0; i &lt; ncases; i++ {
fmt.Fscanf(binput, &quot;%f\n&quot;, &amp;p)
if p &gt;= 0.5 {
gain = 10000 * (1-p) * (2*p -1)
} else {
gain = p*(1-2*p)*10000
}
fmt.Fprintln(boutput, gain+10000)
}
boutput.Flush()
fmt.Println(&quot;Took &quot;, time.Since(now), &quot;seconds&quot;)
}

答案2

得分: 1

我猜在Go中以下代码行的执行速度较慢。

    fmt.Fscanf(input,&quot;%f&quot;,&amp;p)
fmt.Fprintln(output,gain+10000)

Go的魔力在于IO操作。它们看起来是同步的,但实际上是异步的。
Go协程进行异步请求并将控制权返回给调度器。调度器寻找另一个等待控制权的协程,但只有一个等待IO操作。
因此,调度器循环执行无操作。

如果您有2个、10个或100个并发的协程,那么性能会更好。

英文:

I guess following lines work slower in go.

    fmt.Fscanf(input,&quot;%f&quot;,&amp;p)
fmt.Fprintln(output,gain+10000)

Go's magic happens when you do IO. They look like synchronous but in fact they are async.
Go rotine does async request and returns control to scheduler. Scheduler looks for another goroutine waiting for taking control but there is only one waiting io.
So scheduler loops doing nothing.

If you had 2, 10 or 100 concurrent goroutines then will see a better performance.

huangapple
  • 本文由 发表于 2013年6月20日 16:26:52
  • 转载请务必保留本文链接:https://go.coder-hub.com/17208763.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定