如何在Erlang或Elixir中生成n-k个轻量级进程?

huangapple go评论89阅读模式
英文:

How to spawn n-k light-process in erlang or elixir?

问题

在Go语言中,我可以像这样创建goroutine:

// test.go
package main

import (
    "fmt"
    "os"
    "strconv"
    "sync"
    "runtime"
)

func main() {
    var wg sync.WaitGroup
    if len(os.Args) < 2 {
        os.Exit(1)
    }
    k, ok := strconv.Atoi(os.Args[1])
    if ok != nil {
        os.Exit(2)
    }
    wg.Add(k * 1000)
    for z := 0; z < k*1000; z++ {
        go func(x int) {
            defer wg.Done()
            fmt.Println(x)
        }(z)
        if z%k == k-1 {
            // @mattn: 避免忙等待,让Go可以像BEAM一样开始处理
            runtime.Gosched()
        }
    }
    wg.Wait()
}

在Go 1.8.0 (64位)中的运行结果如下:

$ go build test.go ; for k in 5 50 500 5000 50000 500000; do echo -n $k; time ./test $k > /dev/null; done

5
CPU: 0.00s      Real: 0.00s     RAM: 2080KB
50
CPU: 0.06s      Real: 0.01s     RAM: 3048KB
500
CPU: 0.61s      Real: 0.12s     RAM: 7760KB
5000
CPU: 6.02s      Real: 1.23s     RAM: 17712KB # 17 MB
50000
CPU: 62.30s     Real: 12.53s    RAM: 207720KB # 207 MB
500000
CPU: 649.47s    Real: 131.53s   RAM: 3008180KB # 3 GB

你想知道在Erlang或Elixir中的等效代码是什么。我尝试过以下代码:

# test.exs
defmodule Recursion do
  def print_multiple_times(n) when n <= 1 do
    spawn fn -> IO.puts n end
  end

  def print_multiple_times(n) do
    spawn fn -> IO.puts n end
    print_multiple_times(n - 1)
  end
end

[x] = System.argv()
{k, _} = Integer.parse(x)
k = k * 1000
Recursion.print_multiple_times(k)

在elixir 1.4.2 (erts-8.2.2)中的运行结果如下:

$ for k in 5 50 500 5000 50000 ; do echo -n $k; time elixir --erl "+P 90000000" test.exs $k > /dev/null; done

5
CPU: 0.53s      Real: 0.50s     RAM: 842384KB # 842 MB
50
CPU: 1.50s      Real: 0.62s     RAM: 934276KB # 934 MB
500
CPU: 11.92s     Real: 2.53s     RAM: 1675872KB # 1.6 GB
5000
CPU: 122.65s    Real: 20.20s    RAM: 4336116KB # 4.3 GB
50000
CPU: 1288.65s   Real: 209.66s   RAM: 6573560KB # 6.5 GB

但我不确定这两者是否等效。它们是等效的吗?

编辑 根据mudasobwa的评论,我缩短了代码,因为之前的版本没有给出正确的输出。

# test2.exs
[x] = System.argv()
{k, _} = Integer.parse(x)
k = k * 1000
1..k |> Enum.each(fn n -> spawn fn -> IO.puts n end end)

在运行for k in 5 50 500 5000 50000 ; do echo -n $k; time elixir --erl "+P 90000000" test.exs $k | wc -l ; done命令后的结果如下:

5
CPU: 0.35s      Real: 0.41s     RAM: 1623344KB # 1.6 GB
2826 # 没有完成,应该是5000
50
CPU: 1.08s      Real: 0.53s     RAM: 1691060KB # 1.6 GB
35062
500
CPU: 8.69s      Real: 1.70s     RAM: 2340200KB # 2.3 GB
373193
5000        
CPU: 109.95s    Real: 18.49s    RAM: 4980500KB # 4.9 GB
4487475
50000
erl_child_setup closed
Crash dump is being written to: erl_crash.dump...Command terminated by signal 9
CPU: 891.35s    Real: 157.52s   RAM: 24361288KB # 24.3 GB

由于测试500m花费的时间太长,并且+P 500000000参数是bad number of processes,所以没有测试500m。

如何在Erlang或Elixir中生成n-k个轻量级进程?

英文:

In Go I can create goroutines like this (EDITED as reported by kelu-thatsall's answer):

// test.go
package main

import (
        &quot;fmt&quot;
        &quot;os&quot;
        &quot;strconv&quot;
        &quot;sync&quot;
        &quot;runtime&quot;
)

func main() {
        var wg sync.WaitGroup
        if len(os.Args) &lt; 2 {
                os.Exit(1)
        }
        k, ok := strconv.Atoi(os.Args[1])
        if ok != nil {
                os.Exit(2)
        }
        wg.Add(k * 1000)
        for z := 0; z &lt; k*1000; z++ {
            go func(x int) {
                    defer wg.Done()
                    fmt.Println(x)
            }(z)
            if z%k == k-1 {
                // @mattn: avoid busy loop, so Go can start processing like BEAM do 
                runtime.Gosched() 
            }
        }
        wg.Wait()
}

The result in Go 1.8.0 (64-bit):

# shell    
$ go build test.go ; for k in 5 50 500 5000 50000 500000; do echo -n $k; time ./test $k &gt; /dev/null; done

5
CPU: 0.00s      Real: 0.00s     RAM: 2080KB
50
CPU: 0.06s      Real: 0.01s     RAM: 3048KB
500
CPU: 0.61s      Real: 0.12s     RAM: 7760KB
5000
CPU: 6.02s      Real: 1.23s     RAM: 17712KB # 17 MB
50000
CPU: 62.30s     Real: 12.53s    RAM: 207720KB # 207 MB
500000
CPU: 649.47s    Real: 131.53s   RAM: 3008180KB # 3 GB

What's the equivalent code in Erlang or Elixir? (EDITED as reported by patrick-oscity's comment)

What I've tried so far is the following:

# test.exs
defmodule Recursion do
  def print_multiple_times(n) when n &lt;= 1 do
    spawn fn -&gt; IO.puts n end
  end

  def print_multiple_times(n) do
    spawn fn -&gt; IO.puts n end
    print_multiple_times(n - 1)
  end
end

[x]=System.argv()
{k,_}=Integer.parse(x)
k=k*1000
Recursion.print_multiple_times(k)

The result in elixir 1.4.2 (erts-8.2.2):

# shell    
$ for k in 5 50 500 5000 50000 ; do echo -n $k; time elixir --erl &quot;+P 90000000&quot; test.exs $k &gt; /dev/null; done

5
CPU: 0.53s      Real: 0.50s     RAM: 842384KB # 842 MB
50
CPU: 1.50s      Real: 0.62s     RAM: 934276KB # 934 MB
500
CPU: 11.92s     Real: 2.53s     RAM: 1675872KB # 1.6 GB
5000
CPU: 122.65s    Real: 20.20s    RAM: 4336116KB # 4.3 GB
50000
CPU: 1288.65s   Real: 209.66s   RAM: 6573560KB # 6.5 GB

But I'm not sure if the two are equivalent. Are they ?

EDIT Shortened version as mudasobwa's comment does not give correct output

# test2.exs
[x]=System.argv()
{k,_}=Integer.parse(x)
k=k*1000
1..k |&gt; Enum.each(fn n -&gt; spawn fn -&gt; IO.puts n end end)

The result for k in 5 50 500 5000 50000 ; do echo -n $k; time elixir --erl &quot;+P 90000000&quot; test.exs $k | wc -l ; done:

5
CPU: 0.35s      Real: 0.41s     RAM: 1623344KB # 1.6 GB
2826 # does not complete, this should be 5000
50
CPU: 1.08s      Real: 0.53s     RAM: 1691060KB # 1.6 GB
35062
500
CPU: 8.69s      Real: 1.70s     RAM: 2340200KB # 2.3 GB
373193
5000        
CPU: 109.95s    Real: 18.49s    RAM: 4980500KB # 4.9 GB
4487475
50000
erl_child_setup closed
Crash dump is being written to: erl_crash.dump...Command terminated by signal 9
CPU: 891.35s    Real: 157.52s   RAM: 24361288KB # 24.3 GB

Not testing 500m for elixir because it took too long and +P 500000000 argument is bad number of processes

如何在Erlang或Elixir中生成n-k个轻量级进程?

答案1

得分: 3

很抱歉,但我不确定这段Go代码是否按预期工作。我不是专家,如果我错了,请纠正我。首先,它打印出z,它似乎是全局范围内的当前值(通常是k*1000)https://play.golang.org/p/a4TJyjKBQh

// test.go
package main
import (
  "fmt"
  "time"
)

func main() {
  
  for z:=0; z<1000; z++ {
    go func(x int) { // I'm passing z to the function with current value now
      fmt.Println(x) 
    }(z)
  }

  time.Sleep(1 * time.Nanosecond)

}

而且,如果我注释掉Sleep,程序甚至在启动任何goroutine之前就会退出(至少不会打印出结果)。我很乐意知道我是否做错了什么,但从这个简单的例子来看,问题似乎不在于Elixir,而是提供的Go代码。有一些Go大师在吗?

我还在本地机器上运行了一些测试:

go run test.go 500 | wc -l
72442 # 期望值为 500000
go run test.go 5000 | wc -l
76274 # 期望值为 5000000
英文:

I'm sorry guys but I'm not convinced that this code in Go is really working as expected. I'm not an expert, so please correct me if I'm wrong. First of all it prints z which it seems is a current value of it in global scope (usually k*1000) https://play.golang.org/p/a4TJyjKBQh

// test.go
package main
import (
  &quot;fmt&quot;
  &quot;time&quot;
)

func main() {
  
  for z:=0; z&lt;1000; z++ {
    go func(x int) { // I&#39;m passing z to the function with current value now
      fmt.Println(x) 
    }(z)
  }

  time.Sleep(1 * time.Nanosecond)

}

And also if I comment out Sleep the program will exit before even starting any goroutines (at least it doesn't print out the results). I would be happy to know if I'm doing something wrong, but from this simple example it seems the problem is not with Elixir, but Go code provided. Some Go gurus out there?

I've also run some test on my local machine:

go run test.go 500 | wc -l
72442 # expected 500000
go run test.go 5000 | wc -l
76274 # expected 5000000

huangapple
  • 本文由 发表于 2017年1月27日 20:29:44
  • 转载请务必保留本文链接:https://go.coder-hub.com/41894046.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定