使用GOMAXPROCS在Go中进行并行编程

huangapple go评论84阅读模式
英文:

Parallel programming in go using GOMAXPROCS

问题

我看到有人将runtime.GOMAXPROCS设置为runtime.NumCPU()以在Go中启用并行处理。官方文档没有提到GOMAXPROCS的上限;我可以将其设置为任意正整数吗,还是它应该始终小于或等于NumCPU的值?

我尝试将其设置为大于逻辑核心数量的数字,我的代码运行得很好。

英文:

I have seen people setting runtime.GOMAXPROCS to runtime.NumCPU() to enable parallel processing in go. Official documentation doesn't say anything about the upper bound of GOMAXPROCS; can I set this to any arbitrary positive integer or should it always be less than eq. to NumCPU value?

I tried setting it to a number that's greater than the # of logical cores, my code works just fine

答案1

得分: 6

大多数情况下,你不需要去调整GOMAXPROCS,因为运行时会自动与操作系统进行交互。在Go 1.5之前,GOMAXPROCS的默认值是1,但现在它默认为NumCPU()

GOMAXPROCS设置得比NumCPU()更高只会给调度器增加(不必要的)处理操作系统线程的工作负担。

英文:

You shouldn't need to mess with GOMAXPROCS most of the time since the runtime interacts with the OS for you. GOMAXPROCS used to default to 1, but with Go 1.5, it now defaults to the NumCPU()

Setting it higher than NumCPU will only give the scheduler more (unnecessary) work for dealing with OS threads.

答案2

得分: 3

最大值是256;但请注意,如果您将其设置得更高,它不会报错。

https://golang.org/src/runtime/debug.go?s=534:560#L7

12	// GOMAXPROCS设置可以同时执行的最大CPU数量,并返回先前的设置。如果n < 1,则不会更改当前设置。
13	// 可以使用NumCPU查询本地计算机上的逻辑CPU数量。
14	// 当调度程序改进时,此调用将消失。
15	func GOMAXPROCS(n int) int {
16		if n > _MaxGomaxprocs {
17			n = _MaxGomaxprocs
18		}
19		lock(&sched.lock)
20		ret := int(gomaxprocs)
21		unlock(&sched.lock)
22		if n <= 0 || n == ret {
23			return ret
24		}
25	
26		stopTheWorld("GOMAXPROCS")
27	
28		// newprocs将由startTheWorld处理
29		newprocs = int32(n)
30	
31		startTheWorld()
32		return ret
33	}

第19行将总数设置为_MaxGomaxprocs

这是...

https://golang.org/src/runtime/runtime2.go?h=_MaxGomaxprocs#L407

const (
    // GOMAXPROCS的最大值。
    // 对该值没有基本限制。
    _MaxGomaxprocs = 1 << 8
)

这个二进制形式的位移是100000000,而1是一个int,在64位系统上,Go将其作为Int64处理,这意味着最大值是256。(在32位系统中,Go中的int将是int32,但值仍为256)

现在,关于将其设置为超过您的核心数 - 这完全取决于您的工作负载和正在发生的CPU上下文切换(例如,您的Go程序强制发生了多少次锁定,或者您是否在各个地方使用了mutex.Lock()等)。

请记住,如果需要进行上下文切换,Golang会将其抽象化处理。

在这里,基准测试是您的朋友。如果您的应用程序以良好的设计模式运行了10,000个goroutine,并且几乎没有CPU上下文切换,那么可以将其设置为256并让其运行。如果您的应用程序由于上下文切换/线程而导致大量的CPU等待时间,则将其设置为8或4,以便为所有进行的mutex锁定提供足够的空间。

英文:

The max is 256; but note, it will not error if you set it higher.

https://golang.org/src/runtime/debug.go?s=534:560#L7

12	// GOMAXPROCS sets the maximum number of CPUs that can be executing
13	// simultaneously and returns the previous setting.  If n &lt; 1, it does not
14	// change the current setting.
15	// The number of logical CPUs on the local machine can be queried with NumCPU.
16	// This call will go away when the scheduler improves.
17	func GOMAXPROCS(n int) int {
18		if n &gt; _MaxGomaxprocs {
19			n = _MaxGomaxprocs
20		}
21		lock(&amp;sched.lock)
22		ret := int(gomaxprocs)
23		unlock(&amp;sched.lock)
24		if n &lt;= 0 || n == ret {
25			return ret
26		}
27	
28		stopTheWorld(&quot;GOMAXPROCS&quot;)
29	
30		// newprocs will be processed by startTheWorld
31		newprocs = int32(n)
32	
33		startTheWorld()
34		return ret
35	}

Line 19 sets the total number to _MaxGomaxprocs.

Which is...

https://golang.org/src/runtime/runtime2.go?h=_MaxGomaxprocs#L407

const (
    // The max value of GOMAXPROCS.
    // There are no fundamental restrictions on the value.
    _MaxGomaxprocs = 1 &lt;&lt; 8
)

That bitshift in binary form is 100000000 and 1 is an int which on 64bit systems Go makes that an Int64, which means the max is 256. (32bit systems with int in Go would be int32, but same value of 256)

Now, as far as setting this to more than your number of cores - it all depends on your workload and CPU context switching that is happening (e.g. how many locks does your go program force to happen, or do you use mutex.Lock() everywhere, etc).

Remember, Golang abstracts away the context switching if it needs to happen or not.

Benchmarking is your friend here. If your app runs 10,000 goroutines with little to no cpu context switching (following good design patterns), then yes bump that sucker to 256 and let it ride. If you app chokes and creates a lot of CPU wait-time with context switching/threads, then set it to 8 or 4 even to give it room to breath with all that mutex locking going on.

huangapple
  • 本文由 发表于 2016年4月8日 13:43:02
  • 转载请务必保留本文链接:https://go.coder-hub.com/36492356.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定