英文:
How to measure system overload when using GO
问题
我正在用GO语言重写一个旧系统,在旧系统中,我通过测量系统的负载平均值来判断是否应该增加线程池中的线程数量。
在Go语言中,人们不使用线程池或goroutine池,因为启动一个goroutine非常廉价。
但是,运行过多的goroutine比仅保持CPU使用率接近100%的数量效率要低。
因此,有没有办法知道有多少个goroutine准备好运行(未被阻塞),但当前没有运行?或者有没有办法获取调度的可运行goroutine的数量("运行队列")?
英文:
I am rewriting an old system in GO, and in the old system I was measuring the system load average to know if I should increase the number of thread in my thread-pool.
In go people are not using threadpool or pool of goroutine because starting a goroutine is very cheap.
But still running too many goroutine is less efficient then just enough to keep the cpu usage near 100%
Thus is there a way to know how many goroutine are ready to run (not blocked) but not currently running. Or is there a way to get the number of scheduled runnable goroutine "Run queue".
答案1
得分: 10
请注意,我将为您提供代码的翻译部分。以下是您提供的代码的翻译:
// 导入所需的包
import (
"fmt"
"math/rand"
"os"
"runtime"
"runtime/pprof"
"strconv"
"sync"
"time"
)
var (
wg sync.WaitGroup
m sync.Mutex
)
// 随机等待一段时间
func randWait() {
defer wg.Done()
m.Lock()
defer m.Unlock()
interval, err := time.ParseDuration(strconv.Itoa(rand.Intn(499)+1) + "ms")
if err != nil {
fmt.Errorf("%s\n", err)
}
time.Sleep(interval)
return
}
// 打印阻塞统计信息
func blockStats() {
for {
pprof.Lookup("block").WriteTo(os.Stdout, 1)
fmt.Println("# Goroutines:", runtime.NumGoroutine())
time.Sleep(5 * time.Second)
}
}
func main() {
rand.Seed(time.Now().Unix())
runtime.SetBlockProfileRate(1)
fmt.Println("Running...")
for i := 0; i < 100; i++ {
wg.Add(1)
go randWait()
}
go blockStats()
wg.Wait()
fmt.Println("Finished.")
}
希望这对您有所帮助。
英文:
Check out the runtime/pprof package.
To print "stack traces of all current goroutines" use:
pprof.Lookup("goroutine").WriteTo(os.Stdout, 1)
To print "stack traces that led to blocking on synchronization primitives" use:
pprof.Lookup("block").WriteTo(os.Stdout, 1)
You can combine these with the functions in the runtime package such as runtime.NumGoroutine
to get some basic reporting.
This example deliberately creates many blocked goroutines and waits for them to complete. Every 5 seconds it prints the output of the block
pprof profile, as well as the number of goroutines still in existence:
package main
import (
"fmt"
"math/rand"
"os"
"runtime"
"runtime/pprof"
"strconv"
"sync"
"time"
)
var (
wg sync.WaitGroup
m sync.Mutex
)
func randWait() {
defer wg.Done()
m.Lock()
defer m.Unlock()
interval, err := time.ParseDuration(strconv.Itoa(rand.Intn(499)+1) + "ms")
if err != nil {
fmt.Errorf("%s\n", err)
}
time.Sleep(interval)
return
}
func blockStats() {
for {
pprof.Lookup("block").WriteTo(os.Stdout, 1)
fmt.Println("# Goroutines:", runtime.NumGoroutine())
time.Sleep(5 * time.Second)
}
}
func main() {
rand.Seed(time.Now().Unix())
runtime.SetBlockProfileRate(1)
fmt.Println("Running...")
for i := 0; i < 100; i++ {
wg.Add(1)
go randWait()
}
go blockStats()
wg.Wait()
fmt.Println("Finished.")
}
I'm not sure if that's what you're after, but you may be able to modify it to suit your needs.
答案2
得分: 2
有没有办法知道有多少个 goroutine 准备好运行(没有被阻塞),但当前没有在运行?
您将能够(2014年第四季度/2015年第一季度)尝试并可视化这些 goroutine,因为正在开发一个新的跟踪器(2014年第四季度):Go 执行跟踪器
跟踪包含以下内容:
- 与 goroutine 调度相关的事件:
- 一个 goroutine 在处理器上开始执行,
- 一个 goroutine 在同步原语上阻塞,
- 一个 goroutine 创建或解除阻塞另一个 goroutine;
- 与网络相关的事件:
- 一个 goroutine 在网络 IO 上阻塞,
- 一个 goroutine 在网络 IO 上解除阻塞;
- 与系统调用相关的事件:
- 一个 goroutine 进入系统调用,
- 一个 goroutine 从系统调用返回;
- 与垃圾回收器相关的事件:
- 垃圾回收开始/停止,
- 并发扫描开始/停止;
- 用户事件。
通过“处理器”,我指的是逻辑处理器,即 GOMAXPROCS
的单位。
每个事件都包含事件 ID、精确时间戳、操作系统线程 ID、处理器 ID、goroutine ID、堆栈跟踪和其他相关信息(例如,解除阻塞的 goroutine ID)。
英文:
> is there a way to know how many goroutine are ready to run (not blocked) but not currently running.?
You will be able (Q4 2014/Q1 2015) to try and visualize those goroutines, with a new tracer being developed (Q4 2014): Go Execution Tracer
> The trace contains:
> - events related to goroutine scheduling:
- a goroutine starts executing on a processor,
- a goroutine blocks on a synchronization primitive,
- a goroutine creates or unblocks another goroutine;
- network-related events:
- a goroutine blocks on network IO,
- a goroutine is unblocked on network IO;
- syscalls-related events:
- a goroutine enters into syscall,
- a goroutine returns from syscall;
- garbage-collector-related events:
- GC start/stop,
- concurrent sweep start/stop; and
- user events.
> By "processor" I mean a logical processor, unit of GOMAXPROCS
.
Each event contains event id, a precise timestamp, OS thread id, processor id, goroutine id, stack trace and other relevant information (e.g. unblocked goroutine id).
<a href="https://lh5.googleusercontent.com/w0znUT_0_xbipG_UlQE5Uc4PbC8Mw1duHRLg_AKTOS4iS6emOD6jnQvSDACybOfCbuSqr2ulkxULXGOBQpZ2IejPHW_8NHufqmn8q5u-fF_MSMCEgu6FwLNtMvowbq74nA">
</a>
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论