英文:
Difference between the main goroutine and spawned goroutines of a Go program
问题
使用gRPC
创建服务器时,如果我在主进程中启动gRPC
服务器,它可以处理来自客户端的大量请求(数千个)。然而,如果我将服务器作为goroutine启动,它只能处理一些请求(数百个)然后卡住。我已经使用一个非常简单的示例进行了测试和确认,示例代码位于google.golang.org/grpc/examples/helloworld。
这是因为生成的goroutine的堆栈大小非常小(2K字节),而主goroutine的堆栈要大得多。主goroutine和生成的goroutine之间有什么区别?
示例代码的修改部分如下所示:
greeter_server/main.go
func main() {
go func() {
lis, err := net.Listen("tcp", port)
if err != nil {
log.Fatalf("failed to listen: %v", err)
}
s := grpc.NewServer()
pb.RegisterGreeterServer(s, &server{})
s.Serve(lis)
}()
for {
}
}
greeter_client/main.go
func main() {
// Set up a connection to the server.
for i := 0; i < 500; i++ {
conn, err := grpc.Dial(address, grpc.WithInsecure())
if err != nil {
log.Fatalf("did not connect: %v", err)
}
defer conn.Close()
c := pb.NewGreeterClient(conn)
for i := 0; i < 500; i++ {
// Contact the server and print out its response.
name := defaultName
if len(os.Args) > 1 {
name = os.Args[1]
}
r, err := c.SayHello(context.Background(), &pb.HelloRequest{Name: name})
if err != nil {
log.Fatalf("could not greet: %v", err)
}
log.Printf("%d's Greeting: %s", i, r.Message)
}
}
}
英文:
When creating a server using gRPC
, if I start the gRPC
server in the main process, it can deal with as many as requests (thousands) from clients. However, if I start the server as a goroutine, it can only handle some requests (hundreds) and after get stuck. I have tested and confirmed this with a very simple example, google.golang.org/grpc/examples/helloworld.
Is it because spawned goroutines stack size is very small (2Kbytes), and the main goroutine's much larger? What's the difference between the main goroutine and spawned goroutines?
Example link. Modified parts of the example as follows.
greeter_server/main.go
func main() {
go func() {
lis, err := net.Listen("tcp", port)
if err != nil {
log.Fatalf("failed to listen: %v", err)
}
s := grpc.NewServer()
pb.RegisterGreeterServer(s, &server{})
s.Serve(lis)
}()
for {
}
}
greeter_client/main.go
func main() {
// Set up a connection to the server.
for i := 0; i < 500; i++ {
conn, err := grpc.Dial(address, grpc.WithInsecure())
if err != nil {
log.Fatalf("did not connect: %v", err)
}
defer conn.Close()
c := pb.NewGreeterClient(conn)
for i := 0; i < 500; i++ {
// Contact the server and print out its response.
name := defaultName
if len(os.Args) > 1 {
name = os.Args[1]
}
r, err := c.SayHello(context.Background(), &pb.HelloRequest{Name: name})
if err != nil {
log.Fatalf("could not greet: %v", err)
}
log.Printf("%d's Greeting: %s", i, r.Message)
}
}
}
答案1
得分: 7
为什么 Goroutine 的堆栈是无限的:
Goroutine 的一个关键特性是它们的成本;从初始内存占用的角度来看,它们的创建成本很低(与传统的 POSIX 线程相比,后者需要 1 到 8 兆字节的内存)。它们的堆栈会根据需要自动增长和收缩。这意味着 Goroutine 可以从一个 4096 字节的堆栈开始,随着需要的增长和收缩,而不会出现堆栈耗尽的风险。
然而,直到现在我才揭示一个细节,它将意外使用递归函数与操作系统的严重内存耗尽问题联系起来,即当需要新的堆栈页时,它们是从堆中分配的。
随着无限函数不断调用自身,新的堆栈页将从堆中分配,使函数能够一次又一次地调用自身。很快,堆的大小将超过机器上的可用物理内存量,此时交换操作将使机器无法使用。
Go 程序可用的堆大小取决于很多因素,包括 CPU 架构和操作系统等,但它通常表示的是一个超过机器物理内存量的内存量,因此在程序耗尽堆之前,机器很可能会频繁进行交换操作。
参考:http://dave.cheney.net/2013/06/02/why-is-a-goroutines-stack-infinite
空循环:
for{
}
会占用一个 CPU 核心的 100% 的使用率,以等待某些操作。根据使用情况,你可以使用以下方法:
> 这是因为生成的 goroutine 的堆栈大小很小(2K字节),而主 goroutine 的堆栈要大得多吗?
不是的,你可以尝试这两个示例来查看 goroutine 的堆栈限制是相同的:
在 Go Playground 上运行一个主 goroutine,
在 Go Playground 上尝试第二个 goroutine:
package main
import (
"fmt"
"sync"
)
var wg sync.WaitGroup
func main() {
wg.Add(1)
go run()
wg.Wait()
}
func run() {
s := &S{a: 1, b: 2}
fmt.Println(s)
wg.Done()
}
type S struct {
a, b int
}
// String implements the fmt.Stringer interface
func (s *S) String() string {
return fmt.Sprintf("%s", s) // Sprintf will call s.String()
}
在 Go Playground 上的输出结果是相同的:
runtime: goroutine stack exceeds 250_000_000-byte limit
fatal error: stack overflow
在一台拥有 8 GB
RAM 的计算机上的输出结果是:
runtime: goroutine stack exceeds 1_000_000_000-byte limit
fatal error: stack overflow
英文:
Why is a Goroutine’s stack infinite:
> One of the key features of Goroutines is their cost; they are cheap to
> create in terms of initial memory footprint (as opposed to the 1 to 8
> megabytes with a traditional POSIX thread) and their stack grows and
> shrinks as necessary. This allows a Goroutine to start with a single
> 4096 byte stack which grows and shrinks as needed without the risk of
> ever running out.
> There is however one detail I have withheld until now, which links the
> accidental use of a recursive function to a serious case of memory
> exhaustion for your operating system, and that is, when new stack
> pages are needed, they are allocated from the heap.
>
> As your infinite function continues to call itself, new stack pages
> are allocated from the heap, permitting the function to continue to
> call itself over and over again. Fairly quickly the size of the heap
> will exceed the amount of free physical memory in your machine, at
> which point swapping will soon make your machine unusable.
>
> The size of the heap available to Go programs depends on a lot of
> things, including the architecture of your CPU and your operating
> system, but it generally represents an amount of memory that exceeds
> the physical memory of your machine, so your machine is likely to swap
> heavily before your program ever exhausts its heap.
ref: http://dave.cheney.net/2013/06/02/why-is-a-goroutines-stack-infinite
Empty loop:
for{
}
uses 100% of a CPU Core, to wait for some operation depending to the use case you may use:
> Is it because spawned goroutines stack size is very small (2Kbytes),
> and the main goroutine's much larger?
No, you may try these two samples to see the stack limit of goroutines are the same:
one main goroutine on The Go Playground,
try second goroutine on The Go Playground:
package main
import (
"fmt"
"sync"
)
var wg sync.WaitGroup
func main() {
wg.Add(1)
go run()
wg.Wait()
}
func run() {
s := &S{a: 1, b: 2}
fmt.Println(s)
wg.Done()
}
type S struct {
a, b int
}
// String implements the fmt.Stringer interface
func (s *S) String() string {
return fmt.Sprintf("%s", s) // Sprintf will call s.String()
}
both outputs are the same on the Go Playground:
runtime: goroutine stack exceeds 250_000_000-byte limit
fatal error: stack overflow
outputs on a PC with 8 GB
RAM:
runtime: goroutine stack exceeds 1_000_000_000-byte limit
fatal error: stack overflow
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论