英文:
Golang, processes and shared memory
问题
今天我的一个朋友告诉我，Go程序可以在多个CPU核心上进行自我扩展。听到这个消息我感到非常惊讶，因为系统任务调度程序并不知道goroutine的存在，所以无法在多个核心上运行它们。
我进行了一些搜索，发现Go程序可以生成多个操作系统任务，以在不同的核心上运行它们（数量由GOMAXPROCS环境变量控制）。但据我所知，fork一个进程会导致进程数据的完全复制，并且不同的进程在不同的地址空间中运行。
那么，在Go程序中的全局变量又是怎样的呢？它们在多个goroutine中使用是安全的吗？它们是否会在系统进程之间进行同步？如果是的话，具体是如何实现的呢？我主要关注Linux和FreeBSD的实现情况。
英文:
Today a friend of mine told me that Go programs can scale themselves on multiple CPU cores. I were quite surprised to hear that knowing that system task schedulers do not know anything about goroutines and hence can't run them on multiple cores.
I did some search and found out that Go programs can spawn multiple OS tasks to run them on different cores (the number is controlled by GOMAXPROCS environment variable). But as far as I know forking a process leads to complete copy of process data and different processes run in different address spaces.
So what about global variables in Go programs? Are they safe to use with multiple goroutines? Do they somehow synchronize between system processes? And if they do then how? I am mainly concerned about linux and freebsd implementations.
答案1
得分: 6
我搞清楚了！一切都在Go源代码中。
有一个Linux系统调用我之前不知道。
它叫做"clone"。它比fork更灵活，允许子进程在父进程的地址空间中运行。
下面是线程创建过程的简要概述。
首先，在src/runtime/proc.go中有一个newm函数。这个函数负责创建一个新的工作线程（或者在注释中称为machine）。
// 创建一个新的m。它将以对fn的调用开始，或者调度器。
// fn需要是静态的，不能是堆分配的闭包。
// 可能在m.p==nil的情况下运行，所以不允许写屏障。
//go:nowritebarrier
func newm(fn func(), _p_ *p) {
// ... 跳过一些代码 ...
newosproc(mp, unsafe.Pointer(mp.g0.stack.hi))
}
这个函数调用了newosproc，它是与操作系统相关的。
对于Linux，可以在src/runtime/os_linux.go中找到。下面是该文件的相关部分：
var (
// ...
cloneFlags = _CLONE_VM | /* 共享内存 */
_CLONE_FS | /* 共享当前工作目录等 */
_CLONE_FILES | /* 共享文件描述符表 */
_CLONE_SIGHAND | /* 共享信号处理程序表 */
_CLONE_THREAD /* 重新审视 - 目前可以接受 */
)
// 可能在m.p==nil的情况下运行，所以不允许写屏障。
//go:nowritebarrier
func newosproc(mp *m, stk unsafe.Pointer) {
// ... 跳过一些代码 ...
ret := clone(cloneFlags, /* ... 其他标志 ... */)
// ... 跳过一些代码 ...
}
clone函数在特定于体系结构的文件中定义。对于amd64，它在src/runtime/sys_linux_amd64.s中。它是实际的系统调用。
因此，Go程序确实在多个操作系统线程中运行，这使得它们可以跨多个CPU运行，但它们使用一个共享的地址空间。
哇...我喜欢Go。
英文:
I figured it out! It's all in go sources.
There is a Linux system call that I were unaware of.
It's called "clone". It is more flexible than fork and it allows
a child process to live in its parent's address space.
Here is a short overview of the thread creation process.
First there is a newm function in src/runtime/proc.go. This
function is responsible for creating a new working thread
(or machine as it is called in comments).
// Create a new m. It will start off with a call to fn, or else the scheduler.
// fn needs to be static and not a heap allocated closure.
// May run with m.p==nil, so write barriers are not allowed.
//go:nowritebarrier
func newm(fn func(), _p_ *p) {
// ... some code skipped ...
newosproc(mp, unsafe.Pointer(mp.g0.stack.hi))
}
This function calls newosproc which is OS-specific.
For Linux it can be found in src/runtime/os_linux.go. Here
are relevant parts of that file:
var (
// ...
cloneFlags = _CLONE_VM | /* share memory */
_CLONE_FS | /* share cwd, etc */
_CLONE_FILES | /* share fd table */
_CLONE_SIGHAND | /* share sig handler table */
_CLONE_THREAD /* revisit - okay for now */
)
// May run with m.p==nil, so write barriers are not allowed.
//go:nowritebarrier
func newosproc(mp *m, stk unsafe.Pointer) {
// ... some code skipped ...
ret := clone(cloneFlags, /* ... other flags ... */)
// ... code skipped
}
And the clone function is defined in architecture-specific
files. For amd64 it is in src/runtime/sys_linux_amd64.s.
It is the actual system call.
So Go programs do run in multiple OS threads which enables
spanning across CPUs, but they use one shared address space.
Phew... I love Go.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论