我应该如何在后台运行我的Golang进程?

huangapple go评论107阅读模式
英文:

How I should run my Golang process in background?

问题

这个问题与编程无关,但对程序员来说肯定很重要。

我写了一个简单的smtp服务器,当我从控制台运行它时一切正常,除了它阻塞了命令行。

我知道我可以通过以下方式运行它:

nohup ... &;

或通过screen / tmux等方式。

但问题是,我应该如何实现我的程序在后台运行,并且对系统管理员来说设置和管理该进程将是一种乐趣?

一些比我经验丰富得多的人在golang-nuts上写道,他们不使用fork等,而是使用一些形式的“包装器”如monit等。

目标平台是基于Debian的,盒子上的所有其他东西都是基于init.d的。

有关该主题的好资源或一个写得很好的示例项目的来源吗?

英文:

This question is not strictly programming related, but for sure important for programmers.

I wrote a simple smtp server, when I run it from console all is fine, except it is blocking the command line.

I know I can run it via

nohup ... &

or via screen / tmux etc

But the question is, how should I implement my program it runs in the background and it will be a pleasure for a system administrator to set it up and manage the process ?

Some guys with far more experience than me, at golang-nuts, wrote, they don't use fork etc, and use some "wrapper" in form from monit etc.

The target platform is Debian based, all other stuff on the box are init.d based.

Any good resources for that topic or sources of a well written example project ?

答案1

得分: 24

正如Nick提到的,Supervisord是一个很好的选择,在我的经验中也运行良好。

Nick提到了关于forking的问题-就forking本身而言,我认为它工作正常。问题不在于forking,而是在于降低权限。由于Go运行时启动了goroutine多路复用的线程池(当GOMAXPROX > 1时),使用setuid系统调用不是一种可靠的降低权限的方式

相反,你应该以非特权用户身份运行程序,并使用setcap工具授予它所需的权限。

例如,要允许绑定到低端口号(如80),需要在可执行文件上运行一次setcap命令:sudo setcap 'cap_net_bind_service=+ep' /opt/yourGoBinary

你可能需要安装setcap:sudo aptitude install libcap2-bin

英文:

As Nick mentioned Supervisord is a great option that has also worked well in my experience.

Nick mentioned problems with forking- forking itself works fine AFAICT. The issue is not forking but dropping privileges. Due to the way the Go runtime starts the thread pool that goroutines are multiplexed over (when GOMAXPROX > 1), the setuid systemcall is not a reliable way to drop permissions.

Instead, you should run your program as a non-privileged user and use the setcap utility to grant it the needed permissions.

For example, to allow binding to a low port number (like 80) run will need to run setcap once on the executable: sudo setcap 'cap_net_bind_service=+ep' /opt/yourGoBinary

You may need to install setcap: sudo aptitude install libcap2-bin

答案2

得分: 16

已经有很好的答案了,但我会添加一些额外的信息。

在Debian上,您不需要安装额外的软件,如supervisord来处理后台进程。

Debian带有一个名为start-stop-daemon的工具,它是在init.d脚本中启动守护进程的标准方式。如果程序本身没有将进程放入后台,它也可以为您完成这个操作。请查看--background选项。

使用/etc/init.d/skeleton作为您的init脚本的基础,但将do_start()函数更改如下:

start-stop-daemon --start --quiet --pidfile $PIDFILE --make-pidfile \
    --background --exec $DAEMON --test > /dev/null \
            || return 1
start-stop-daemon --start --quiet --pidfile $PIDFILE --make-pidfile \
    --background --exec $DAEMON -- $DAEMON_ARGS \
            || return 2

我还添加了--make-pidfile选项,它会为您创建PID文件。

如果您需要以安全方式切换到不同的用户,还有--chuid选项。

在Ubuntu和RHEL/CentOS/SL 6.X上,最简单的方法是编写一个upstart作业配置文件。只需将exec /usr/sbin/yourprogram放入/etc/init/yourprogram.conf配置文件中。使用upstart时,无需强制将程序放入后台。不要添加expect forkexpect daemon,这是传统守护进程所需的。使用upstart时,最好进程不要分叉。

英文:

There are already good answers but I will add some additional information.

You do not need to install additional software such as supervisord on Debian to take care of backgrounding the process.

Debian comes with a tool called start-stop-daemon which is a standard way for starting daemons in init.d scripts. It can also also put the process in background for you if the program does not do it on its own. Have a look at the --background option.

Use /etc/init.d/skeleton as the basis of your init script, but change the do_start() function as follows:

start-stop-daemon --start --quiet --pidfile $PIDFILE --make-pidfile \
    --background --exec $DAEMON --test > /dev/null \
            || return 1
start-stop-daemon --start --quiet --pidfile $PIDFILE --make-pidfile \
    --background --exec $DAEMON -- $DAEMON_ARGS \
            || return 2

I also added the --make-pidfile option which creates the PID file for you.

In case you need to switch to a different user in a secure way, there is also --chuid option.

On Ubuntu and RHEL/CentOS/SL 6.X the simplest way is to write an upstart job configuration file. Just put exec /usr/sbin/yourprogram in the /etc/init/yourprogram.conf configuration file. With upstart there is no need to force the program in background. Do not add expect fork or expect daemon which you need with traditional daemons. With upstart it is better if the process does not fork.

答案3

得分: 10

我之前写过一篇关于这个的博客文章。对我来说,将进程变为守护进程的想法似乎是错误的,因为这会让你担心很多其他事情(例如,当它失败时会发生什么?如何管理进程重启?如何处理日志、工作目录、核心、系统重启等等...)

事实证明,如果你试图做所有这些,事情会变得更容易。

英文:

I wrote a blog post about this a while back. The idea of daemonizing seems wrong to me as it leaves you with a lot of other things to worry about (e.g. what happens when it fails? how do you manage a process restart? how do you handle logging, working directory, cores, system restart, etc...)

Turns out, if you don't try to do all that, things get much easier.

答案4

得分: 5

Supervisord在我的经验中非常适合这个任务。

你可以编写你的应用程序在命令行上运行,打印一些东西等等,而supervisord会负责所有的守护进程、出错时的重启、速率限制等等。

我相信,以传统的Unix方式将Go程序分叉到后台是困难的,因为运行时在运行你的main()函数之前会启动一些线程。

英文:

Supervisord works really well for this in my experience.

You write your app to run on the command line, print stuff etc and supervisord takes care of all the daemonising, restarting if it goes wrong, rate limiting etc, etc

I believe that forking go programs into the background in the traditional unix way is difficult because the runtime starts some threads before it runs your main() routine

答案5

得分: 3

但问题是,我应该如何实现我的程序,使其在后台运行,并且对系统管理员来说设置和管理该进程将是一种乐趣?

这里有一些想法。

#提供软件包和存储库

安装软件是一回事,维护和运行它是完全不同的故事。当然,我可以下载一个zip文件,解压缩它,将文件放入适当的目录中(请记住,软件包通常分散在整个文件系统中),创建一个系统用户来运行守护进程,并设置相应的权限。但这很繁琐,容易出错(这将导致大量的票据,如“噢!无法运行!修复它!”),如果软件将安装在许多系统上,几乎不实用。

因此,我们需要一个软件包来降低采用的障碍。通常提供这些软件包的存储库并不是什么难事,并且使安装和/或更新变得更加容易。“下载->分发->安装/更新”与单个命令/服务器之间存在差异,例如:

$ awesomePm update coolApplication

至少为RedHat和基于Debian的系统提供软件包。就个人而言,我会选择CentOS(这将使您的软件包与几乎任何RHEL衍生版本兼容)和基本的Debian。后者应该很容易为Ubuntu提供软件包。由于我不再使用Debian或其衍生版本,我不太确定它们是否真的兼容,当我最后构建一个.deb时,启动时出现了一些问题。

提供适当的文档。记录安装的内容,位置和原因。提供与相应文档的链接。对依赖项的manpage引用就足够了。这样,即使是最没有经验的管理员也可以配置您的软件包。

使用最防御性和合理的默认设置

关于golang的特别说明:大多数软件包构建工具默认情况下会剥离软件包中包含的二进制文件。Go不支持这一点,所以要小心。

#完整

没有什么比不完整的软件包更令人恼火了。

尽可能使用syslog并遵守其约定。这样,您的日志将被放置在系统管理员期望的位置,并且如果过时,它们将自动处理,防止您的应用程序导致磁盘空间不足。如果系统管理员希望对应用程序的日志进行特殊处理,他将以这种方式进行配置。

不要通过应用程序旋转日志。用户可以自行决定如何处理它们(这可能与他们的SLA有关)。即使您使日志旋转的方式可配置,管理员也必须学习如何配置它-这引入了不必要的冗余。

如果必须写入日志文件,请遵守目标系统的日志策略,并提供一个日志旋转配置文件。您不希望您的应用程序因机器磁盘空间不足而导致停机,对吗?

使用系统工具而不是重新发明轮子。如果您的应用程序需要进行一些维护工作,请不要费心在应用程序内部使用调度程序。编写一个专门的维护工具(单体应用程序已经过时了)并利用cron。特别是将相应文件添加到/etc/cron*目录之一。

**提供适当的初始化脚本!**这样,管理员可以使用诸如systemctl之类的众所周知的工具来管理应用程序的启动和关闭。当您必须su到用户或使用sudo -u在启动时调用shell脚本时,这是非常烦人的。即使此脚本被称为@onboot,与标准的偏离也很烦人。仅仅因为启动方法有效并不意味着应该使用它。

为添加SE-Linux配置文件加分!

这应该是不言而喻的,但我经常看到配置错误的软件包,所以:**测试您的软件包!**从目标操作系统的最小安装开始,安装您的软件包并确保它按预期运行。检查您提供的每个配置。

如果您计划将软件包放入官方的Debian存储库中,您应该计划一些时间:Debian之所以如此稳定,是因为对软件包的要求非常严格,即使您满足所有要求,也必须从测试经过不稳定到稳定的所有步骤。

#准确

**不要仅仅因为方便而使用现有用户。**如果您创建一个Web应用程序,请不要重用“apache”或“www”用户。创建一个专用于您的软件包的用户,并将此用户添加到相应的组中

遵守最少必要权限原则。几乎没有理由使二进制文件具有全局可执行权限,更不用说全局可写权限了(这将是一个极端**的安全漏洞)。在这里,您经常在SO上看到的建议是将权限设置为[0]777,这允许每个用户对文件进行任何修改。实际上,几乎没有理由使二进制文件对任何用户可写:无论如何进行更新的root用户始终可以写入任何内容。因此,二进制文件的权限应为0550。该原则也适用于数据目录等。在这里投入一些时间和精力。您不希望您的应用程序成为成功攻击的向量,对吗?即使潜在的安全风险也往往会对您和您的声誉产生反作用。我倾向于将所有数据文件设置为0600,以便由应用程序的系统用户写入的文件,对于只读文件设置为0400,对于二进制文件设置为0500。然后,我会对组权限进行详细分析。例如:组可能会更改Web应用程序的各个模板-但很可能不会更改资源目录子树的目录结构。

如果您在安全性方面付出了努力,您将增加信任。请注意,软件包通常在决定是否采用之前会对其安全影响进行检查。

**遵守FHS(!!)!**即使如此:仅仅因为您可以在/opt/yourapplication下执行任何操作,这并不总是一个好主意。而是安装到/usr/var中(假设您的应用程序在引导时不是必需的)。

**如果有依赖关系,请定义它们。**不要仅仅假设软件包存在。

如果您对本地SMTP服务器有依赖关系,请不要声明对postfix的依赖关系。也许管理员更喜欢sendmail(出于任何原因)。因此,定义对mail-transport-agent(Debian)或mta(RH,我记得)的依赖关系。

结论

这就是我对一个好软件的期望-与现有软件很好地集成,并使其易于安装,维护,配置和运行,而无需学习冗余配置。如果我看到一个软件包的SELinux配置文件,那真的会给供应商加分-除非配置文件非常粗糙,这表明供应商非常重视安全性。

英文:

> But the question is, how should I implement my programm it runs in the background and it will be a pleasure for a system administrator to set it up and manage the process ?

A few thoughts here.

Provide packages and a repository

Installing a software is one thing, maintaining and running it is a totally different story. Sure, I can download a zip, unpack it, put the files into the proper directories (keep in mind that packages are usually scattered throughout the file system), create a system user to run the daemon, set the according permissions. But that's tedious, error prone (which will lead to a high influx of tickets like "Bah! Does not run! Fix it!") and hardly practical if the software is going to be installed on many systems.

So we need a package to lower the hurd of adoption. Providing a repository for those packages usually is no rocket science and makes installation and/or updates much easier. There is a difference between "Download->Distribute->Install/Update" and a single command/server like

$ awesomePm update coolApplication

Provide at least packages for RedHat and Debian based systems. Personally, I'd go for CentOS (which would make your package compatible to almost any RHEL derivate) and basic Debian. The latter should make it trivial to provide a package for Ubuntu, too. Since I don't use Debian or derivates any more, I am not too sure wether they are really compatible, there have been startup issues when I last built a .deb.

Provide proper documentation. Document what is installed, where, and why. Provide links to the according documentation. manpage references to dependencies are sufficient. This way, you enable even the most inexperienced admin to configure your package.

Use the most defensive, sane defaults.

Special note with regard to golang: Most package build tools strip the binaries contained in the package by default. Go does not support that, so take care here.

Be complete

There is nothing more annoying than an incomplete package.

Use syslog if at all possible and adhere to it's conventions. This way, your logs will be put into places where the system admin expects them and they are taken care of if outdated automatically,preventing your application to cause disks running full. If the system admin wants special treatment of your application's logs, he will configure it this way.

Do not rotate the logs via your application. It is the users choice on what to do with them (which might be relevant for their SLAs). Even if you make the way logs are rotated configurable, the admin has to learn how to configure it – which introduces unnecessary redundancy.

If you have to write to a log file, adhere to the target system's logging policy and provide a log rotate configuration file. You don't want you application be the cause of downtime just because the machine ran out of disk space, do you?

Use system tools instead of reinventing the wheel. If your application needs to do some maintenance, do not bother to use a scheduler inside your application. Write a specialized tool for the maintenance (monolithic applications are so '00s) and utilize cron. Specifically add according files to one of the /etc/cron* directories.

Provide proper init scripts! This way, the admin can use well known tools like systemctl to mange startup and shutdown of your application. It is pretty annoying when you have to su to a user or use sudo -u to call a shell script on startup. Even when this script is called @onboot, the deviation from standards is annoying. Just because a startup method works it does not mean that it should be used.

Bonus points for adding an SE-Linux profile!

It should go without saying, but I have seen misconfigured packages way to often, so: Test your packages! Start with a minimal install of the target OS, install your package and make sure it runs as expected. Check every configuration you provide.

If you plan to have a package put into the official Debian repositories, you should plan some time: the reason why Debian is as stable is that the requirements on packages are pretty tight, and even when you fulfill all requirements, you have to go all the way from testing via unstable to stable.

Be precise

Do not use existing users just because it is convenient. If you create a web application, do not reuse the "apache" or "www" user. Create a user dedicated to your package and add this user to the according groups.

Adhere to the principle of least necessary permissions. There is hardly a reason to have a binary world executable, much less world writable (which would be an extreme security hole). What you often see here on SO if an application does not run is the suggestion to set the permissions to [0]777, which allows every user to make modifications whatsoever on the file. Actually, there is almost no reason to make a binary writable to any user: root, who does the updates anyway, can always write anything. So the permissions should be 0550 for binaries. The principle also applies to data directories and such. Invest some time and effort here. You don't want your app to be the vector for an successful attack, do you? Even potential security risks tend to fire back on you and your reputation. What I tend to do is to set all data files to 0600 for files that need to be written by the system user of the application, 0400 for read-only files and 0500 for binaries. Then, I do a granular analysis what the group permissions should be. For example: A group might change the individual templates for a web application – but most likely not the directory structure of the resource directory subtree.

If you put effort into security, you will increase trust. Be aware that packages are often checked for their security impact prior to the decision wether they are adopted.

Adhere to the FHS(!!)! And even then: just because you can do anything under /opt/yourapplication, it is not always a good idea to do so. Rather install into /usr and /var respectively (assuming your application is not necessary during boot time).

If you have dependencies, define them. Do not merely assume that a package is present.

If you have a dependency for a local SMTP server, do not declare a dependency on postfix. Maybe the admin prefers sendmail (for whatever reason that may be). So define a dependency on mail-transport-agent (Debian) or mta (RH, iirc) instead.

Conclusion

This is what I expect of a good software – integrating nicely with existing software and to make it easy to install, maintain, configure and run it without having to learn redundant configuration. If I see a SELinux profile for a package, that really gives the vendor a bonus – unless the profile is extremely sloppy, it shows that the vendor takes security very serious.

huangapple
  • 本文由 发表于 2013年1月26日 20:45:39
  • 转载请务必保留本文链接:https://go.coder-hub.com/14537045.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定