英文:
How can I externally rate-limit a golang http upload?
问题
我有一个使用Golang编写的实用程序，通过HTTPS将大量数据上传到OpenStack对象存储。我在Ubuntu Linux上运行它，并且希望确保其最大上传带宽不超过2,500KB/s，最好不影响同一系统上的其他用户（即不减慢以太网接口）。
如何做到这一点，最好不修改我的源代码？到目前为止，我尝试了几种方法：
- 流量整形： 我一直无法找到适当的网络整形技术来影响单个进程，尽管我知道可以使用
iptables和tc来实现。如果您能指导我如何做到这一点，我将不胜感激。我的网络背景不是很广泛。 - 流速限制： 我尝试使用Go中的
flowrate包来限制我对上传的写入速率。但是，这似乎没有任何明显的效果。我认为这不起作用是因为github.com/ncw/swift.ObjectCreateFile.Write()方法不会在接收到数据时立即上传数据，而是在关闭时上传。不过，我可能对此有所误解。 - trickle： 我尝试了
trickle命令，但它与Golang可执行文件不兼容（参见此问题）。
英文:
I have a golang utility that uploads a lot of data via https to OpenStack Object Storage. I'm running it on Ubuntu Linux, and I would like to ensure that its maximum upload bandwidth does not exceed 2,500KB/s, preferably without affecting other users on the same system (i.e. slowing down the ethernet interface).
How can I do this, preferably without altering my source code? I've tried a few approaches so far:
- Traffic Shaping: I've been unable to figure out the appropriate network shaping techniques to affect a single process, though I know that should be possible with
iptablesandtc. Any guidance on how to do this would be appreciated. My networking background isn't very extensive. - Flowrate Limiting: I tried using the
flowratepackage in go to rate-limit my writes to the upload. This didn't have any discernable effect. I think this doesn't work because thegithub.com/ncw/swift.ObjectCreateFile.Write()method doesn't upload the data as it is recieved, but rather when it is closed. I could be mistaken about this though. - trickle: I tried the
tricklecommand, but it isn't compatible with golang executables (see this question)
答案1
得分: 2
原来可以将Golang进程封装在Docker容器中，然后使用tc工具在Docker容器内部限制容器的网络接口速率。可以参考这个答案中的示例Dockerfile来实现这一点。
英文:
It turns out that you can wrap the Golang process in a Docker container and then rate-limit the Docker container's network interface (from inside of the container) with the tc utility. See this answer for an example Dockerfile to do just that.
答案2
得分: 1
如果你可以控制实际运行实用程序的方式，你可以使用trickle。
sudo apt-get install trickle #因为你使用的是Ubuntu
trickle -u（上传限制，以KB/s为单位）-d（下载限制，以KB/s为单位）可执行文件
这里有一些关于trickle的额外文档：https://wiki.archlinux.org/index.php/Trickle
英文:
If you can control how you actually run the utility, you can use trickle
sudo apt-get install trickle #since you're on Ubuntu
trickle -u (upload limit in KB/s) -d (download limit in KB/s) executable
Here is some additional documentation about it https://wiki.archlinux.org/index.php/Trickle
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论