英文:
Very long latence only for one of my module
问题
我们的问题始于2015年9月2日17:46(巴黎时间),目前仍存在问题。
我们有两个应用程序:一个用于测试和开发,另一个用于生产。问题出现在生产应用程序中。
-
在过去的5天里,我们没有修改生产应用程序的代码,但突然在上周五下午出现了非常长的延迟。之前的延迟是100~300毫秒,但从那一刻开始,延迟变成了30秒,并且对于所有请求都是如此!
-
我们在测试应用程序中使用相同的代码,但对于这个应用程序一切正常。
-
在生产应用程序中有几个模块,只有一个模块出现了这个问题,其他所有模块都正常工作。
-
在周末期间,我们进行了一些测试,但仍然找不到原因。例如,这段代码:
package app import ( "net/http" ) func init() { http.HandleFunc("/", HandlerHeartBeat) } func HandlerHeartBeat(w http.ResponseWriter, r *http.Request) { w.WriteHeader(202) }
当我将这段代码部署到生产应用程序中的有问题的模块时,延迟为2秒。然后我尝试将其部署到有问题模块的另一个版本中,延迟始终非常长。但是,如果将此代码部署到生产应用程序中的其他模块(或只是创建一个新模块),我会得到正常的延迟(约50毫秒)。
-
我们的配置文件非常简单:
application: APP_NAME module: MODULE_NAME version: 1 runtime: go api_version: go1 handlers: - url: /.* script: _go_app
我不知道是否有其他人遇到了和我一样的问题。谢谢。
英文:
Our problem began 02/Sep/2015 17:46 (Paris time), and there is still issus at present.
We have 2 applications: one for test and development and the other one for the production. The problem is in the production application.
-
There had been 5 days that we didn't modify the code in production application, but suddenly the latence became very very long the last friday afternoon. The latence was 100~300ms before, but became 30s from from that moment on, and for all requests!
-
We use the same code for the test application, but all work well for this application.
-
We have several modules in the production application, and there is only one module has this problem, all others mudules work well.
-
During the weekend, we make some tests, but still can not find why.
For exemple, the code here:package app import ( "net/http" ) func init() { http.HandleFunc("/", HandlerHeartBeat) } func HandlerHeartBeat(w http.ResponseWriter, r *http.Request) { w.WriteHeader(202) }
When I deploy this code to the problematic module in the production application, the latence is 2s. Then I try to deploy it to another version in the problematic module, the latence is always very long.
But if deploy this code to other module in the production application (or just create a new module), I get the normal latence(~50ms).
-
Our configuration file is very simple:
application: APP_NAME module: MODULE_NAME version: 1 runtime: go api_version: go1 handlers: - url: /.* script: _go_app
I don't know if there is other people meet the same problem like me. Thank you
答案1
得分: 1
很可能Google基础设施中存在问题,导致了有问题的模块。
可能是它为一个模块计算了一些哈希码,并尝试在同一台机器上实例化相同模块的实例(如果可能的话),以便将应用代码缓存在(虚拟)机器级别。如果该机器存在问题(例如硬件问题),则会导致应用的延迟增加。
我建议:
- 部署代码到新的模块并使用它(如果可能),同时保留旧模块进行故障排除。
- 缩小使用案例(例如基本功能如ping),并向Google提出问题。
英文:
It's very likely there is an issue with Google infrastructure that serve the problematic module.
Probably it calculate some hash code for a module and try to instantiate an instance of the same module on the same machine if possible so app code is cached on the (virtual) machine level if possible. If that machine has issues (for example with hardware) it translates to increased latency for your app.
I would advise:
- Deploy code to new module and use it (if possible) while keeping old for troubleshooting.
- Narrow use cases (basic functionality like ping) & raise an issue with Google.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论