英文:
Can kubernetes resource request/limit definition slow down pods?
问题
我在一个Kubernetes集群上运行了相同应用的3个部署。最近，我开始为其中一个部署设置资源请求和限制。
resources:
limits:
cpu: 350m
memory: 225Mi
requests:
cpu: 250m
memory: 150Mi
设置完这些后，受影响的Pod的计算时间明显增加，与2个未更改的部署相比，这与我理解的Kubernetes文档不符合。
运行kubectl top pods允许我确认我的Pod正在以请求的资源量或以下的资源上运行。在可视化计算时间（使用Prometheus+Grafana）时，很明显其中一个部署明显较慢：
两个部署大约在60毫秒左右，而一个部署大约在120毫秒左右
因为这是我做出的唯一更改，我不明白为什么会有性能下降。我是否漏掉了什么？
编辑
删除CPU的limit但保留request可以恢复Pod的性能至预期水平。请记住，这些Pod正在以CPU请求级别运行（大约250mCPU），低于限制的100mCPU。
额外信息：这些Pod正在运行一个NodeJS应用。
英文:
I am running 3 deployments of the same app on a kubernetes cluster. I recently got around to setting resource requests and limits for one of the deployments.
resources:
limits:
cpu: 350m
memory: 225Mi
requests:
cpu: 250m
memory: 150Mi
After setting these, the affected pods have a much higher computation time compared to the 2 unchanged deployments, which does not make sense regarding kubernetes documentation as I understand it.
Running kubectl top pods allows me to confirm my pods are running at or below requested resources. When visualizing computation time (Prometheus+Grafana), it is clear one of the deployments is significantly slower:
Two deployments at ~ 60ms and one at ~ 120ms
As this is the only change I have made, I don't understand why there should be any performance degradation. Am I missing something?
Edit
Removing the cpu limit but keeping the request brings pod performance back to what it's supposed to be. Keeping in mind that these pods are running at cpu request level (around 250mCPU) which is 100mCPU below the limit.
Additional information: these pods are running a NodeJS app.
答案1
得分: 3
阅读此链接，我理解如果一个 Pod 成功启动，容器将获得所请求的资源数量，因此调度是基于 YAML 中的请求字段而不是限制字段进行的，但 Pod 及其容器不允许超过 YAML 中指定的限制。
如果 Pod 超过其限制，将会被限制。如果未指定限制，则 Pod 可以在可用时使用额外的 CPU 资源。
请参考以下链接以获取完整信息：
https://github.com/kubernetes/community/blob/master/contributors/design-proposals/node/resource-qos.md#compressible-resource-guarantees
英文:
Reading this link i understand that if a pod is successfully started, the container is guaranteed the amount of resources requested so scheduling is based on requests field in yaml and not the limit field but the pods and its containers will not be allowed to exceed the specified limit on yaml.
Pods will be throttled if they exceed their limit. If limit is unspecified, then the pods can use excess CPU when available.
Refer the link for complete read
https://github.com/kubernetes/community/blob/master/contributors/design-proposals/node/resource-qos.md#compressible-resource-guarantees
答案2
得分: 1
Kubernetes CPU limits might not work as one would assume. I suggest to watch this presentation starting at 13:38.
A solution to the negative effect of CPU limits in k8s might be to set a different CFS quota value. By default it's set to 100ms, a better value might be 5ms. There is also an issue about this.
英文:
Kubernetes CPU limits might not work as one would assume. I suggest to watch this presentation starting at 13:38.
A solution to the negative effect of CPU limits in k8s might be to set a different CFS quota value. By default it's set to 100ms, a better value might be 5ms. There is also an issue about this.
答案3
得分: 0
以下是已翻译的内容：
不具有任何请求和限制的Pod可能会无限制地使用节点资源，因此它可以更快。
具有限制的Pod将受到限制，因此它们可能会变慢。
请检查两个部署的资源消耗指标。
英文:
The pods which are not having any requests and limits might be using the node's resources with out any restriction. so it can be faster.
The pods which are having limits will be limited so they can be slow.
Please check resource consumption metrics of both deployments.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论