英文:
cloud function instance seems to upgrade automatically the memory if needed (gen 2)
问题
I'm using cloud function (gen 2). The cloud function is running in python 3.10. During the execution of the cloud function there are a lot of heavy files that are cached in memory.
It happens quite often that there is a message error of this type:
Memory limit of 1907 MiB exceeded with 2059 MiB used. Consider increasing the memory limit, see https://cloud.google.com/functions/docs/configuring/memory
Most of the time, after this message the cloud function is still running (but sometimes the cloud function fails few seconds after).
Does it mean there is an automatic upgrade of the memory ?
I put some logs with the memory used during the execution with psutil.virtual_memory().
So here is an example:
- memory log before memory alert:
INFO:svmem(total=2147483648, available=498733056, percent=76.8, used=1433911296, free=498733056, active=1547333632, inactive=101404672, buffers=0, cached=214839296, shared=12029952, slab=0)
- message of error:
Memory limit of 1907 MiB exceeded with 2059 MiB used. Consider increasing the memory limit, see https://cloud.google.com/functions/docs/configuring/memory
- after the memory error:
INFO:svmem(total=3221225472, available=1300992000, percent=59.6, used=1705369600, free=1300992000, active=1818804224, inactive=101416960, buffers=0, cached=214863872, shared=12029952, slab=0)
What I see here is the total memory has been increased more than 1 GB. Am I correct, is the library also accurate ?
Is there really an upgrade of the memory ?
Have you observed that also ?
英文:
I'm using cloud function (gen 2). The cloud function is running in python 3.10. During the execution of the cloud function there are a lot of heavy files that are cached in memory.
It happens quite often that there is a message error of this type:
Memory limit of 1907 MiB exceeded with 2059 MiB used. Consider increasing the memory limit, see https://cloud.google.com/functions/docs/configuring/memory
Most of the time, after this message the cloud function is still running (but sometimes the cloud function fails few seconds after).
Does it mean there is an automatic upgrade of the memory ?
I put some logs with the memory used during the execution with psutil.virtual_memory().
So here is an example:
- memory log before memory alert:
INFO:svmem(total=2147483648, available=498733056, percent=76.8, used=1433911296, free=498733056, active=1547333632, inactive=101404672, buffers=0, cached=214839296, shared=12029952, slab=0)
- message of error:
Memory limit of 1907 MiB exceeded with 2059 MiB used. Consider increasing the memory limit, see https://cloud.google.com/functions/docs/configuring/memory
- after the memory error:
INFO:svmem(total=3221225472, available=1300992000, percent=59.6, used=1705369600, free=1300992000, active=1818804224, inactive=101416960, buffers=0, cached=214863872, shared=12029952, slab=0)
What I see here is the total memory has been increased more than 1 GB. Am I correct, is the library also accurate ?
Is there really an upgrade of the memory ?
Have you observed that also ?
答案1
得分: 1
不，这是不正确的。云函数只使用允许的内存。当出现内存错误消息时，意味着您的云函数实例已崩溃，新实例正在生成。
因为实例是全新的，所有内存都被交换，您将拥有更多的可用空间。
关于"可用内存"，您不能依赖于您自己的分析工具。环境是虚拟化的并且被沙箱化了。即使您的工具总计显示3GB，如果2GB内存已满，您仍然会出现相同的崩溃问题。
英文:
No, it's not correct. Cloud Functions use only the memory allowed. When you have a memory error message, it means your Cloud Functions instance has crashed, and a new instance is spawned.
Because the instance is fresh, all the memory is swapped, and you have more free space.
About the "available memory", you can't rely on your own analysis tool. The environment is virtualized and sandboxed. You will have the same crash issue because of 2gb of memory are full even if the total with your tool displays 3Gb.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论