在App Engine和Firestore中的内存问题。

huangapple go评论91阅读模式
英文:

Memory issue with App Engine and Firestore

问题

我正在使用 Kotlin 和 Micronaut 开发一个访问 Firestore 数据库的微服务(MS)。当我在本地运行这个微服务时,由于它非常简单,只是读写 Firestore 数据,且数据量不大,实际上是像这样非常小的数据:

{
    "project": "DUMMY",
    "columns": [
        {
            "name": "TODO",
            "taskStatus": "TODO"
        },
        {
            "name": "IN_PROGRESS",
            "taskStatus": "IN_PROGRESS"
        },
        {
            "name": "DONE",
            "taskStatus": "DONE"
        }
    ],
    "tasks": {}
}

我在 App Engine 标准环境中以 F1 实例(256 MB,600 MHz)运行此微服务,使用以下 app.yaml 配置:

runtime: java11
instance_class: F1 # 256 MB 	600 MHz
entrypoint: java -Xmx200m -jar MY_JAR.jar
service: data-connector
env_variables:
  JAVA_TOOL_OPTIONS: "-Xmx230m"
  GAE_MEMORY_MB: 128M

automatic_scaling:
  max_instances: 1
  max_idle_instances: 1

我知道处理内存的所有这些属性都不是必需的,但我曾经尝试了很多解决方案,因为我的第一个错误信息是:

Exceeded soft memory limit of 256 MB with 263 MB after servicing 1 requests total. Consider setting a larger instance class in app.yaml.

下面的错误并没有通过 app.yaml 中的属性解决,但是每次调用返回该 JSON 时,我都会收到以下错误:

2020-04-10 12:09:15.953 CEST
While handling this request, the process that handled this request was found to be using too much memory and was terminated. This is likely to cause a new process to be used for the next request to your application. If you see this message frequently, you may have a memory leak in your application or may be using an instance with insufficient memory. Consider setting a larger instance class in app.yaml.

第一个请求总是耗时更长,我认为这是由于某些 Firestore 配置引起的,但问题是我无法使其工作,始终收到相同的错误。您是否知道我可能做错了什么,或者需要什么来修复这个问题?

英文:

I'm developing a MS with Kotlin and Micronaut which access a Firestore database. When I run this MS locally I can make it work with 128M because it's very simple just read and write data to Firestore, and not big amounts of data, really small data like this:

{
    "project": "DUMMY",
    "columns": [
        {
            "name": "TODO",
            "taskStatus": "TODO"
        },
        {
            "name": "IN_PROGRESS",
            "taskStatus": "IN_PROGRESS"
        },
        {
            "name": "DONE",
            "taskStatus": "DONE"
        }
    ],
    "tasks": {}
}

I'm running this in App Engine Standard in a F1 instance (256 MB 600 MHz) with this properties in my app.yaml

runtime: java11
instance_class: F1 # 256 MB 	600 MHz
entrypoint: java -Xmx200m -jar MY_JAR.jar
service: data-connector
env_variables:
  JAVA_TOOL_OPTIONS: "-Xmx230m"
  GAE_MEMORY_MB: 128M

automatic_scaling:
  max_instances: 1
  max_idle_instances: 1

I know all that properties for handling memory are not necessary but I was desperate trying to make this work and just tried a lot of solutions because my first error message was:

Exceeded soft memory limit of 256 MB with 263 MB after servicing 1 requests total. Consider setting a larger instance class in app.yaml.

The error below is not fixed with the properties in the app.yaml, but now everytime I make a call to return that JSON I get this error

2020-04-10 12:09:15.953 CEST
While handling this request, the process that handled this request was found to be using too much memory and was terminated. This is likely to cause a new process to be used for the next request to your application. If you see this message frequently, you may have a memory leak in your application or may be using an instance with insufficient memory. Consider setting a larger instance class in app.yaml.

It always last longer in the first request, I think due to some Firestore configuration, but the thing is that I cannot make that work, always getting the same error.

Do you have any idea what I could be doing wrong or what I need to fix this?

答案1

得分: 2

问题是我尝试使用了一个非常小的实例来运行一个简单的应用程序,但即使如此,我仍然需要更多的内存。

好的,一个朋友帮助了我解决了这个问题。我之前使用了一个非常小的实例,即使我没有遇到内存限制的错误,但问题仍然出现在内存方面。

将我的实例升级为 F2(512 MB 1.2 GHz)解决了问题,并且使用 siege 对我的应用进行测试,结果表现非常出色:

事务数:                  5012 次点击
可用性:                100.00%
经过时间:             59.47 秒
数据传输量:         0.45 MB
响应时间:             0.30 秒
事务速率:            84.28 事务/秒
吞吐量:                0.01 MB/秒
并发性:               24.95
成功事务:             3946 次
失败事务:             0 次
最长事务:             1.08 秒
最短事务:             0.09 秒

我的系统运维朋友告诉我,这些实例更适用于Python脚本代码之类的用途,而不适用于JVM REST服务器。

英文:

TL;DR The problem was I tried to used a very small instance for a simple application, but even with that I needed more memory.

Ok, a friend helped me with this. I was using a very small instance and even when I didn't get the error of memory limit it was a memory problem.

Updating my instance to a F2 (512 MB 1.2 GHz) solved the problem and testing my app with siege resulted in a very nice performance:

Transactions:		        5012 hits
Availability:		      100.00 %
Elapsed time:		       59.47 secs
Data transferred:	        0.45 MB
Response time:		        0.30 secs
Transaction rate:	       84.28 trans/sec
Throughput:		        0.01 MB/sec
Concurrency:		       24.95
Successful transactions:        3946
Failed transactions:	           0
Longest transaction:	        1.08
Shortest transaction:	        0.09

My sysops friends tells me that this instances are more for python scripting code and things like that, not JVM REST servers.

huangapple
  • 本文由 发表于 2020年4月10日 18:24:20
  • 转载请务必保留本文链接:https://go.coder-hub.com/61138315.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定