如何锁定 Google Cloud 函数的并发性?

huangapple go评论59阅读模式
英文:

How to lock a google cloud function concurrency?

问题

我想在云函数中实现较大的并发性,因此我正在使用云函数 v2。但是,在其中的一个步骤中,我只希望有1个进程能够执行它,而其他进程需要等待。

假设我有以下逻辑用于我的函数:

if (notGenerated()) {
    generateSomething()
}
step2()
step3()
...

如果它尚未生成,然后同时有2个或更多的请求到达,我应该如何阻止它,以便只有1个进程/请求可以执行生成操作?

在传统方式中,我会使用Redis作为内存缓存,以确保只有1个进程可以通过。其他进程可以等待或进行重试。

但是,如果我需要将相同的方式实现到我的云函数中,我需要使用memorystore for Redisserverless vps connector,这不便宜。特别是在memorystore中,我将被收费24小时,而我可能只使用它3-4小时。

你如何在云函数 v2 中检查/使用内存缓存?
我能否在云函数 v2 中使用类似node-cache的东西?
云函数 v2 中的并发性是否共享相同的内存?如果云函数扩展我的函数,它是否仍然使用相同的内存?

英文:

i want to have a big concurrencies with the cloud function, so I'm using the cloud function v2. But, in one of the steps, I want only 1 process can do it, while other processes need to wait for it.

Let's say that I have below logic for my function

if (notGenerated()) {
    generateSomething()
}
step2()
step3()
...

If its 'not generated yet', and then there are 2 or more request coming at the same time, how can I block it so that only 1 process/request can do the generation?

In the conventional way, I would use redis as memory cache, to make sure that only 1 process can go through. The other processes could wait, or do any retries.

But if I need to implement the same style to my cloud function, I need to use the memorystore for redis and serverless vps connector, which is not cheap. Especially that in the memorystore, I would be charged for 24 hours, while I might only use it for 3-4 hours only.

How do you check / use the memory cache in cloud function v2?
Could I use something like node-cache in cloud function v2?
Does the concurrency in cloud function v2 share the same memory? If the cloud functions scale my functions, does it still using the same memory?

答案1

得分: 1

根据我的理解,您的云函数不需要共享,也可能不会共享相同的内存,或者通常不会共享相同的底层物理硬件。

更重要的是,在我看来,建议永远不要这样思考您的函数:它们是一个独立的计算单元,应该考虑到自己的物理资源来执行其工作。

如果共享资源不是一个选项,您可以尝试实现某种分布式锁定/互斥机制。

我同意您的观点,对于这个目的,Redis是一个非常合适的用例,尽管为了避免与memorystore相关的成本,您可以尝试使用其他替代方法。

我不确定它是否符合您的要求,但是,根据基于文件系统的文件锁定的想法,您可以尝试使用例如GCS桶并根据需要创建blob。不幸的是,无法绝对保证“锁定”blob的存在检查和创建操作将由同一函数执行,可能会发生竞争条件,但这可能是一个合适的解决方案 - 尽管抱歉,因为即使在我的眼中,它似乎也不是一个理想的解决方案。

虽然我理解您的要求,请考虑修改您的函数以避免这种类型的相互依赖,我认为在某种程度上,这违反了它们的概念和无服务器方法:尝试将它们构想为idempotent服务。

英文:

AFAIK your cloud functions don't need to share and probably will not share the same memory or, in general, the same underlying physical hardware between them.

More important, in my opinion, it is advisable to never think in your functions in such way: they are an isolated computed unit that should perform their job taking into account their own physical resources.

If sharing resources is not an option you could try implementing some type of distributed locking/mutex mechanism.

I agree with you that for this purpose Redis is a very suitable use case, although to avoid the costs associated with memorystore you could trying using another alternative approaches.

I am not aware that it will fit your requirements but, extrapolating the idea of filesystem based file locking, you could try using for example a GCS bucket and create blobs on demand. Unfortunately, there is no absolutely guarantee that the operations of checking the existence of the "lock" blob and creating them will be performed by the same function, a race condition could happen, but it may be a suitable solution - although sorry, because even on my own eyes, it doesn't seem an ideal one.

Although I understand your requirements, please, consider modifying your functions to avoid this type of interdependencies, I think that in a certain way it is against their concept and the serverless approach: try conceiving them as idempotent services instead.

huangapple
  • 本文由 发表于 2023年7月17日 13:19:27
  • 转载请务必保留本文链接:https://go.coder-hub.com/76701656.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定