Handling duplicate requests due to cold starts in a Cloud Run API backend

I am currently working with a Cloud Run API backend that processes incoming HTTP POST requests. When the system has been idle for some time, it goes into a "cold start". Upon receiving a new request during a cold start, the request is processed twice, resulting in duplicate actions. This duplication is not desirable and I am looking for a solution to prevent it.

Here is a simplified description of the system:

The API receives an HTTP POST request containing a message payload.
The API processes the request, which involves making a search request to an external service and returning the search result.
Despite this seemingly straightforward process, the logs show that during a cold start, the incoming request is processed twice.

Here's what I've tried so far:

1. Returning a response immediately: As soon as the POST request is received, I tried returning a response to the sender, and then proceeding to process the request in a background task. However, this did not solve the issue, as the request is still being processed twice.

2. Creating a temporary set and storing the message_ids from the post request so if I get the same message_id I can check with the set and ignore it.

So when I created the second mechanism I thought I solved it however it's not catching it.

Here is also a simplified version of my logs:

2023-06-15 21:00:58.675 BST
POST200660 B5 msUnknown https://my-cloud-run-app.com/message
2023-06-15 21:01:00.485 BST
INFO: Started server process [1]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit)
Default STARTUP TCP probe succeeded after 1 attempt for container "message-1" on port 8080.
Incoming request: POST [{"type":"inbound","message_id":1935379776,...}]
2023-06-15 21:01:25.539 BST
Message Processed.
...
2023-06-15 21:02:16.550 BST
POST200660 B7 msUnknown https://my-cloud-run-app.com/message
2023-06-15 21:02:16.560 BST
Incoming request: POST [{"type":"inbound","message_id":1935379776,...}]
2023-06-15 21:02:16.563 BST
Message Processed.
...
2023-06-15 21:17:31.301 BST
INFO: Shutting down
2023-06-15 21:17:31.403 BST
INFO: Waiting for application shutdown.
2023-06-15 21:17:31.403 BST
INFO: Application shutdown complete.
2023-06-15 21:17:31.404 BST
INFO: Finished server process [1]

As you can see same post request comes twice and I'm checking the message_id but still It makes it.

One particularly interesting things which might help is I'm using 2 different 3rd party API.
One is google_search.
The other is for sending the message.

Although it shows that it process the Message twice It only sends one request to google_search but twice to SMS API

It seems that the request is being sent twice because during the 1st request the instance that's going to receive it is not started. To prevent this you can enable idle instance for your service using the minimum instance setting. By default, instances are kept idle for 15 mins before shutting down after all requests are handled, but by enabling min-instance, you can keep one or more instances idle to minimize the impact of cold starts. You can refer to this document for more information about minimum instances and how to enable it.<br><br>

I solved the issue by adding a cpu-boost to my cold-start. This helped me to not to hit timeout so I didn't get duplicate messages.

You can add that to your gcloud deploy function as follows:

gcloud run deploy <YOUR SERVICE> --image etc... --cpu-boost