Firebase HTTPS Functions - Responding with a 200 status code Before Completing Background Work

We are triggering a cloud function with a HTTPS request to:

1. Respond quickly (<10s) to the caller with a 200 Response to avoid more retries from our caller.
2. THEN make two other POST (time-sensitive) requests before ending the function.

In general, what are the best solutions to deal with this problem? We have seen:

1. PubSub to have subscribers complete the POST requests async after sending off our 200 response.
2. Enqueue Cloud Tasks to a task queue. We are not sure if these are affected by the HTTPS termination like we have seen with our attempt approaches.
3. Using an Event Based Trigger like Cloud Firestore to write to a document with some id which triggers an onCreate event to finish out the POST requests.

Other relevant details are that we have small traffic now as a smaller app (much less than 1000 QPS which may rule out Pub Sub) and would like to target creating the background task in <1-2 seconds to still have a good user experience (messaging app).

Originally, we had our code like:

exports.example = functions
  .https
  .onRequest(async (request, response) => {
    const authHeader = request.get("authorization");
    const {statusCode, httpsResponseMessage} =
      verifyAuthHeader(authHeader, token);
    // Send 200 response to our caller within 10 seconds to avoid more retries.
    response.status(statusCode).send(
     {
      message: httpsResponseMessage,
     }
    );
    // do other POST requests
  }

We also tried:

exports.example = functions
  .https
  .onRequest(async (request, response) => {
    const authHeader = request.get("authorization");
    const {statusCode, httpsResponseMessage} =
      verifyAuthHeader(authHeader, token);
    // Send 200 response to our caller within 10 seconds to avoid more retries.
    response.write(
      JSON.stringify(
        {
          message: httpsResponseMessage
        }
      )
    );
    // do other POST requests
    res.end()
  }

But noticed unpredictable behavior with our code AFTER response.status(statusCode).send() as noted in the tips & tricks section of Firebase docs and also that sending a partial result with response.write does not avoid future retries from the caller.

You need to be very careful with HTTP functions and ensure that all your work is complete before you initiate the response. This is documented in The Node.js Runtime:

> When working with asynchronous tasks that involve callbacks or Promise objects, you must explicitly inform the runtime that your function has finished executing these tasks. You can do this in several different ways, as shown in the samples below. The key is that your code must wait for the asynchronous task or Promise to complete before returning; otherwise the asynchronous component of your function may be terminated before it completes.
>
>
> // OK: await-ing a Promise before sending an HTTP response
> await Promise.resolve();
>
> // WRONG: HTTP functions should send an
> // HTTP response instead of returning.
> return Promise.resolve();
>
> // HTTP functions should signal termination by returning an HTTP response.
> // This should not be done until all background tasks are complete.
> res.send(200);
> res.end();
>
> // WRONG: this may not execute since an
> // HTTP response has already been sent.
> return Promise.resolve();
>

In practice, this means that your two examples will never work as expected because you are initiating the response before you complete the work as described by // do other POST requests.

From your description it sounds like:

1. You need to respond quickly to the caller
2. You need to fire two other functions when calls are made to this function
3. But the responses from the other two functions are not used in the response to the original caller

If that is the case then I recommend that you use Pub/Sub-triggered background functions and structure this function as follows:

const { PubSub } = require('@google-cloud/pubsub')

const pubSubTopic1 = new PubSub('your-project-id').topic('the-first-topic-name')
const pubSubTopic2 = new PubSub('your-project-id').topic('the-second-topic-name')

exports.example = functions
  .https
  .onRequest(async (request, response) => {
    const authHeader = request.get("authorization");
    const {statusCode, httpsResponseMessage} = verifyAuthHeader(authHeader, token);

    // Publish messages to the Pub/Sub topics to trigger those functions
    await Promise.all([
      pubSubTopic1.publishMessage('message-1'),
      pubSubTopic2.publishMessage('message-2')
    ]);

    // Send 200 response to our caller within 10 seconds to avoid more retries.
    response.status(statusCode).send(
     {
      message: httpsResponseMessage,
     }
    );

    // Do nothing after calling .send() because the function has terminated
  }

Pub/Sub is fast with low latency and given the amount of concurrency you're handling your functions should remain in a warm state and fire quickly.

Don't use Firestore for this feature; you'll run into cost issues by creating a document and firing a function, and you'll run into cost issues by creating a huge number of documents that you'll either have to retain or delete, and every create/delete operation incurs cost. And regardless, the way Firestore document triggers work is by sending a Pub/Sub message, so introducing a document write in your process is only going to add latency to the overall process.

Example PubSub function

As mentioned in the comments below using firebase-functions is heavy and clunky so here is an example PubSub-triggered function that does not require it.

Assume that this function expects to receive a JSON string as the message sent to the PubSub topic. For example, in the HTTP function above you might publish a message like this:

const message1 = {
  "foo": "bar",
  "bar": "baz"
}
let statusCode = 202 // Accepted

try {
  await pubSubTopic1.publishMessage({ json: message1 })
} catch (error) {
  statusCode = 502 // Bad Gateway
} finally {
  res.status(statusCode).send({ message: httpsResponseMessage })
}

Your receiving PubSub function can be as small and simple as this:

export const examplePubSubFunction = message => {
  const data = JSON.parse(Buffer.from(message.data, 'base64').toString())
  console.log(data.foo) // prints "bar"
  console.log(data.bar) // prints "baz"
}

You can then deploy the function with:

gcloud functions deploy examplePubSubFunction --runtime=nodejs18 --trigger-topic=the-first-topic-name