在 lambda 函数内部调用 API 时,当有大量请求时会失败。

huangapple go评论62阅读模式
英文:

Calling API inside a lambda function fails when there is lot of requests

问题

在文档上传过程中,有一个用于更新数据库的API端点。该端点通过AWS API网关公开,并已指向AWS SQS以处理请求。该队列触发了一个Lambda函数,并调用一个API方法来在Lambda内部更新数据库。当有大量请求(15-20个文档上传请求)时,该Lambda函数会失败,抛出'响应状态码不表示成功:400'(400错误)。当请求较少时,它正常工作。可能的原因是什么?

Lambda 代码。

public async Task FunctionHandler(SQSEvent evnt, ILambdaContext context)
{
    try
    {
        HttpClient client = new();

        foreach (var message in evnt.Records)
        {
            await ProcessMessageAsync(message, context, client);
        }
    }
    catch (Exception ex)
    {
        throw new UploaderException(ex.Message);
    }
}

//Private method
private async Task ProcessMessageAsync(SQSEvent.SQSMessage message, ILambdaContext context, HttpClient client) {

    string item = string.Empty;
    string methodName = string.Empty;
    string httpMethod = string.Empty;

    foreach (var attribute in message.MessageAttributes)
    {
        if (attribute.Key.ToLower() == "item")
        {
            item = attribute.Value.StringValue;
        }
        if (attribute.Key.ToLower() == "methodname")
        {
            methodName = attribute.Value.StringValue;
        }
        if (attribute.Key.ToLower() == "httpmethod")
        {
            httpMethod = attribute.Value.StringValue;
        }

        if (attribute.Key.ToLower() != "methodname" || attribute.Key.ToLower() != "httpmethod")
        {
            client.DefaultRequestHeaders.Add(attribute.Key, attribute.Value.StringValue);
        }
    }

    if (string.IsNullOrWhiteSpace(item))
    {
        throw new UploaderException("无法找到项目");
    }

    string baseUrl = Environment.GetEnvironmentVariable(item.ToUpper());
    var content = new StringContent(message.Body, System.Text.Encoding.UTF8, "application/json");

    context.Logger.LogLine($"URL: {baseUrl}{methodName}");

    HttpResponseMessage response;
    if (httpMethod.ToUpper() == "POST")
    {
        response = await client.PostAsync($"{baseUrl}{methodName}", content);
    }
    else
    {
        response = await client.PutAsync($"{baseUrl}{methodName}", content);
    }
    
    response.EnsureSuccessStatusCode();
    context.Logger.LogLine("文档上传成功");

    await Task.CompletedTask;
}

希望这有助于解决问题。

英文:

In a document upload process, there is an API endpoint which use to update the database. That endpoint exposes through a Aws API gateway and it has been pointed to a AWS SQS to process requests. That Queue triggers a lambda function and call an API method to update the database inside the lambda. When there is large number of requests,(15-20 document upload requests) that lambda function fails throwing 'Response status code does not indicate success : 400' (400 error). Its working normally when there is small number of requests. What would be the reason?

Lambda Code.

 public async Task FunctionHandler(SQSEvent evnt, ILambdaContext context)
    {
        try
        {
            HttpClient client = new();

            foreach (var message in evnt.Records)
            {
                await ProcessMessageAsync(message, context, client);
            }
        }
        catch (Exception ex)
        {
            throw new UploaderException(ex.Message);
        }
    }

//Private method
private async Task ProcessMessageAsync(SQSEvent.SQSMessage message, ILambdaContext context, HttpClient client) {
 
        string item = string.Empty;
        string methodName = string.Empty;
        string httpMethod = string.Empty;

        foreach (var attribute in message.MessageAttributes)
        {
            if (attribute.Key.ToLower() == "item")
            {
                item = attribute.Value.StringValue;
            }
            if (attribute.Key.ToLower() == "methodname")
            {
                methodName = attribute.Value.StringValue;
            }
            if (attribute.Key.ToLower() == "httpmethod")
            {
                httpMethod = attribute.Value.StringValue;
            }

            if (attribute.Key.ToLower() != "methodname" || attribute.Key.ToLower() != "httpmethod")
            {
                client.DefaultRequestHeaders.Add(attribute.Key, attribute.Value.StringValue);
            }
        }

        if (string.IsNullOrWhiteSpace(item))
        {
            throw new UploaderException("Could not find item");
        }

        string baseUrl = Environment.GetEnvironmentVariable(item.ToUpper());
        var content = new StringContent(message.Body, System.Text.Encoding.UTF8, "application/json");

        context.Logger.LogLine($"URL: {baseUrl}{methodName}");

        HttpResponseMessage response;
        if (httpMethod.ToUpper() == "POST")
        {
            response = await client.PostAsync($"{baseUrl}{methodName}", content);
        }
        else
        {
            response = await client.PutAsync($"{baseUrl}{methodName}", content);
        }
        
        response.EnsureSuccessStatusCode();
        context.Logger.LogLine("Document upload success");

        await Task.CompletedTask;
 }

答案1

得分: 0

有许多不同的过程在同一时间修改相同的客户端(client.DefaultRequestHeaders.Add(..))- 这可能是一个问题。

我建议为每个消息/HTTP请求创建一个单独的标头对象,完全不依赖默认标头(除非它们被所有请求共享)。

英文:

There are many different processes modifying the same client at the same time (client.DefaultRequestHeaders.Add(..)) - that could be an issue.

I would suggest creating a separate headers-object per message/HTTP request, and not rely on the default headers at all (unless they are shared across all requests).

huangapple
  • 本文由 发表于 2023年2月6日 16:05:01
  • 转载请务必保留本文链接:https://go.coder-hub.com/75358717.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定