在 lambda 函数内部调用 API 时，当有大量请求时会失败。

huangapple
go评论93阅读模式
英文:

Calling API inside a lambda function fails when there is lot of requests

问题

在文档上传过程中，有一个用于更新数据库的API端点。该端点通过AWS API网关公开，并已指向AWS SQS以处理请求。该队列触发了一个Lambda函数，并调用一个API方法来在Lambda内部更新数据库。当有大量请求（15-20个文档上传请求）时，该Lambda函数会失败，抛出'响应状态码不表示成功：400'（400错误）。当请求较少时，它正常工作。可能的原因是什么？

Lambda 代码。

  1. public async Task FunctionHandler(SQSEvent evnt, ILambdaContext context)
  2. {
  3.     try
  4.     {
  5.         HttpClient client = new();
  7.         foreach (var message in evnt.Records)
  8.         {
  9.             await ProcessMessageAsync(message, context, client);
  10.         }
  11.     }
  12.     catch (Exception ex)
  13.     {
  14.         throw new UploaderException(ex.Message);
  15.     }
  16. }
  18. //Private method
  19. private async Task ProcessMessageAsync(SQSEvent.SQSMessage message, ILambdaContext context, HttpClient client) {
  21.     string item = string.Empty;
  22.     string methodName = string.Empty;
  23.     string httpMethod = string.Empty;
  25.     foreach (var attribute in message.MessageAttributes)
  26.     {
  27.         if (attribute.Key.ToLower() == "item")
  28.         {
  29.             item = attribute.Value.StringValue;
  30.         }
  31.         if (attribute.Key.ToLower() == "methodname")
  32.         {
  33.             methodName = attribute.Value.StringValue;
  34.         }
  35.         if (attribute.Key.ToLower() == "httpmethod")
  36.         {
  37.             httpMethod = attribute.Value.StringValue;
  38.         }
  40.         if (attribute.Key.ToLower() != "methodname" || attribute.Key.ToLower() != "httpmethod")
  41.         {
  42.             client.DefaultRequestHeaders.Add(attribute.Key, attribute.Value.StringValue);
  43.         }
  44.     }
  46.     if (string.IsNullOrWhiteSpace(item))
  47.     {
  48.         throw new UploaderException("无法找到项目");
  49.     }
  51.     string baseUrl = Environment.GetEnvironmentVariable(item.ToUpper());
  52.     var content = new StringContent(message.Body, System.Text.Encoding.UTF8, "application/json");
  54.     context.Logger.LogLine($"URL: {baseUrl}{methodName}");
  56.     HttpResponseMessage response;
  57.     if (httpMethod.ToUpper() == "POST")
  58.     {
  59.         response = await client.PostAsync($"{baseUrl}{methodName}", content);
  60.     }
  61.     else
  62.     {
  63.         response = await client.PutAsync($"{baseUrl}{methodName}", content);
  64.     }
  65.     
  66.     response.EnsureSuccessStatusCode();
  67.     context.Logger.LogLine("文档上传成功");
  69.     await Task.CompletedTask;
  70. }

希望这有助于解决问题。

英文:

In a document upload process, there is an API endpoint which use to update the database. That endpoint exposes through a Aws API gateway and it has been pointed to a AWS SQS to process requests. That Queue triggers a lambda function and call an API method to update the database inside the lambda. When there is large number of requests,(15-20 document upload requests) that lambda function fails throwing 'Response status code does not indicate success : 400' (400 error). Its working normally when there is small number of requests. What would be the reason?

Lambda Code.

  1.  public async Task FunctionHandler(SQSEvent evnt, ILambdaContext context)
  2.     {
  3.         try
  4.         {
  5.             HttpClient client = new();
  7.             foreach (var message in evnt.Records)
  8.             {
  9.                 await ProcessMessageAsync(message, context, client);
  10.             }
  11.         }
  12.         catch (Exception ex)
  13.         {
  14.             throw new UploaderException(ex.Message);
  15.         }
  16.     }
  18. //Private method
  19. private async Task ProcessMessageAsync(SQSEvent.SQSMessage message, ILambdaContext context, HttpClient client) {
  20.  
  21.         string item = string.Empty;
  22.         string methodName = string.Empty;
  23.         string httpMethod = string.Empty;
  25.         foreach (var attribute in message.MessageAttributes)
  26.         {
  27.             if (attribute.Key.ToLower() == "item")
  28.             {
  29.                 item = attribute.Value.StringValue;
  30.             }
  31.             if (attribute.Key.ToLower() == "methodname")
  32.             {
  33.                 methodName = attribute.Value.StringValue;
  34.             }
  35.             if (attribute.Key.ToLower() == "httpmethod")
  36.             {
  37.                 httpMethod = attribute.Value.StringValue;
  38.             }
  40.             if (attribute.Key.ToLower() != "methodname" || attribute.Key.ToLower() != "httpmethod")
  41.             {
  42.                 client.DefaultRequestHeaders.Add(attribute.Key, attribute.Value.StringValue);
  43.             }
  44.         }
  46.         if (string.IsNullOrWhiteSpace(item))
  47.         {
  48.             throw new UploaderException("Could not find item");
  49.         }
  51.         string baseUrl = Environment.GetEnvironmentVariable(item.ToUpper());
  52.         var content = new StringContent(message.Body, System.Text.Encoding.UTF8, "application/json");
  54.         context.Logger.LogLine($"URL: {baseUrl}{methodName}");
  56.         HttpResponseMessage response;
  57.         if (httpMethod.ToUpper() == "POST")
  58.         {
  59.             response = await client.PostAsync($"{baseUrl}{methodName}", content);
  60.         }
  61.         else
  62.         {
  63.             response = await client.PutAsync($"{baseUrl}{methodName}", content);
  64.         }
  65.         
  66.         response.EnsureSuccessStatusCode();
  67.         context.Logger.LogLine("Document upload success");
  69.         await Task.CompletedTask;
  70.  }

答案1

得分: 0

有许多不同的过程在同一时间修改相同的客户端（client.DefaultRequestHeaders.Add(..)）- 这可能是一个问题。

我建议为每个消息/HTTP请求创建一个单独的标头对象，完全不依赖默认标头（除非它们被所有请求共享）。

英文:

There are many different processes modifying the same client at the same time (client.DefaultRequestHeaders.Add(..)) - that could be an issue.

I would suggest creating a separate headers-object per message/HTTP request, and not rely on the default headers at all (unless they are shared across all requests).

huangapple
  • 本文由 发表于 2023年2月6日 16:05:01
  • 转载请务必保留本文链接：https://go.coder-hub.com/75358717.html
  • .net
  • amazon-sqs
  • amazon-web-services
  • api
  • aws-lambda
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定