将项目放入 Dynamo 失败,没有抛出错误(AWS SDK v3)。

huangapple go评论68阅读模式
英文:

Putting item in dynamo failing with no error thrown (AWS SDK v3)

问题

所以我目前正在开发一个 AWS Lambda,除了做其他几件事情之外,还负责将项目放入 DynamoDB 表中。我在 SAM 模板中设置了所有的权限(下面是一个简短的示例)

     - Effect: "Allow"
            Action:
               - "dynamodb:GetItem"
               - "dynamodb:PutItem"
               - "dynamodb:UpdateItem"
               - "dynamodb:DeleteItem"
            Resource:
             - !Join ["/", [!Sub "arn:aws:dynamodb:${AWS::Region}:${AWS::AccountId}:table", '{{resolve:ssm:/examples/example/tableName}}'] ]


用于调用表格并放入项目的逻辑如下:

**dynamoStorage.ts**

    export const setThings = async (accountId: string, clientKey: string, things: EnhancedSearchFilter[]): Promise<void> => {
        if (!tableName) {
            return Promise.reject(new Error(DynamoError.InvalidTableName));
        }
        // 将所有的项目发送到 Dynamo
        const promises = things.map((thing) => putThing(accountId, clientKey, thing));
        return Promise.allSettled(promises).then((results) => {
            if (results.some((result) => result.status === 'rejected')) {
                customLogger.error('Failed to put things', { clientKey, things });
                throw new Error('Failed to set all filters');
            }
        });
    };

**调用 Dynamo 的 putThing 函数**

    export const putThing = async (accountId: string, clientKey: string, thing: EnhancedThing): Promise<void> => {

        if (!tableName) {
            return Promise.reject(new Error(DynamoError.InvalidTableName));
        }
    
        const thingRef = createThingRef(accountId, thing.id);
        const params: PutCommandInput = {
            Item: thingToItem(clientKey, accountId, thing),
            TableName: tableName
        };
    
        try {
            await dynamoClient.send(new PutCommand(params));
            return Promise.resolve();
        } catch (cause) {
            customLogger.error('Failed to put thing', { clientKey, thingRef, cause });
            throw new Error(`Failed to put thing`, { cause });
        }
    };

在 **CloudWatch** 中我可以看到 `customLogger.error('Failed to put thing', { clientKey, thingRef, cause });` 出现了很多次,但原因显示为空对象 `{}`。

我的问题是,我在这种情况下是否错误处理错误,如果是,为什么 Dynamo 失败了却没有返回错误?
英文:

So I'm currently working on an AWS lambda that, amongst several different things, has the responsibility to put items into a dynamoDb table. I have all the permissions setup in the SAM template (short example below)

 - Effect: &quot;Allow&quot;
        Action:
           - &quot;dynamodb:GetItem&quot;
           - &quot;dynamodb:PutItem&quot;
           - &quot;dynamodb:UpdateItem&quot;
           - &quot;dynamodb:DeleteItem&quot;
        Resource:
         - !Join [&quot;/&quot;, [!Sub &quot;arn:aws:dynamodb:${AWS::Region}:${AWS::AccountId}:table&quot;, &#39;{{resolve:ssm:/examples/example/tableName}}&#39;] ]

My logic to call the table and put the item is the following

dynamoStorage.ts

export const setThings = async (accountId: string, clientKey: string, things: EnhancedSearchFilter[]): Promise&lt;void&gt; =&gt; {
    if (!tableName) {
        return Promise.reject(new Error(DynamoError.InvalidTableName));
    }
    // send all the things over to dynamo
    const promises = things.map((thing) =&gt; putThing(accountId, clientKey, thing));
    return Promise.allSettled(promises).then((results) =&gt; {
        if (results.some((result) =&gt; result.status === &#39;rejected&#39;)) {
            customLogger.error(&#39;Failed to put things&#39;, { clientKey, things });
            throw new Error(&#39;Failed to set all filters&#39;);
        }
    });
};

putThing function calling dynamo

export const putThing = async (accountId: string, clientKey: string, thing: EnhancedThing): Promise&lt;void&gt; =&gt; {

    if (!tableName) {
        return Promise.reject(new Error(DynamoError.InvalidTableName));
    }

    const thingRef = createThingRef(accountId, thing.id);
    const params: PutCommandInput = {
        Item: thingToItem(clientKey, accountId, thing),
        TableName: tableName
    };

    try {
        await dynamoClient.send(new PutCommand(params));
        return Promise.resolve();
    } catch (cause) {
        customLogger.error(&#39;Failed to put thing&#39;, { clientKey, thingRef, cause });
        throw new Error(`Failed to put thing`, { cause });
    }
};

In cloudwatch I can see this customLogger.error(&#39;Failed to put thing&#39;, { clientKey, thingRef, cause }); quite a few times but the cause is displayed as an empty object {}

My question is, am I doing the error handling wrong in this scenario, and if so, how can I improve, or if it is right, why on earth is dynamo failing giving no error back?

Thanks

答案1

得分: 2

因为你在使用 Promise.allSettled,错误消息(或错误的原因)会在数组 results 的每个项目中返回,如果你检查这部分的 result

return Promise.allSettled(promises).then((results) => {
    // 这个 `result` 对象有一个 `result.reason` 属性
    if (results.some((result) => result.status === 'rejected')) {
        customLogger.error('Failed to put things', { clientKey, things });
        throw new Error('Failed to set all filters');
    }
});

你可以在特定的 promise 中找到错误消息。Promise.allSettled 并不处理错误消息。你需要检查每个 promise 是否有自己的错误。

你可以查看 MDN 文档中有关这个 Promise API 的说明:https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/allSettled

在你的代码片段中的这部分,你隐藏了 DynamoDB 返回的错误/忽略了它。

你需要改变你对 rejected 状态的 if 条件/查找。你可以使用 Array.find,例如。

批量写入 DynamoDB

另一个建议是探索 DynamoDB SDK v3 中的 BatchWriteItemCommand

你正在使用的代码模式是:“创建一个 PutItem 的 promises 数组,等待它们全部解析。”

这可能会在 DynamoDB 中创建瓶颈,具体取决于项目的大小和数量以及你的表是如何配置的(按需还是预配置容量)。

通过单个调用 BatchWriteItem,你可以在网络上传输多达 16MB 的数据,其中包括多达 25 个项目的 put 或 delete 操作。

你可能需要进行一些性能测试,但它可以改善你的应用程序性能。减少 Lambda 中的内存和时间,而不是进行 N 个 promises 的调用,你只需进行一次调用并等待来自 DynamoDB 的网络响应。与 N 个 promises 调用不同,你发送一个操作列表,DynamoDB 在他们的一侧处理写入。

英文:

> My question is, am I doing the error handling wrong in this scenario, and if so, how can I improve, or if it is right, why on earth is dynamo failing giving no error back?

Because you are using Promise.allSettled, the error message (or the reason of the error), is returned in each item of the array results, if you inspect the result of this part:

return Promise.allSettled(promises).then((results) =&gt; {
    //                &#128071;&#128071;&#128071; this `result` object have a `result.reason` property
    if (results.some((result) =&gt; result.status === &#39;rejected&#39;)) {
        customLogger.error(&#39;Failed to put things&#39;, { clientKey, things });
        throw new Error(&#39;Failed to set all filters&#39;);
    }
});

You can find the error message of that specific promise. The Promise.allSettled doesn't handle the error message. You need to inspect each promise for its own error.

You can check the MDN documentation of this Promise API: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/allSettled

In that part of the code in your snippet, you are hiding the error/swallowing the error returned by DynamoDB.

You will need to change the if / lookup you do to the rejected state. You can use a Array.find, for example.

Batch writing to DynamoDB

Another suggestion here, is to explore the BatchWriteItemCommand from DynamoDB SDK v3:

The code pattern you are using: "Create an array of PutItem promises and wait for all of them to resolve."

It can create bottlenecks or be throttled by DynamoDB, depending on the size x number of items and how your table is configured (On-demand x Provisioned Capacity).

With a single call to BatchWriteItem, you can transmit up to 16MB of data over the network, consisting of up to 25 items put or delete operations.

You may need to measure it, but it can improve your application performance. Reducing memory and time in the lambda, instead of N promises call, you have a single call and waiting for a network response from DynamoDB. Instead of N promises calls, you send a list of actions, and DynamoDB handles the write on their side)

huangapple
  • 本文由 发表于 2023年2月19日 18:35:04
  • 转载请务必保留本文链接:https://go.coder-hub.com/75499503.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定