AWS DynamoDB BatchWriteCommand fails with ValidationException with marshalled data in typescript

huangapple go评论76阅读模式
英文:

AWS DynamoDB BatchWriteCommand fails with ValidationException with marshalled data in typescript

问题

I am trying to persist marshalled data in AWS DynamoDB using BatchWriteCommand from AWS Lambda. However, it fails with a ValidationException:

{
  "errorType": "ValidationException",
  "errorMessage": "The provided key element does not match the schema",
  "trace": [
    "ValidationException: The provided key element does not match the schema",
    "    at throwDefaultError (/var/runtime/node_modules/@aws-sdk/smithy-client/dist-cjs/default-error-handler.js:8:22)",
    "    at deserializeAws_json1_0BatchWriteItemCommandError (/var/runtime/node_modules/@aws-sdk/client-dynamodb/dist-cjs/protocols/Aws_json1_0.js:665:51)",
    "    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)",
    "    at async /var/runtime/node_modules/@aws-sdk/middleware-serde/dist-cjs/deserializerMiddleware.js:7:24",
    "    at async /var/runtime/node_modules/@aws-sdk/lib-dynamodb/dist-cjs/baseCommand/DynamoDBDocumentClientCommand.js:18:34",
    "    at async /opt/nodejs/node_modules/@aws-sdk/client-dynamodb/node_modules/@aws-sdk/middleware-signing/dist-cjs/middleware.js:13:20",
    "    at async StandardRetryStrategy.retry (/opt/nodejs/node_modules/@aws-sdk/client-dynamodb/node_modules/@aws-sdk/middleware-retry/dist-cjs/StandardRetryStrategy.js:51:46)",
    "    at async /opt/nodejs/node_modules/@aws-sdk/client-dynamodb/node_modules/@aws-sdk/middleware-logger/dist-cjs/loggerMiddleware.js:6:22",
    "    at async Runtime.handler (/var/task/main.js:95:22)"
  ]
}

Code:

export interface MyEvent {
    eventId: number
    eventTime: number
}

export interface MyModel {
    category: string
    status: string
    details: MyEvent[]
}

const dbClient = new DynamoDB({ apiVersion: "2012-08-10", "ap-southeast-2" })
export const ddbDocClient = DynamoDBDocument.from(dbClient)

export async function save(myItems: MyModel[]): Promise<void> {
    const marshalledData = myItems.map(event => ({
        PutRequest: {
            Item: marshall({
                myPK: "111111",
                mySortKey: event.category,
                status: event.status,
                eventDetails: event.details
            })
        }
    }))
    const command = new BatchWriteCommand({
        RequestItems: {
            ["events"]: marshalledData
        }
    })
    try {
        await ddbDocClient.send(command)
    } catch (e) {
        throw Error(`Error ${(e as Error).message} `)
    }
}

Can anyone please help me understand what could be the issue? If I marshall only the eventDetails like below, then the data gets persisted; however, the data is not saved as MapAttribute:

eventDetails: marshall(event.details)

Data gets saved as:

[ { "M" : { "M" : { "M" : ....

The correct way should be:

{
    "L": [
      {
        "M": {

Thanks

Update:
Please find the stringify of the marshalledData:

{
	category: { S: 'Accounts' },
	status: { S: 'Active' },
	details: {
		L: [
			{
				M: {
					eventId: { N: '101' },
					eventTime: { N: '1686369780' }
				}
			},
			{
				M: {
					eventId: { N: '102' },
					eventTime: { N: '1686371880' }
				}
			}
		]
	}
}

(Note: I've formatted the provided JSON for better readability, but I have not translated the code and JSON keys, as requested.)

英文:

I am trying to persist marshalled data in AWS DynamoDB using BatchWriteCommand from AWS Lambda. However it fails with ValidationException :

{
  &quot;errorType&quot;: &quot;ValidationException&quot;,
  &quot;errorMessage&quot;: &quot;The provided key element does not match the schema&quot;,
  &quot;trace&quot;: [
    &quot;ValidationException: The provided key element does not match the schema&quot;,
    &quot;    at throwDefaultError (/var/runtime/node_modules/@aws-sdk/smithy-client/dist-cjs/default-error-handler.js:8:22)&quot;,
    &quot;    at deserializeAws_json1_0BatchWriteItemCommandError (/var/runtime/node_modules/@aws-sdk/client-dynamodb/dist-cjs/protocols/Aws_json1_0.js:665:51)&quot;,
    &quot;    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)&quot;,
    &quot;    at async /var/runtime/node_modules/@aws-sdk/middleware-serde/dist-cjs/deserializerMiddleware.js:7:24&quot;,
    &quot;    at async /var/runtime/node_modules/@aws-sdk/lib-dynamodb/dist-cjs/baseCommand/DynamoDBDocumentClientCommand.js:18:34&quot;,
    &quot;    at async /opt/nodejs/node_modules/@aws-sdk/client-dynamodb/node_modules/@aws-sdk/middleware-signing/dist-cjs/middleware.js:13:20&quot;,
    &quot;    at async StandardRetryStrategy.retry (/opt/nodejs/node_modules/@aws-sdk/client-dynamodb/node_modules/@aws-sdk/middleware-retry/dist-cjs/StandardRetryStrategy.js:51:46)&quot;,
    &quot;    at async /opt/nodejs/node_modules/@aws-sdk/client-dynamodb/node_modules/@aws-sdk/middleware-logger/dist-cjs/loggerMiddleware.js:6:22&quot;,
    &quot;    at async Runtime.handler (/var/task/main.js:95:22)&quot;
  ]
}

Code:

export interface MyEvent {
    eventId: number
    eventTime: number
}

export interface MyModel {
    category: string
    status: string
    details: MyEvent[]
}

const dbClient = new DynamoDB({ apiVersion: &quot;2012-08-10&quot;, &quot;ap-southeast-2&quot; })
export const ddbDocClient = DynamoDBDocument.from(dbClient)

export async function save(myItems: MyModel[]): Promise&lt;void&gt; {
    const marshalledData = myItems.map(event =&gt; ({
        PutRequest: {
            Item: marshall({
                myPK: &quot;111111&quot;,
                mySortKey: event.category,
                status: event.status,
                eventDetails: event.details
            })
        }
    }))
    const command = new BatchWriteCommand({
        RequestItems: {
            [&quot;events&quot;]: marshalledData
        }
    })
    try {
        await ddbDocClient.send(command)
    } catch (e) {
        throw Error(`Error ${(&lt;Error&gt;e).message} `)
    }
}

Can anyone please help me understand what could be the issue? If I marshall only the eventDetails like below, then the data gets persisted, however the data is not saved as MapAttribute:

eventDetails: marshall(event.details)

Data gets saved as:

[ { &quot;M&quot; : { &quot;M&quot; : { &quot;M&quot; ....

Correct way should be:

{
    L: [
      {
        M: {

Thanks

Update:
Please find the stringify of the marshalledData:

{
	category: { S: &#39;Accounts&#39; },
	status: { S: &#39;Active&#39; },
	details: {
		L: [
			{
				M: {
					eventId: { N: &#39;101&#39; },
					eventTime: { N: &#39;1686369780&#39; }
				}
			},
			{
				M: {
					eventId: { N: &#39;102&#39; },
					eventTime: { N: &#39;1686371880&#39; }
				}
			}
		]
	}
}

答案1

得分: 0

谢谢更新,我现在明白你的问题。你正在使用文档客户端,但同时也在使用 marshall,这会导致你对 DynamoDB-JSON 进行双重处理。文档客户端默认会执行编组,这是它的实用之处。尝试:

    const marshalledData = myItems.map(event =&gt; ({
        PutRequest: {
            Item:{
                myPK: &quot;111111&quot;,
                mySortKey: event.category,
                status: event.status,
                eventDetails: event.details
            }
        }
    }))
英文:

Thanks for the update, I see your issue now. You are using the Document Client but you are also using marshall which in turn is causing you to double on the DynamoDB-JSON. Document Client marshalls by default, thats what makes it useful. Try:

    const marshalledData = myItems.map(event =&gt; ({
        PutRequest: {
            Item:{
                myPK: &quot;111111&quot;,
                mySortKey: event.category,
                status: event.status,
                eventDetails: event.details
            }
        }
    }))

huangapple
  • 本文由 发表于 2023年6月8日 17:21:02
  • 转载请务必保留本文链接:https://go.coder-hub.com/76430364.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定