英文:
Why is DynamoDB not splitting a hot partition?
问题
我已阅读这篇AWS博客文章：
https://aws.amazon.com/blogs/database/part-2-scaling-dynamodb-how-partitions-hot-keys-and-split-for-heat-impact-performance/
我正试图复制DynamoDB分区拆分。
我的设置如下：
- 1个DynamoDB表按需包含具有相同分区键的大型项目
我在50个不同的EC2实例上运行此命令：
> watch -n 0.1 "aws dynamodb query --table-name "TABLE_NAME" --consistent-read
> --key-condition-expression 'PK = :PK' --expression-attribute-values '{":PK":{"S":"uniquepartition"}}'"
这是我得到的结果
如你所见，它达到了分区的3000 RCU，但然后不进行拆分（读取使用量一段时间后甚至会下降）。我不知道为什么...
（我已在CLI上禁用了重试）
有什么想法？
英文:
I have read this AWS blog post:
https://aws.amazon.com/blogs/database/part-2-scaling-dynamodb-how-partitions-hot-keys-and-split-for-heat-impact-performance/
I am trying to reproduce the DynamoDB partition split.
My setup is the following:
- 1 DynamoDB table on-demand containing big items with the same partition key for all
I am running this command on 50 different EC2 instances:
> watch -n 0.1 "aws dynamodb query --table-name "TABLE_NAME" --consistent-read
> --key-condition-expression 'PK = :PK' --expression-attribute-values '{":PK":{"S":"uniquepartition"}}'"
Here is the result I get
As you can see, it hits the 3000 RCU of the partition but then doesn't split (and the Read Usage even goes down after a while). I don't know why...
(I have disabled retries on the CLI)
Any idea ?
答案1
得分: 4
我没有完全理解您对测试用例的描述，但我假设您只是使用以下命令不断查询具有相同分区键值的内容：
watch -n 0.1 "aws dynamodb query \
--table-name TABLE_NAME \
--consistent-read \
--key-condition-expression 'PK = :PK' \
--expression-attribute-values '{\":PK\":{\"S\":\"uniquepartition\"}}'"
由于您没有排序键，uniquepartition 目标是单个项目，单个项目的最大吞吐量为 3000 RCU，用于强一致性读取。DynamoDB 无法拆分单个项目，因此这个限制是一个无法避免的硬限制。
例如，如果您有一个单个分区，其中包含许多具有唯一键的项目，并且您的读取访问模式均匀分布在这些键上，那么 DynamoDB 将开始为热点进行拆分，并且可能会继续拆分，直到每个项目都有自己的专用分区。
因此，总结一下，您似乎正如预期地受到了3000 RCU的单个项目限制。
再次强调，这是基于假设的，如果我理解正确，请告诉我。
英文:
I don't fully grasp your description of your test case, but my assumption is you are just Querying with the same partition key value constantly with the following command:
watch -n 0.1 "aws dynamodb query \
--table-name "TABLE_NAME" \
--consistent-read \
--key-condition-expression 'PK = :PK' \
--expression-attribute-values '{":PK":{"S":"uniquepartition"}}'"
Since you do not have a sort key, uniquepartition is targetting a single item, and a single items maximum throughput is 3000 RCU for strongly consistent reads. DynamoDB cannot split a single item, so the limit is a hard limit which cannot be avoided.
If for example you had a single partition that held many items all with unique keys and your read access pattern was distributed across the keys evenly, then DynamoDB would begin to split for heat, and could potentially continue to split until each item had its own dedicated partition.
So in summary, you seem to be hitting a single item limit of 3000 RCU as expected.
Again this is based on assumption, let me know if I understood you correctly.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论