英文:
Querying across a time range
问题
我们需要查询存储在我们的表中的事件,跨越一段时间范围。用户可以要求从一天的特定时间到另一天的另一个特定时间的事件。
我们一直在考虑将我们的分区键创建为截断值,可能是日期。这样,我们可以通过一次查询获取一整天的事件,然后使用完整的时间属性进行特定时间范围的客户端筛选。
现在,如果我们需要跨越多天进行查询,可能会出现问题,这意味着需要多次查询。一天的事件数量可能从几千个到几十万个不等。
截断的时间戳(精确到一天)是最佳方法,还是有更好的方法?
我们还在思考是否有更好的 AWS 无服务器解决方案适合我们的用例,同时仍然使用 DynamoDB 进行事件存储。
英文:
We have the need to query events stored in our table across a time range. A user could ask for events from a specfic time in one day, to another specfic time in another day.
We have been thinking about creating our partition key as a truncated value, possibly the day. That way with one query we could get a full day of events and then do some client side filtering for the specfic time range using the full time attribute.
Now this may be a problem if we have to query across multiple days, meaning multiple queries. A day could habe anything from a few thousand events, to a couple of hundred thousand.
Is the truncated timestamp (down to a day) the best approach, or is there a better one?
We were also wondering could there be a better aws serverless solution for us to use for this use case, while still using dynamodb for event storage.
答案1
得分: 0
最佳实现方法是将静态值作为GSI分区键,将时间戳作为排序键。
GSI1PK | GSI1SK | 数据 |
---|---|---|
1 | 2023-02-01T00:00:000 | 数据 |
1 | 2023-03-01T00:00:000 | 数据 |
1 | 2023-04-01T00:00:000 | 数据 |
1 | 2023-05-01T00:00:000 | 数据 |
现在,您可以高效地查询任何数据范围,无需进行过滤。
请注意,将单个值作为索引的分区键会限制您的写入速率,约为每秒1000个WCU。如果您打算扩展到超过这个速率,请考虑对该GSI值进行分片。
英文:
The best way to achieve it is by having a static value as the gsi Partition key and a timestamp as the sort key.
GSI1PK | GSI1SK | Data |
---|---|---|
1 | 2023-02-01T00:00:000 | data |
1 | 2023-03-01T00:00:000 | data |
1 | 2023-04-01T00:00:000 | data |
1 | 2023-05-01T00:00:000 | data |
Now you can efficiently query for any data range, be it a specific minute of a day without filtering.
Be mindful that keeping a single value as a partition key of your index limits your writes to approx 1000 WCU per second. If you intend to scale beyond that, consider sharding that GSI value.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论