在时间范围内查询

huangapple go评论85阅读模式
英文:

Querying across a time range

问题

我们需要查询存储在我们的表中的事件,跨越一段时间范围。用户可以要求从一天的特定时间到另一天的另一个特定时间的事件。

我们一直在考虑将我们的分区键创建为截断值,可能是日期。这样,我们可以通过一次查询获取一整天的事件,然后使用完整的时间属性进行特定时间范围的客户端筛选。

现在,如果我们需要跨越多天进行查询,可能会出现问题,这意味着需要多次查询。一天的事件数量可能从几千个到几十万个不等。

截断的时间戳(精确到一天)是最佳方法,还是有更好的方法?

我们还在思考是否有更好的 AWS 无服务器解决方案适合我们的用例,同时仍然使用 DynamoDB 进行事件存储。

英文:

We have the need to query events stored in our table across a time range. A user could ask for events from a specfic time in one day, to another specfic time in another day.

We have been thinking about creating our partition key as a truncated value, possibly the day. That way with one query we could get a full day of events and then do some client side filtering for the specfic time range using the full time attribute.

Now this may be a problem if we have to query across multiple days, meaning multiple queries. A day could habe anything from a few thousand events, to a couple of hundred thousand.

Is the truncated timestamp (down to a day) the best approach, or is there a better one?

We were also wondering could there be a better aws serverless solution for us to use for this use case, while still using dynamodb for event storage.

答案1

得分: 0

最佳实现方法是将静态值作为GSI分区键,将时间戳作为排序键。

GSI1PK GSI1SK 数据
1 2023-02-01T00:00:000 数据
1 2023-03-01T00:00:000 数据
1 2023-04-01T00:00:000 数据
1 2023-05-01T00:00:000 数据

现在,您可以高效地查询任何数据范围,无需进行过滤。

请注意,将单个值作为索引的分区键会限制您的写入速率,约为每秒1000个WCU。如果您打算扩展到超过这个速率,请考虑对该GSI值进行分片。

英文:

The best way to achieve it is by having a static value as the gsi Partition key and a timestamp as the sort key.

GSI1PK GSI1SK Data
1 2023-02-01T00:00:000 data
1 2023-03-01T00:00:000 data
1 2023-04-01T00:00:000 data
1 2023-05-01T00:00:000 data

Now you can efficiently query for any data range, be it a specific minute of a day without filtering.

Be mindful that keeping a single value as a partition key of your index limits your writes to approx 1000 WCU per second. If you intend to scale beyond that, consider sharding that GSI value.

huangapple
  • 本文由 发表于 2023年8月4日 04:52:36
  • 转载请务必保留本文链接:https://go.coder-hub.com/76831538.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定