英文:
Does having athena query result have an impact to performance of services that uses it?
问题
我知道Athena将每个查询结果存储在存储桶中，查询数据会随着时间累积。我想知道在S3中保留先前的查询结果是否会影响我的查询性能。
背景信息，我有AWS服务（Glue和Lambda）使用Athena返回数据，而且我的查询结果大多会频繁更改。我注意到我的S3中有200GB的数据。目前，它仅有归档配置。我正在考虑添加生命周期规则，仅保留7天或30天的数据。如果我们实际上不使用查询结果，是否有必要将查询结果存储在S3中？
英文:
I know that athena store every query results in the bucket and query data will just accumulate over time. I want to know whether retaining previous query results in S3 would make an impact to performance of my queries.
For background, I have AWS services(Glue and Lambda) that uses athena to return data and mostly my query results would change frequently. I noticed that there are 200GB worth of data in my S3 now. Currently, it has only archive configurations. I'm thinking of adding life cycle rule that will only retain worth 7 days or 30 days. Is query result really important to be in s3 if we are not really using it?
答案1
得分: 1
查询结果可以在 Athena 中使用一段有限的时间，如果您使用了重用查询结果功能，或者在 AWS 数据处理库中进行缓存。对于其他情况，性能不会受到影响。
几小时前的查询结果仅供审计/调试目的使用。
我强烈建议设置一个生命周期规则来清理超过 x 天的旧对象，其中 x 可以是3天或7天之类的值。
这样做可以降低 S3 存储成本。
英文:
Query results can be used for a limited amount of time by athena if you benefit of reuse query results feature, or caching in AWS data wrangler library. For the remaining scenarios there is no impact on performances.
Query results older than few hours can be used just for auditing/debugging purpouses.
I definitely recommend to put a lifecycle rule to clean up objects older than x days, where x can be something like 3 or 7 days.
Doing so you will reduce s3 storage cost.
答案2
得分: 0
这是两个完全不同的事情。查询结果存储在 S3 结果位置，而 Glue Crawler 在源文件上运行。拥有查询结果历史记录不会对性能产生任何影响。
英文:
These are two completely different things. Query results are stored in S3 results location and the Glue Crawler runs over the Source Files. There is NO performance impact of having history of Query results.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论