英文:
Get S3 bucket size from aws GO SDK
问题
我正在使用Amazon S3存储桶来上传文件（使用GO SDK）。我有一个要求，当客户的目录大小超过2GB时要收费。
存储桶内的目录层次结构如下：/BUCKET/uploads/CLIENTID/yyyy/mm/dd
为此，我已经进行了大量的搜索，但没有找到相关信息。
如何使用SDK获取存储桶内目录的大小？
英文:
I am using Amazon S3 bucket to upload files (using GO SDK). I have a requirement to charge client when their directory size exceeds 2GB.
The directory hierarchy inside bucket is like: /BUCKET/uploads/CLIENTID/yyyy/mm/dd
For this, I have searched a lot about it. But could not find anything.
How can I get the directory size inside a bucket using SDK ?
答案1
得分: 2
首先，/uploads/CLIENTID/yyyy/mm/dd 不是 S3 存储桶中的一个目录，而是一个 前缀。AWS 控制台中的 S3 管理界面可能会让你误以为存储桶有子目录，就像计算机文件系统一样，但它们实际上是前缀。
你的问题实际上是：如何获取具有给定前缀的存储桶中所有对象的总大小？
希望下面的代码片段能够解答你的疑问。
package main
import (
"context"
"fmt"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/service/s3"
)
// 遍历给定 S3 存储桶和前缀中的所有对象，累加对象的总大小（以字节为单位）
// 使用方法：size, err := S3ObjectsSize("example-bucket-name", "/a/b/c")
func S3ObjectsSize(bucket string, prefix string, s3client S3Client) (int64, error) {
output, err := s3client.ListObjectsV2(context.TODO(), &s3.ListObjectsV2Input{
Bucket: aws.String(bucket),
Prefix: aws.String(prefix),
})
if err != nil {
return -1, fmt.Errorf("无法在 %s/%s 中列出对象：%s", bucket, prefix, err.Error())
}
var size int64
for _, object := range output.Contents {
size += object.Size
}
return size, nil
}
// 用于依赖注入的 s3.Client 存根
type S3Client interface {
ListObjectsV2(ctx context.Context, params *s3.ListObjectsV2Input, optFns ...func(*s3.Options)) (*s3.ListObjectsV2Output, error)
}
英文:
First of all, /uploads/CLIENTID/yyyy/mm/dd is not a directory in S3 bucket, but a prefix. The S3 management UI in AWS Console may trick you to think a bucket has subdirectories, just like your computer file system, but they are prefixes.
Your question really is: How can I get the total size of all objects inside a bucket, with a given prefix?
Hope this code snippet can clear your doubts.
package main
import (
"context"
"fmt"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/service/s3"
)
// iterate all objects in a given S3 bucket and prefix, sum up objects' total size in bytes
// use: size, err := S3ObjectsSize("example-bucket-name", "/a/b/c")
func S3ObjectsSize(bucket string, prefix string, s3client S3Client) (int64, error) {
output, err := s3client.ListObjectsV2(context.TODO(), &s3.ListObjectsV2Input{
Bucket: aws.String(bucket),
Prefix: aws.String(prefix),
})
if err != nil {
return -1, fmt.Errorf("cannot ListObjectsV2 in %s/%s: %s", bucket, prefix, err.Error())
}
var size int64
for _, object := range output.Contents {
size += object.Size
}
return size, nil
}
// stub of s3.Client for dependency injection
type S3Client interface {
ListObjectsV2(ctx context.Context, params *s3.ListObjectsV2Input, optFns ...func(*s3.Options)) (*s3.ListObjectsV2Output, error)
}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论