英文:
How to download object from AWS S3 into memory and send it via request in Go?
问题
有人可以帮我解决我在尝试从S3下载文件时遇到的错误吗？
所以我想通过Go Gin创建存储服务API，并且我想要一个路由，它接受S3对象键以下载对象，然后将此对象发送回客户端。
cfg,_ := config.LoadDefaultConfig(context.TODO())
// 创建 Amazon S3 服务客户端
s3Client := s3.NewFromConfig(cfg)
downloader := manager.NewDownloader(s3Client)
router.POST("/download-s3", func(ctx *gin.Context) {
var data map[string]string
if err := ctx.ShouldBindJSON(&data); err != nil {
log.Println("Error: bind error")
ctx.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
log.Println("Body Request data (fileKey): ", data["fileKey"])
fileKey := data["fileKey"]
// 缓冲区读取
buf := make([]byte, 100)
// 使用 aws.WriteAtBuffer 进行包装
w := manager.NewWriteAtBuffer(buf)
// 将文件下载到内存中
numBytesDownloaded, err := downloader.Download(ctx, w, &s3.GetObjectInput{
Bucket: aws.String(bucketName),
Key: aws.String(fileKey),
})
if err != nil {
log.Println("Error: download error")
ctx.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
ctx.JSON(http.StatusAccepted,
gin.H{
"message": fmt.Sprintf("'%s' downloaded!", "test.jpg"),
"numBytesDownloaded": numBytesDownloaded,
},
)
ctx.Data(http.StatusOK, "application/octet-stream", w.Bytes())
})
上述代码给我返回了以下错误：
operation error S3: GetObject, https response error StatusCode: 403, RequestID: 8KHYW3RCS2ZA7D95, HostID: kOLo+gY/dTzrROVswsB1bxinlN8w+XDL0LNMySUWJNHUPMBVk4ItdfP+mQ2xuo8ehaSnX/FaI4I=, api error AccessDenied: Access Denied
请求体的格式如下：
{
"filekey": "https://xxx-xxx.s3.ap-southeast-1.amazonaws.com/agencies/photo/06%3A03%3A2023_17%3A42%3A42_gggi_db_er.png"
}
我在.env文件中存储了所有这些AWS_S3_BUCKET_NAME、AWS_REGION、AWS_ACCESS_KEY_ID、AWS_SECRET_ACCESS_KEY，而且我认为我已经正确设置了它们，因为我有另一个将文件上传到S3的路由（这个路由工作得很好）。
英文:
Can someone please help me solve this error I got while trying to download a file from S3.
So I want to create storage service api via Go Gin, and I want a route where it take S3 object key to download the object then send this object back to client.
cfg,_ := config.LoadDefaultConfig(context.TODO())
// Create an Amazon S3 service client
s3Client := s3.NewFromConfig(cfg)
downloader := manager.NewDownloader(s3Client)
router.POST("/download-s3", func(ctx *gin.Context) {
var data map[string]string
if err := ctx.ShouldBindJSON(&data); err != nil {
log.Println("Error: bind error")
ctx.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
log.Println("Body Request data (fileKey): ", data["fileKey"])
fileKey := data["fileKey"]
/// Buffer read
buf := make([]byte, 100)
// wrap with aws.WriteAtBuffer
w := manager.NewWriteAtBuffer(buf)
// download file into the memory
numBytesDownloaded, err := downloader.Download(ctx, w, &s3.GetObjectInput{
Bucket: aws.String(bucketName),
Key: aws.String(fileKey),
})
if err != nil {
log.Println("Error: download error")
ctx.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
ctx.JSON(http.StatusAccepted,
gin.H{
"message": fmt.Sprintf("'%s' downloaded!", "test.jpg"),
"numBytesDownloaded": numBytesDownloaded,
},
)
ctx.Data(http.StatusOK, "application/octet-stream", w.Bytes())
})
This above code gave me this error of :
operation error S3: GetObject, https response error StatusCode: 403, RequestID: 8KHYW3RCS2ZA7D95, HostID: kOLo+gY/dTzrROVswsB1bxinlN8w+XDL0LNMySUWJNHUPMBVk4ItdfP+mQ2xuo8ehaSnX/FaI4I=, api error AccessDenied: Access Denied
Request body is something like this:
{
"filekey": "https://xxx-xxx.s3.ap-southeast-1.amazonaws.com/agencies/photo/06%3A03%3A2023_17%3A42%3A42_gggi_db_er.png"
}
I got all these AWS_S3_BUCKET_NAME, AWS_REGION, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY stored in .env file, plus I think I set them right since I got another route to upload file to S3 (which work just fine).
答案1
得分: 2
你的fileKey是一个像这样的URL：
"https://xxx-xxx.s3.ap-southeast-1.amazonaws.com/agencies/photo/06%3A03%3A2023_17%3A42%3A42_gggi_db_er.png"
这只能在HTTP客户端（如Web浏览器或curl）中使用。
具体来说，它不是一个有效的S3对象键。在使用GetObject或类似的S3 API时，它需要是一个有效的键，例如agencies/photo/xyz.png，而且不应该进行URL编码。
英文:
Your fileKey is a URL like this:
"https://xxx-xxx.s3.ap-southeast-1.amazonaws.com/agencies/photo/06%3A03%3A2023_17%3A42%3A42_gggi_db_er.png"
That is only usable from an HTTP client such as a web browser or curl.
Specifically, it is not a valid S3 object key. When using GetObject or similar S3 APIs, it needs to be a valid key such as agencies/photo/xyz.png and should not be URL-encoded.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论