AWS S3预签名URL策略

huangapple go评论106阅读模式
英文:

AWS S3 Presigned URLs Policies

问题

目前正在进行一个项目,基本上是一个存储在AWS S3上的在线画廊。目前的问题是,前端网页发送一个请求给托管在EC2上的API服务,然后返回一个预签名URL。该服务将使用Go语言和aws-sdk-go-v2。

URL是有时间限制的,设置为PUT对象类型。不幸的是,我还没有找到如何限制将要上传的文件的其他方面。理想情况下,URL应该限制它可以接受的内容,比如只接受图片。

我在搜索中得到了各种各样的结果,有些说可能可以实现,有些说不可能,还有一些根本没有提到我要做的事情。

关于设置POST/PUT策略的答案有很多,但它们要么是针对V1 SDK,要么是完全不同的语言。这个答案甚至提供了一些可以实现的库,但我不想立即使用它,因为它需要将访问密钥放置在代码中或环境中的某个位置(我想利用EC2的IAM角色自动化该过程的好处)。

有人知道在SDK的V2版本中是否仍然可以实现这个功能吗?出于一致性的考虑,我希望保持在SDK的V2版本上。

编辑:几乎忘了。我看到S3在上传完成后也可以跟随一个链接。这个功能还存在吗?这将是一个很好的验证方式,以便记录已上传的内容。

英文:

Currently working on a project that involves what is essentially an online gallery stored on AWS S3. The current process in question is that the frontend webpage sends a request for an API service hosted on EC2, which then returns with a presigned URL. The service will be using Go, with aws-sdk-go-v2.

The URL is time limited, set as a PUT object type. Unfortunately I haven't figured out how to limit the other aspects of the file that will be uploaded yet. Ideally, the URL should be limited in what it can accept, IE images only.

My searchs around have come up with mixed results, saying both it's possible, not possible, or just outright not mentioned for what I'm doing.

There's plenty of answers regarding setting a POST/PUT policy, but I they're either for the V1 SDK, or just outright a different language. This answer here even has a few libraries that does it, but I don't want to resort to using that yet since it requires access keys to be placed in the code, or somewhere in the environment (Trying to reap the benefits of EC2's IAM role automating the process).

Anyone know if this is still a thing on V2 of the SDK? I kinda want to keep it on V2 of the SDK just for the sake of consistency.

EDIT: Almost forgot. I saw that it was possible for S3 to follow a link upon upload completion as well. Is this still possible? Would be great as a verification that something was uploaded so that it could be logged.

答案1

得分: 4

你可以在返回预签名的PUT URL之前,通过后端API验证文件名的有效性。另外一个不太安全但也可行的方法是在前端客户端验证文件内容。

英文:

You can try to validate the filename through the your backend API before returning the PreSigned PUT URL. And a less sequre but can be good, is to validate the file content in the frontend client.

答案2

得分: 1

很抱歉,我目前还没有找到一种通过Content-Type限制上传的方法,但我找到了一些可能会对你有帮助的技巧。

首先,虽然这可能有点过于粗暴,但你可以通过应用存储桶策略来限制文件名。下面是一个示例,来自AWS知识中心,只允许上传几种图像类型的文件。

{
  "Version": "2012-10-17",
  "Id": "Policy1464968545158",
  "Statement": [
    {
      "Sid": "Stmt1464968483619",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::111111111111:user/exampleuser"
      },
      "Action": "s3:PutObject",
      "Resource": [
        "arn:aws:s3:::DOC-EXAMPLE-BUCKET/*.jpg",
        "arn:aws:s3:::DOC-EXAMPLE-BUCKET/*.png",
        "arn:aws:s3:::DOC-EXAMPLE-BUCKET/*.gif"
      ]
    },
    {
      "Sid": "Stmt1464968483619",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:PutObject",
      "NotResource": [
        "arn:aws:s3:::DOC-EXAMPLE-BUCKET/*.jpg",
        "arn:aws:s3:::DOC-EXAMPLE-BUCKET/*.png",
        "arn:aws:s3:::DOC-EXAMPLE-BUCKET/*.gif"
      ]
    }
  ]
}

其次,你可以在文件上传后自动触发一个Lambda函数。通过这种方法,你可以在文件上传后检查任何你想要的文件信息。一般来说,这是我首选的方法,但它的问题是没有即时的反馈告诉你文件是否有效。

最后一个选项需要客户端做更多的工作,这是我最近使用的方法。如果你要上传非常大的文件,S3要求你使用分块上传。这里的分块上传不是指HTML表单中的分块,而是将文件分成多个块并逐个上传。如果你的文件足够小(我认为限制是5GB),你可以只上传一个部分。一旦所有部分都上传完毕,你必须向服务器发出另一个请求,以完成与S3的上传。在这里,你可以对文件进行一些验证,并在客户端仍然在上传页面时向其响应。

英文:

I unfortunately have not discovered a way to restrict uploads via the Content-Type, but I have found a few tricks that might help you.

First, while this is a bit of a sledgehammer when you might want a scalpel, you can apply a bucket policy to restrict by filename. This example from AWS Knowledge Center only allows a few image types.

{
  "Version": "2012-10-17",
  "Id": "Policy1464968545158",
  "Statement": [
    {
      "Sid": "Stmt1464968483619",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::111111111111:user/exampleuser"
      },
      "Action": "s3:PutObject",
      "Resource": [
        "arn:aws:s3:::DOC-EXAMPLE-BUCKET/*.jpg",
        "arn:aws:s3:::DOC-EXAMPLE-BUCKET/*.png",
        "arn:aws:s3:::DOC-EXAMPLE-BUCKET/*.gif"
      ]
    },
    {
      "Sid": "Stmt1464968483619",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:PutObject",
      "NotResource": [
        "arn:aws:s3:::DOC-EXAMPLE-BUCKET/*.jpg",
        "arn:aws:s3:::DOC-EXAMPLE-BUCKET/*.png",
        "arn:aws:s3:::DOC-EXAMPLE-BUCKET/*.gif"
      ]
    }
  ]
}

Second, you can automatically trigger a Lambda function after the file is uploaded. With this method, you can inspect anything about the file you want after it is uploaded. This is generally my preferred approach, but the problem with it is there's not really any immediate feedback the file was invalid.

My last option requires the client to do some more work, and it's what I used most recently. If you are uploading very large files, S3 requires you to use multipart uploads. This isn't multipart like an HTML form; this is breaking the file up into chunks and uploading each of them. If your file is sufficiently small--I think the limit is 5GB, you can just do a single part. Once all the parts are uploaded, you must make another call to your server that finalizes the upload with S3. This is where you can add some validation of the file and respond to the client while they're still on the upload page.

huangapple
  • 本文由 发表于 2022年2月2日 21:13:25
  • 转载请务必保留本文链接:https://go.coder-hub.com/70956274.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定