英文:
aws-sdk-go-v2 PutObject api error AccessDenied
问题
在我们的Staging环境中,我们可以无需凭证访问私有的S3存储桶。访问权限是授予各个Docker容器的。我正在尝试使用aws-sdk-go-v2 SDK库的PutObject
方法上传文件,但是一直收到403 AccessDenied的API错误。
我的上传代码如下所示:
var uploadFileFunc = func(s3Details S3Details, key string, payload []byte, params MetadataParams) (*s3.PutObjectOutput, error) {
client := getS3Client(s3Details)
return client.PutObject(context.TODO(), &s3.PutObjectInput{
Bucket: aws.String(s3Details.Bucket),
Key: aws.String(key),
Body: bytes.NewReader(payload),
ContentType: aws.String("text/xml"),
})
}
func getS3Client(s3Details S3Details) *s3.Client {
endpointResolver := aws.EndpointResolverWithOptionsFunc(func(service, region string, options ...interface{}) (aws.Endpoint, error) {
if s3Details.EndpointUrl != "" {
return aws.Endpoint{
PartitionID: "aws",
URL: s3Details.EndpointUrl,
SigningRegion: s3Details.Region,
SigningMethod: s3Details.SignatureVersion,
}, nil
}
return aws.Endpoint{}, &aws.EndpointNotFoundError{}
})
cfg, _ := config.LoadDefaultConfig(context.TODO(),
config.WithEndpointDiscovery(aws.EndpointDiscoveryEnabled),
config.WithEndpointResolverWithOptions(endpointResolver))
return s3.NewFromConfig(cfg, func(o *s3.Options) {
o.Region = s3Details.Region
o.Credentials = aws.AnonymousCredentials{}
o.UsePathStyle = true
})
}
我正在使用aws.AnonymousCredentials{}
(因为我们是无凭证访问),但这只适用于未签名的请求。我不能使用NewStaticCredentialsProvider
并将AccessKeyID
和/或SecretAccessKey
的值留空,因为这会在Retrieve()
过程中引发StaticCredentialsEmptyError
错误。添加虚拟凭证会引发错误,表示它们未记录。我认为这是导致我收到403 AccessDenied
错误的原因。
在Go SDK中,如何在不提供凭证的情况下对请求进行签名?这是否可能?在Python的boto3库中,这个操作是可行的。
英文:
In our Staging environment, we have credential-less access to our private S3 buckets. Access is granted to individual Docker containers. I am trying to upload a file using PutObject
using the aws-sdk-go-v2 SDK library, but I'm continually getting a 403 AccessDenied api error.
My upload code looks like this:
var uploadFileFunc = func(s3Details S3Details, key string, payload []byte, params MetadataParams) (*s3.PutObjectOutput, error) {
client := getS3Client(s3Details)
return client.PutObject(context.TODO(), &s3.PutObjectInput{
Bucket: aws.String(s3Details.Bucket),
Key: aws.String(key),
Body: bytes.NewReader(payload),
ContentType: aws.String("text/xml"),
})
}
func getS3Client(s3Details S3Details) *s3.Client {
endpointResolver := aws.EndpointResolverWithOptionsFunc(func(service, region string, options ...interface{}) (aws.Endpoint, error) {
if s3Details.EndpointUrl != "" {
return aws.Endpoint{
PartitionID: "aws",
URL: s3Details.EndpointUrl,
SigningRegion: s3Details.Region,
SigningMethod: s3Details.SignatureVersion,
}, nil
}
return aws.Endpoint{}, &aws.EndpointNotFoundError{}
})
cfg, _ := config.LoadDefaultConfig(context.TODO(),
config.WithEndpointDiscovery(aws.EndpointDiscoveryEnabled),
config.WithEndpointResolverWithOptions(endpointResolver))
return s3.NewFromConfig(cfg, func(o *s3.Options) {
o.Region = s3Details.Region
o.Credentials = aws.AnonymousCredentials{}
o.UsePathStyle = true
})
}
I am using aws.AnonymousCredentials{}
(as our access is credential-less) but this is only to be used for unsigned requests. I cannot use NewStaticCredentialsProvider
with empty values for AccessKeyID
and/or SecretAccessKey
as this will throw a StaticCredentialsEmptyError
during the Retrieve()
. Adding dummy credentials will throw an error that they are not on record. I am assuming that this is the cause of my 403 AccessDenied
.
How do I sign requests without providing credentials in the Go SDK? Is it even possible? In the boto3 Python library this works fine.
答案1
得分: 1
首先,我强烈建议您使用Go的AWS SDK的v2版本。我将在这里展示我目前如何做到这一点。
首先,我获取用于此代码的AWS配置(仅显示相关部分):
cfg, err := config.LoadDefaultConfig(context.TODO())
if err != nil {
Log.Fatal(err)
}
这里使用的包是github.com/aws/aws-sdk-go-v2/config
。
然后,我实例化一个s3Client来与AWS S3服务进行通信:
s3Client := s3.NewFromConfig(*cfg)
在这里,我们使用的是github.com/aws/aws-sdk-go-v2/service/s3
包。最后,要发布您的对象,您需要运行此代码:
input := &s3.PutObjectInput{
Key: aws.String("test"),
Bucket: aws.String("test"),
Body: bytes.NewReader([]byte("test")),
ACL: types.ObjectCannedACLPrivate,
}
if _, err := s3Client.PutObject(context.TODO(), input); err != nil {
return "", fmt.Errorf("fn UploadFile %w", err)
}
这里使用的新包是github.com/aws/aws-sdk-go-v2/service/s3/types
。
这段代码是一个简化版,但您应该能够实现您所需的功能。此外,更新SDK版本应该需要很少的时间,如果您需要处理大型代码库,您可以同时依赖于两个版本。
如果有帮助,请告诉我!
编辑
我通过使用aws.AnonymousCredentials{}
选项更新了我的解决方案。现在,我成功地使用这些选项将文件上传到了一个s3存储桶中。下面是完整的解决方案:
package main
import (
"bytes"
"context"
"crypto/tls"
"net/http"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/s3"
"github.com/aws/aws-sdk-go-v2/service/s3/types"
)
func GetAwsConfig() (*aws.Config, error) {
cfg, err := config.LoadDefaultConfig(context.TODO(),
// config.WithClientLogMode(aws.LogRequestWithBody|aws.LogResponseWithBody),
config.WithRegion("eu-west-1"),
config.WithHTTPClient(&http.Client{Transport: &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}}),
config.WithEndpointResolverWithOptions(
aws.EndpointResolverWithOptionsFunc(func(service, region string, options ...interface{}) (aws.Endpoint, error) {
return aws.Endpoint{
PartitionID: "aws",
URL: "http://127.0.0.1:4566",
SigningRegion: "eu-west-1",
HostnameImmutable: true,
}, nil
}),
))
if err != nil {
return nil, err
}
return &cfg, err
}
func main() {
cfg, _ := GetAwsConfig()
s3Client := s3.NewFromConfig(*cfg, func(o *s3.Options) {
o.Credentials = aws.AnonymousCredentials{}
})
if _, err := s3Client.PutObject(context.Background(), &s3.PutObjectInput{
Bucket: aws.String("mybucket"),
Key: aws.String("myfile"),
Body: bytes.NewReader([]byte("hello")),
ACL: types.ObjectCannedACLPrivate,
}); err != nil {
panic(err)
}
}
在运行代码之前,您需要创建存储桶。我使用了以下命令:
aws --endpoint-url=http://localhost:4566 s3 mb s3://mybucket
通过这样做,您可以将文件上传到mybucket
s3存储桶中。要检查文件是否存在,您可以使用以下命令:
aws --endpoint-url=http://localhost:4566 s3 ls s3://mybucket --recursive --human-readable
希望这能帮助您解决问题!
英文:
First of all, I'll strongly suggest you use the v2 of the AWS SDK of Go. I'll present here how I do this so far.
First, I get the AWS config to use with this code (only relevant parts are shown):
cfg, err := config.LoadDefaultConfig(context.TODO())
if err != nil {
Log.Fatal(err)
}
Here the package used is github.com/aws/aws-sdk-go-v2/config
.
Then, I instantiate an s3Client to use for contacting AWS S3 service:
s3Client := s3.NewFromConfig(*cfg)
Here, we use this package github.com/aws/aws-sdk-go-v2/service/s3
. Finally, to post your object you have to run this code:
input := &s3.PutObjectInput{
Key: aws.String("test"),
Bucket: aws.String("test"),
Body: bytes.NewReader([]byte("test")),
ACL: types.ObjectCannedACLPrivate,
}
if _, err := s3Client.PutObject(context.TODO(), input); err != nil {
return "", fmt.Errorf("fn UploadFile %w", err)
}
The new package used here is github.com/aws/aws-sdk-go-v2/service/s3/types
.
This code is a simplification but you should able to achieve what you need. Furthermore, it should take very little time to update the version of the SDK and you can rely on both of them simultaneously if you've to work with a huge codebase.
Let me know if this helps!
Edit
I updated my solution by using the aws.AnonymousCredentials{}
option. Now I was successfully able to upload a file into an s3 bucket with these options. Below you can find the entire solution:
package main
import (
"bytes"
"context"
"crypto/tls"
"net/http"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/s3"
"github.com/aws/aws-sdk-go-v2/service/s3/types"
)
func GetAwsConfig() (*aws.Config, error) {
cfg, err := config.LoadDefaultConfig(context.TODO(),
// config.WithClientLogMode(aws.LogRequestWithBody|aws.LogResponseWithBody),
config.WithRegion("eu-west-1"),
config.WithHTTPClient(&http.Client{Transport: &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}}),
config.WithEndpointResolverWithOptions(
aws.EndpointResolverWithOptionsFunc(func(service, region string, options ...interface{}) (aws.Endpoint, error) {
return aws.Endpoint{
PartitionID: "aws",
URL: "http://127.0.0.1:4566",
SigningRegion: "eu-west-1",
HostnameImmutable: true,
}, nil
}),
))
if err != nil {
return nil, err
}
return &cfg, err
}
func main() {
cfg, _ := GetAwsConfig()
s3Client := s3.NewFromConfig(*cfg, func(o *s3.Options) {
o.Credentials = aws.AnonymousCredentials{}
})
if _, err := s3Client.PutObject(context.Background(), &s3.PutObjectInput{
Bucket: aws.String("mybucket"),
Key: aws.String("myfile"),
Body: bytes.NewReader([]byte("hello")),
ACL: types.ObjectCannedACLPrivate,
}); err != nil {
panic(err)
}
}
Before running the code, you've to create the bucket. I used the below command:
aws --endpoint-url=http://localhost:4566 s3 mb s3://mybucket
Thanks to this you can upload the file into the mybucket
s3 bucket. To check for the file existence you can issue this command:
aws --endpoint-url=http://localhost:4566 s3 ls s3://mybucket --recursive --human-readable
Hope this helps in solving your issue!
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论