aws-sdk-go-v2 PutObject API 错误:AccessDenied

huangapple go评论89阅读模式
英文:

aws-sdk-go-v2 PutObject api error AccessDenied

问题

在我们的Staging环境中,我们可以无需凭证访问私有的S3存储桶。访问权限是授予各个Docker容器的。我正在尝试使用aws-sdk-go-v2 SDK库的PutObject方法上传文件,但是一直收到403 AccessDenied的API错误。

我的上传代码如下所示:

var uploadFileFunc = func(s3Details S3Details, key string, payload []byte, params MetadataParams) (*s3.PutObjectOutput, error) {
	client := getS3Client(s3Details)

	return client.PutObject(context.TODO(), &s3.PutObjectInput{
		Bucket:      aws.String(s3Details.Bucket),
		Key:         aws.String(key),
		Body:        bytes.NewReader(payload),
		ContentType: aws.String("text/xml"),
	})
}

func getS3Client(s3Details S3Details) *s3.Client {
	endpointResolver := aws.EndpointResolverWithOptionsFunc(func(service, region string, options ...interface{}) (aws.Endpoint, error) {
		if s3Details.EndpointUrl != "" {
			return aws.Endpoint{
				PartitionID:   "aws",
				URL:           s3Details.EndpointUrl,
				SigningRegion: s3Details.Region,
				SigningMethod: s3Details.SignatureVersion,
			}, nil
		}
		return aws.Endpoint{}, &aws.EndpointNotFoundError{}
	})

	cfg, _ := config.LoadDefaultConfig(context.TODO(),
		config.WithEndpointDiscovery(aws.EndpointDiscoveryEnabled),
		config.WithEndpointResolverWithOptions(endpointResolver))

	return s3.NewFromConfig(cfg, func(o *s3.Options) {
		o.Region = s3Details.Region
		o.Credentials = aws.AnonymousCredentials{}
		o.UsePathStyle = true
	})
}

我正在使用aws.AnonymousCredentials{}(因为我们是无凭证访问),但这只适用于未签名的请求。我不能使用NewStaticCredentialsProvider并将AccessKeyID和/或SecretAccessKey的值留空,因为这会在Retrieve()过程中引发StaticCredentialsEmptyError错误。添加虚拟凭证会引发错误,表示它们未记录。我认为这是导致我收到403 AccessDenied错误的原因。

在Go SDK中,如何在不提供凭证的情况下对请求进行签名?这是否可能?在Python的boto3库中,这个操作是可行的。

英文:

In our Staging environment, we have credential-less access to our private S3 buckets. Access is granted to individual Docker containers. I am trying to upload a file using PutObject using the aws-sdk-go-v2 SDK library, but I'm continually getting a 403 AccessDenied api error.

My upload code looks like this:

var uploadFileFunc = func(s3Details S3Details, key string, payload []byte, params MetadataParams) (*s3.PutObjectOutput, error) {
	client := getS3Client(s3Details)

	return client.PutObject(context.TODO(), &s3.PutObjectInput{
		Bucket:      aws.String(s3Details.Bucket),
		Key:         aws.String(key),
		Body:        bytes.NewReader(payload),
		ContentType: aws.String("text/xml"),
	})
}

func getS3Client(s3Details S3Details) *s3.Client {
	endpointResolver := aws.EndpointResolverWithOptionsFunc(func(service, region string, options ...interface{}) (aws.Endpoint, error) {
		if s3Details.EndpointUrl != "" {
			return aws.Endpoint{
				PartitionID:   "aws",
				URL:           s3Details.EndpointUrl,
				SigningRegion: s3Details.Region,
				SigningMethod: s3Details.SignatureVersion,
			}, nil
		}
		return aws.Endpoint{}, &aws.EndpointNotFoundError{}
	})

	cfg, _ := config.LoadDefaultConfig(context.TODO(),
		config.WithEndpointDiscovery(aws.EndpointDiscoveryEnabled),
		config.WithEndpointResolverWithOptions(endpointResolver))

	return s3.NewFromConfig(cfg, func(o *s3.Options) {
		o.Region = s3Details.Region
		o.Credentials = aws.AnonymousCredentials{}
		o.UsePathStyle = true
	})
}

I am using aws.AnonymousCredentials{} (as our access is credential-less) but this is only to be used for unsigned requests. I cannot use NewStaticCredentialsProvider with empty values for AccessKeyID and/or SecretAccessKey as this will throw a StaticCredentialsEmptyError during the Retrieve(). Adding dummy credentials will throw an error that they are not on record. I am assuming that this is the cause of my 403 AccessDenied.

How do I sign requests without providing credentials in the Go SDK? Is it even possible? In the boto3 Python library this works fine.

答案1

得分: 1

首先,我强烈建议您使用Go的AWS SDK的v2版本。我将在这里展示我目前如何做到这一点。
首先,我获取用于此代码的AWS配置(仅显示相关部分):

cfg, err := config.LoadDefaultConfig(context.TODO())
	if err != nil {
		Log.Fatal(err)
	}

这里使用的包是github.com/aws/aws-sdk-go-v2/config
然后,我实例化一个s3Client来与AWS S3服务进行通信:

s3Client := s3.NewFromConfig(*cfg)

在这里,我们使用的是github.com/aws/aws-sdk-go-v2/service/s3包。最后,要发布您的对象,您需要运行此代码:

input := &s3.PutObjectInput{
	Key:    aws.String("test"),
	Bucket: aws.String("test"),
	Body:   bytes.NewReader([]byte("test")),
	ACL:    types.ObjectCannedACLPrivate,
}

if _, err := s3Client.PutObject(context.TODO(), input); err != nil {
	return "", fmt.Errorf("fn UploadFile %w", err)
}

这里使用的新包是github.com/aws/aws-sdk-go-v2/service/s3/types
这段代码是一个简化版,但您应该能够实现您所需的功能。此外,更新SDK版本应该需要很少的时间,如果您需要处理大型代码库,您可以同时依赖于两个版本。
如果有帮助,请告诉我!

编辑

我通过使用aws.AnonymousCredentials{}选项更新了我的解决方案。现在,我成功地使用这些选项将文件上传到了一个s3存储桶中。下面是完整的解决方案:

package main

import (
	"bytes"
	"context"
	"crypto/tls"
	"net/http"

	"github.com/aws/aws-sdk-go-v2/aws"
	"github.com/aws/aws-sdk-go-v2/config"
	"github.com/aws/aws-sdk-go-v2/service/s3"
	"github.com/aws/aws-sdk-go-v2/service/s3/types"
)

func GetAwsConfig() (*aws.Config, error) {
	cfg, err := config.LoadDefaultConfig(context.TODO(),
		// config.WithClientLogMode(aws.LogRequestWithBody|aws.LogResponseWithBody),
		config.WithRegion("eu-west-1"),
		config.WithHTTPClient(&http.Client{Transport: &http.Transport{
			TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
		}}),
		config.WithEndpointResolverWithOptions(
			aws.EndpointResolverWithOptionsFunc(func(service, region string, options ...interface{}) (aws.Endpoint, error) {
				return aws.Endpoint{
					PartitionID:       "aws",
					URL:               "http://127.0.0.1:4566",
					SigningRegion:     "eu-west-1",
					HostnameImmutable: true,
				}, nil
			}),
		))
	if err != nil {
		return nil, err
	}
	return &cfg, err
}

func main() {
	cfg, _ := GetAwsConfig()
	s3Client := s3.NewFromConfig(*cfg, func(o *s3.Options) {
		o.Credentials = aws.AnonymousCredentials{}
	})

	if _, err := s3Client.PutObject(context.Background(), &s3.PutObjectInput{
		Bucket: aws.String("mybucket"),
		Key:    aws.String("myfile"),
		Body:   bytes.NewReader([]byte("hello")),
		ACL:    types.ObjectCannedACLPrivate,
	}); err != nil {
		panic(err)
	}
}

在运行代码之前,您需要创建存储桶。我使用了以下命令:
aws --endpoint-url=http://localhost:4566 s3 mb s3://mybucket
通过这样做,您可以将文件上传到mybucket s3存储桶中。要检查文件是否存在,您可以使用以下命令:
aws --endpoint-url=http://localhost:4566 s3 ls s3://mybucket --recursive --human-readable
希望这能帮助您解决问题!

英文:

First of all, I'll strongly suggest you use the v2 of the AWS SDK of Go. I'll present here how I do this so far.
First, I get the AWS config to use with this code (only relevant parts are shown):

cfg, err := config.LoadDefaultConfig(context.TODO())
	if err != nil {
		Log.Fatal(err)
	}

Here the package used is github.com/aws/aws-sdk-go-v2/config.
Then, I instantiate an s3Client to use for contacting AWS S3 service:

s3Client := s3.NewFromConfig(*cfg)

Here, we use this package github.com/aws/aws-sdk-go-v2/service/s3. Finally, to post your object you have to run this code:

input := &s3.PutObjectInput{
	Key:    aws.String("test"),
	Bucket: aws.String("test"),
	Body:   bytes.NewReader([]byte("test")),
	ACL:    types.ObjectCannedACLPrivate,
}

if _, err := s3Client.PutObject(context.TODO(), input); err != nil {
	return "", fmt.Errorf("fn UploadFile %w", err)
}

The new package used here is github.com/aws/aws-sdk-go-v2/service/s3/types.
This code is a simplification but you should able to achieve what you need. Furthermore, it should take very little time to update the version of the SDK and you can rely on both of them simultaneously if you've to work with a huge codebase.
Let me know if this helps!

Edit

I updated my solution by using the aws.AnonymousCredentials{} option. Now I was successfully able to upload a file into an s3 bucket with these options. Below you can find the entire solution:

package main

import (
	"bytes"
	"context"
	"crypto/tls"
	"net/http"

	"github.com/aws/aws-sdk-go-v2/aws"
	"github.com/aws/aws-sdk-go-v2/config"
	"github.com/aws/aws-sdk-go-v2/service/s3"
	"github.com/aws/aws-sdk-go-v2/service/s3/types"
)

func GetAwsConfig() (*aws.Config, error) {
	cfg, err := config.LoadDefaultConfig(context.TODO(),
		// config.WithClientLogMode(aws.LogRequestWithBody|aws.LogResponseWithBody),
		config.WithRegion("eu-west-1"),
		config.WithHTTPClient(&http.Client{Transport: &http.Transport{
			TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
		}}),
		config.WithEndpointResolverWithOptions(
			aws.EndpointResolverWithOptionsFunc(func(service, region string, options ...interface{}) (aws.Endpoint, error) {
				return aws.Endpoint{
					PartitionID:       "aws",
					URL:               "http://127.0.0.1:4566",
					SigningRegion:     "eu-west-1",
					HostnameImmutable: true,
				}, nil
			}),
		))
	if err != nil {
		return nil, err
	}
	return &cfg, err
}

func main() {
	cfg, _ := GetAwsConfig()
	s3Client := s3.NewFromConfig(*cfg, func(o *s3.Options) {
		o.Credentials = aws.AnonymousCredentials{}
	})

	if _, err := s3Client.PutObject(context.Background(), &s3.PutObjectInput{
		Bucket: aws.String("mybucket"),
		Key:    aws.String("myfile"),
		Body:   bytes.NewReader([]byte("hello")),
		ACL:    types.ObjectCannedACLPrivate,
	}); err != nil {
		panic(err)
	}
}

Before running the code, you've to create the bucket. I used the below command:
aws --endpoint-url=http://localhost:4566 s3 mb s3://mybucket
Thanks to this you can upload the file into the mybucket s3 bucket. To check for the file existence you can issue this command:
aws --endpoint-url=http://localhost:4566 s3 ls s3://mybucket --recursive --human-readable
Hope this helps in solving your issue!

huangapple
  • 本文由 发表于 2022年11月15日 20:41:55
  • 转载请务必保留本文链接:https://go.coder-hub.com/74445816.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定