英文:
Uploading a file with s3manager
问题
我正在尝试从我的网站下载一些文件并将它们上传到Amazon S3。
以下是我的代码:
import (
"log"
"net/http"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/s3/s3manager"
)
func main() {
// 发送GET请求以下载并上传到s3的图像
resp, err := http.Get("http://my-site.com/image.jpg")
if err != nil {
log.Fatal(err)
}
// 获取AWS凭证
creds := credentials.NewSharedCredentials("/Users/name/.aws/credentials", "default")
// 设置配置
config := &aws.Config{
Region: aws.String("US Standard"),
Credentials: creds,
}
// 创建新的AWS会话
sess := session.New(config)
// 创建新的s3manager客户端
uploader := s3manager.NewUploader(sess)
result, err := uploader.Upload(&s3manager.UploadInput{
Body: resp.Body,
Bucket: aws.String("mimi-movies"),
Key: aws.String("mimi.jpg"),
})
if err != nil {
log.Fatalln("上传失败", err)
}
log.Println("成功上传至", result.Location)
}
我一直收到一个运行时错误,指向空指针解引用。错误信息并没有提供太多帮助,但我认为可能与SSL有关。以下是完整的错误信息:
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x236ffe]
goroutine 1 [running]:
panic(0x4743c0, 0xc82000a0e0)
/usr/local/Cellar/go/1.6/libexec/src/runtime/panic.go:464 +0x3e6
github.com/aws/aws-sdk-go/service/s3.validateSSERequiresSSL(0xc820066380)
/Users/name/work/src/github.com/aws/aws-sdk-go/service/s3/sse.go:15 +0x17e
github.com/aws/aws-sdk-go/aws/request.(*HandlerList).Run(0xc820066478, 0xc820066380)
/Users/name/work/src/github.com/aws/aws-sdk-go/aws/request/handlers.go:136 +0xc3
github.com/aws/aws-sdk-go/aws/request.(*Request).Build(0xc820066380, 0x0, 0x0)
/Users/name/work/src/github.com/aws/aws-sdk-go/aws/request/request.go:189 +0x69
github.com/aws/aws-sdk-go/aws/request.(*Request).Sign(0xc820066380, 0x0, 0x0)
/Users/name/work/src/github.com/aws/aws-sdk-go/aws/request/request.go:210 +0x31
github.com/aws/aws-sdk-go/aws/request.(*Request).Send(0xc820066380, 0x0, 0x0)
/Users/name/work/src/github.com/aws/aws-sdk-go/aws/request/request.go:261 +0x6f4
github.com/aws/aws-sdk-go/service/s3/s3manager.(*uploader).singlePart(0xc8201520f0, 0x8bb980, 0xc82014a030, 0xc82000a140, 0x0, 0x0)
/Users/name/work/src/github.com/aws/aws-sdk-go/service/s3/s3manager/upload.go:449 +0x1fc
github.com/aws/aws-sdk-go/service/s3/s3manager.(*uploader).upload(0xc8201520f0, 0xc8201520f0, 0x0, 0x0)
/Users/name/work/src/github.com/aws/aws-sdk-go/service/s3/s3manager/upload.go:355 +0x563
github.com/aws/aws-sdk-go/service/s3/s3manager.Uploader.Upload(0x500000, 0x5, 0x0, 0x2710, 0xe00240, 0xc820156018, 0xc82016e000, 0x0, 0x0, 0x0, ...)
/Users/name/work/src/github.com/aws/aws-sdk-go/service/s3/s3manager/upload.go:329 +0x15a
main.main()
/Users/name/work/src/gitlab.com/user/project/test/main.go:39 +0x59f
exit status 2
希望这可以帮助你解决问题。
英文:
I'm trying to download a few files from my website and upload them to amazons3.
Here is my code:
import (
"log"
"net/http"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/s3/s3manager"
)
func main() {
// Make a GET request to an image to be downloaded and uploaded to s3
resp, err := http.Get("http://my-site.com/image.jpg")
if err != nil {
log.Fatal(err)
}
// Get the AWS credentials
creds := credentials.NewSharedCredentials("/Users/name/.aws/credentials", "default")
// Set up config
config := &aws.Config{
Region: aws.String("US Standard"),
Credentials: creds,
}
// Set up a new AWS session
sess := session.New(config)
// Set up a new s3manager client
uploader := s3manager.NewUploader(sess)
result, err := uploader.Upload(&s3manager.UploadInput{
Body: resp.Body,
Bucket: aws.String("mimi-movies"),
Key: aws.String("mimi.jpg"),
})
if err != nil {
log.Fatalln("Failed to upload", err)
}
log.Println("Successfully uploaded to", result.Location)
}
I keep getting a run time error of a nil pointer dereference the error doesn't help much but for some reason I think it might have to do with ssl but here is the entire error:
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x236ffe]
goroutine 1 [running]:
panic(0x4743c0, 0xc82000a0e0)
/usr/local/Cellar/go/1.6/libexec/src/runtime/panic.go:464 +0x3e6
github.com/aws/aws-sdk-go/service/s3.validateSSERequiresSSL(0xc820066380)
/Users/name/work/src/github.com/aws/aws-sdk-go/service/s3/sse.go:15 +0x17e
github.com/aws/aws-sdk-go/aws/request.(*HandlerList).Run(0xc820066478, 0xc820066380)
/Users/name/work/src/github.com/aws/aws-sdk-go/aws/request/handlers.go:136 +0xc3
github.com/aws/aws-sdk-go/aws/request.(*Request).Build(0xc820066380, 0x0, 0x0)
/Users/name/work/src/github.com/aws/aws-sdk-go/aws/request/request.go:189 +0x69
github.com/aws/aws-sdk-go/aws/request.(*Request).Sign(0xc820066380, 0x0, 0x0)
/Users/name/work/src/github.com/aws/aws-sdk-go/aws/request/request.go:210 +0x31
github.com/aws/aws-sdk-go/aws/request.(*Request).Send(0xc820066380, 0x0, 0x0)
/Users/name/work/src/github.com/aws/aws-sdk-go/aws/request/request.go:261 +0x6f4
github.com/aws/aws-sdk-go/service/s3/s3manager.(*uploader).singlePart(0xc8201520f0, 0x8bb980, 0xc82014a030, 0xc82000a140, 0x0, 0x0)
/Users/name/work/src/github.com/aws/aws-sdk-go/service/s3/s3manager/upload.go:449 +0x1fc
github.com/aws/aws-sdk-go/service/s3/s3manager.(*uploader).upload(0xc8201520f0, 0xc8201520f0, 0x0, 0x0)
/Users/name/work/src/github.com/aws/aws-sdk-go/service/s3/s3manager/upload.go:355 +0x563
github.com/aws/aws-sdk-go/service/s3/s3manager.Uploader.Upload(0x500000, 0x5, 0x0, 0x2710, 0xe00240, 0xc820156018, 0xc82016e000, 0x0, 0x0, 0x0, ...)
/Users/name/work/src/github.com/aws/aws-sdk-go/service/s3/s3manager/upload.go:329 +0x15a
main.main()
/Users/name/work/src/gitlab.com/user/project/test/main.go:39 +0x59f
exit status 2
答案1
得分: 4
如@user3666882所提到的,aws-go-sdk中存在一个错误,你应该检查一下你的区域。
在你的代码中,你已经编写了获取AWS凭证的代码。
// 获取AWS凭证
creds := credentials.NewSharedCredentials("/Users/name/.aws/credentials", "default")
你可以尝试在终端中执行以下命令:
aws configure
并配置你的aws-sdk。
否则,你可以使用Minio。但是你必须将你的ACCESS_KEY_ID和SECRET_ACCESS_KEY放入你的代码中,这对于生产环境来说是非常不好的。
s3Client, err := minio.New("s3.amazonaws.com", "YOUR-ACCESSKEYID", "YOUR-SECRETACCESSKEY", false)
另外,你可以使用外部API Viper 来隐藏这些密钥。但我建议你使用官方的AWS SDK for Go。
英文:
As @user3666882 mentioned there's bug in aws-go-sdk and you should probably check your region.
In your code, you have written code to get AWS Credential.
// Get the AWS credentials
creds := credentials.NewSharedCredentials("/Users/name/.aws/credentials", "default")
You might try this in your terminal :
> aws configure
And configure your aws-sdk.
Else you can use Minio. But your have to put your ACCESS_KEY_ID and SECRET_ACCESS_KEY in your code, which is very bad for production use.
s3Client, err := minio.New("s3.amazonaws.com", "YOUR-ACCESSKEYID", "YOUR-SECRETACCESSKEY", false)
Also, you can use external API Viper to hide these keys. But I recommend you to use official AWS SDK for Go.
答案2
得分: 2
有一个 aws-go-sdk 的 bug,但解决方法是将区域更改为 "us-east-1"
// 设置配置
config := &aws.Config{
Region: aws.String("us-east-1"),
Credentials: creds,
}
英文:
There is a bug on the aws-go-sdk but the solution is to change the region to "us-east-1"
// Set up config
config := &aws.Config{
Region: aws.String("us-east-1"),
Credentials: creds,
}
答案3
得分: 0
你可以选择使用Minio-go客户端库,它是开源的,并且与AWS S3兼容。
这里是一个简单的示例来上传对象
package main
import (
"log"
"os"
"github.com/minio/minio-go"
)
func main() {
// 注意:YOUR-ACCESSKEYID、YOUR-SECRETACCESSKEY、my-testfile、my-bucketname和my-objectname是虚拟值,请将它们替换为实际值。
// 默认情况下,请求始终是安全的(HTTPS)。设置insecure=true以启用不安全的(HTTP)访问。
// 这个布尔值是New()的最后一个参数。
// New返回一个兼容Amazon S3的客户端对象。API兼容性(v2或v4)会根据Endpoint值自动确定。
s3Client, err := minio.New("s3.amazonaws.com", "YOUR-ACCESSKEYID", "YOUR-SECRETACCESSKEY", false)
if err != nil {
log.Fatalln(err)
}
object, err := os.Open("my-testfile")
if err != nil {
log.Fatalln(err)
}
defer object.Close()
n, err := s3Client.PutObject("my-bucketname", "my-objectname", object, "application/octet-stream")
if err != nil {
log.Fatalln(err)
}
log.Println("已上传", "my-objectname", "大小为:", n, "成功.")
}
希望对你有帮助。
免责声明:我为Minio工作。
英文:
You could have alternatively used Minio-go client libraries, its Open Source & compatible with AWS S3.
Here is a simple example to Put an object
<pre>
package main
import (
"log"
"os"
"github.com/minio/minio-go"
)
func main() {
// Note: YOUR-ACCESSKEYID, YOUR-SECRETACCESSKEY, my-testfile, my-bucketname and
// my-objectname are dummy values, please replace them with original values.
// Requests are always secure (HTTPS) by default. Set insecure=true to enable insecure (HTTP) access.
// This boolean value is the last argument for New().
// New returns an Amazon S3 compatible client object. API compatibility (v2 or v4) is automatically
// determined based on the Endpoint value.
s3Client, err := minio.New("s3.amazonaws.com", "YOUR-ACCESSKEYID", "YOUR-SECRETACCESSKEY", false)
if err != nil {
log.Fatalln(err)
}
object, err := os.Open("my-testfile")
if err != nil {
log.Fatalln(err)
}
defer object.Close()
n, err := s3Client.PutObject("my-bucketname", "my-objectname", object, "application/octet-stream")
if err != nil {
log.Fatalln(err)
}
log.Println("Uploaded", "my-objectname", " of size: ", n, "Successfully.")
}
Status API Training Shop Blog About
</pre>
Hope it helps.
Disclaimer: I work for Minio
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论