Upload a struct or object to S3 bucket using GoLang?

huangapple go评论96阅读模式
英文:

Upload a struct or object to S3 bucket using GoLang?

问题

我正在使用GoLang中的AWS S3 SDK,尝试上传和下载到不同的存储桶。我想知道是否有一种更简单的方法可以直接将结构体或对象上传到存储桶中?

我有一个表示事件的结构体:

type Event struct {
	ID               string
	ProcessID        string                 
	TxnID            string
    Inputs           map[string]interface{}                      
}

我想将其上传到S3存储桶中。但是我在文档中找到的代码只适用于上传字符串。

func Save(client S3Client, T interface{}, key string) bool {
    svc := client.S3clientObject
    input := &s3.PutObjectInput{
        Body: aws.ReadSeekCloser(strings.NewReader("testing this one")),
        Bucket: aws.String(GetS3Bucket()),
        Key: aws.String(GetObjectKey(T, key)),
        Metadata: map[string]*string{
            "metadata1": aws.String("value1"),
            "metadata2": aws.String("value2"),
        },
    }

这段代码成功将一个基本文件上传到S3存储桶中,当打开时只会读取"testing this one"。有没有一种方法可以上传一个对象而不仅仅是一个字符串值?

由于我对Go和S3都不太熟悉,所以任何帮助都将不胜感激。

编辑

这是我用于Get函数的代码:

func GetIt(client S3Client, T interface{}, key string) interface{} {
    svc := client.S3clientObject
    s3Key := GetObjectKey(T, key)
    resp, err := svc.GetObject(&s3.GetObjectInput{
        Bucket: aws.String(GetS3Bucket()),
        Key: aws.String(s3Key),
    })
    if err != nil {
        fmt.Println(err)
        return err
    }    
    result := json.NewDecoder(resp.Body).Decode(&T)
    fmt.Println(result) 
    return json.NewDecoder(resp.Body).Decode(&T)
}
func main() {
    client := b.CreateS3Client()
    event := b.CreateEvent()
    GetIt(client, event, key)
}
英文:

I am working with the AWS S3 SDK in GoLang, playing with uploads and downloads to various buckets. I am wondering if there is a simpler way to upload structs or objects directly to the bucket?

I have a struct representing an event:

type Event struct {
	ID               string
	ProcessID        string                 
	TxnID            string
    Inputs           map[string]interface{}                      
}

That I would like to upload into the S3 bucket. But the code that I found in the documentation only works for uploading strings.

func Save(client S3Client, T interface{}, key string) bool {
    svc := client.S3clientObject
    input := &s3.PutObjectInput{
        Body: aws.ReadSeekCloser(strings.NewReader("testing this one")),
        Bucket: aws.String(GetS3Bucket()),
        Key: aws.String(GetObjectKey(T, key)),
        Metadata: map[string]*string{
            "metadata1": aws.String("value1"),
            "metadata2": aws.String("value2"),
        },

    }

This is successful in uploading a basic file to the S3 bucket that when opened simply reads "testing this one". Is there a way to upload to the bucket so that it is uploading an object rather than simply just a string value??

Any help is appreciated as I am new to Go and S3.

edit

This is the code I'm using for the Get function:

func GetIt(client S3Client, T interface{}, key string) interface{} {
    svc := client.S3clientObject
    s3Key := GetObjectKey(T, key)
    resp, err := svc.GetObject(&s3.GetObjectInput{
        Bucket: aws.String(GetS3Bucket()),
        Key: aws.String(s3Key),
    })
    if err != nil {
        fmt.Println(err)
        return err
    }    
    result := json.NewDecoder(resp.Body).Decode(&T)
    fmt.Println(result) 
    return json.NewDecoder(resp.Body).Decode(&T)
}
func main() {
    client := b.CreateS3Client()
    event := b.CreateEvent()
    GetIt(client, event, key)
}

答案1

得分: 2

将值编码为字节并上传字节。以下是将值编码为JSON字节的方法:

func Save(client S3Client, value interface{}, key string) error {

    p, err := json.Marshal(value)
    if err != nil {
        return err
    }
    input := &s3.PutObjectInput{
        Body:   aws.ReadSeekCloser(bytes.NewReader(p)),
        
    }
    
}

使用要上传的值调用Save:

value := &Event{ID: "an id", }
err := Save(, value, )
if err != nil {
  // 处理错误
}

有许多可能的编码格式,包括gobxmljsonmsgpack等。最佳的编码格式取决于您的应用程序要求。

在获取对象时进行反向过程:

func GetIt(client S3Client, T interface{}, key string) error {
    svc := client.S3clientObject
    resp, err := svc.GetObject(&s3.GetObjectInput{
        Bucket: aws.String(GetS3Bucket()),
        Key: aws.String(key),
    })
    if err != nil {
        return err
    }    
    return json.NewDecoder(resp.Body).Decode(T)
}

使用指向目标值的指针调用GetIt:

var value model.Event
err := GetIt(client, &value, key)
if err != nil {
    // 处理错误
}
fmt.Println(value) // 打印解码后的值。
英文:

Encode the value as bytes and upload the bytes. Here's how to encode the value as JSON bytes:

func Save(client S3Client, value interface{}, key string) error {

    p, err := json.Marshal(value)
    if err != nil {
	    return err
	}
	input := &s3.PutObjectInput{
		Body:   aws.ReadSeekCloser(bytes.NewReader(p)),
		…
	}
	…
}

Call Save with the value you want to upload:

value := &Event{ID: "an id", …}
err := Save(…, value, …)
if err != nil {
  // handle error
}

There are many possible including including gob, xml and json, msgpack, etc. The best encoding format will depend on your application requirements.

Reverse the process when getting an object:

func GetIt(client S3Client, T interface{}, key string) error {
    svc := client.S3clientObject
    resp, err := svc.GetObject(&s3.GetObjectInput{
        Bucket: aws.String(GetS3Bucket()),
        Key: aws.String(key),
    })
    if err != nil {
        return err
    }    
    return json.NewDecoder(resp.Body).Decode(T)
}

Call GetIt with a pointer to the destination value:

var value model.Event
err := GetIt(client, &value, key)
if err != nil {
    // handle error
}
fmt.Println(value) // prints the decoded value.

答案2

得分: 1

这里引用的示例显示,S3允许您上传任何实现io.Reader接口的内容。该示例使用strings.NewReader语法创建一个io.Reader,它知道如何向调用者提供指定的字符串。根据AWS的说法,您的任务是弄清楚如何将需要存储的内容适应为io.Reader

您可以直接将字节以JSON编码的方式存储,如下所示:

package main

import (
	"bytes"
	"encoding/json"
)

type Event struct {
	ID               string
	ProcessID        string
	TxnID            string
	Inputs           map[string]interface{}
}

func main() {
	// 准备写入对象
	event := Event{
		ID:        "123",
		ProcessID: "456",
		TxnID:     "789",
		Inputs: map[string]interface{}{
			"key1": "value1",
			"key2": "value2",
		},
	}

	// 将对象转换为字节
	b, err := json.Marshal(event)
	if err != nil {
		return
	}

	// 将字节转换为io.Reader
	reader := bytes.NewReader(b)
}

将这个reader传递给aws.ReadSeekCloser(...)函数即可。

英文:

The example cited here shows that S3 allows you to upload anything that implements the io.Reader interface. The example is using the strings.NewReader syntax create a io.Reader that knows how to provide the specified string to the caller. Your job (according to AWS here) is to figure out how to adapt whatever you need to store into an io.Reader.

You can store the bytes directly JSON encoded like this

package main

import (
	"bytes"
	"encoding/json"
)

type Event struct {
	ID               string
	ProcessID        string
	TxnID            string
	Inputs           map[string]interface{}
}

func main() {
	// To prepare the object for writing
	b, err := json.Marshal(event)
	if err != nil {
		return
	}
	// pass this reader into aws.ReadSeekCloser(...)
	reader := bytes.NewReader(b)
}

huangapple
  • 本文由 发表于 2021年6月10日 06:10:14
  • 转载请务必保留本文链接:https://go.coder-hub.com/67912295.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定