Empty string for DynamoDB index key in Golang

huangapple go评论85阅读模式
英文:

Empty string for DynamoDB index key in Golang

问题

我一直在使用Protobuf来定义与DynamoDB通信时使用的对象。到目前为止,这些对象看起来像这样:

type Record struct {
	state         protoimpl.MessageState
	sizeCache     protoimpl.SizeCache
	unknownFields protoimpl.UnknownFields

	Id               string        `protobuf:"bytes,2,opt,name=id,json=id,proto3" dynamodbav:"id,omitempty" json:"id,omitempty"`
	Name             string        `protobuf:"bytes,3,opt,name=name,json=name,proto3" dynamodbav:"name,omitempty" json:"name,omitempty"`
	OrgId            string        `protobuf:"bytes,4,opt,name=org_id,json=orgId,proto3" dynamodbav:"org_id,omitempty" json:"org_id,omitempty"`
	AccountId        string        `protobuf:"bytes,7,opt,name=account_id,json=accountId,proto3" dynamodbav:"account_id,omitempty" json:"account_id,omitempty"`
	Address          string        `protobuf:"bytes,9,opt,name=address,proto3" dynamodbav:"address,omitempty" json:"address,omitempty"`
	BillingTitle     string        `protobuf:"bytes,10,opt,name=billing_title,json=billingTitle,proto3" dynamodbav:"billing_title,omitempty" json:"billing_title,omitempty"`
	Language         string        `protobuf:"bytes,11,opt,name=language,proto3" dynamodbav:"language,omitempty" json:"language,omitempty"`
	Personal         string        `protobuf:"bytes,12,opt,name=personal,proto3" dynamodbav:"personal,omitempty" json:"personal,omitempty"`
	Phonenumber      string        `protobuf:"bytes,13,opt,name=phonenumber,proto3" dynamodbav:"phonenumber,omitempty" json:"phonenumber,omitempty"`
	Postalcode       string        `protobuf:"bytes,14,opt,name=postalcode,proto3" dynamodbav:"postalcode,omitempty" json:"postalcode,omitempty"`
	ProjectId        string        `protobuf:"bytes,15,opt,name=project_id,json=projectId,proto3" dynamodbav:"project_id,omitempty" json:"project_id,omitempty"`
	Remarks          string        `protobuf:"bytes,16,opt,name=remarks,proto3" dynamodbav:"remarks,omitempty" json:"remarks,omitempty"`
}

我要将这些对象添加到的表中有一个全局二级索引,其中project_id是排序键,但这个字段并不总是有值。所以,当我执行PutObject操作时,代码如下:

record := Record {
    Id: "A000",
    Name: "Some Record",
    OrgId: "O000",
}

attrs, err := dynamodbattribute.MarshalMap(&record)
if err != nil {
    panic(err)
}

if _, err := conn.PutItem(&dynamodb.PutItemInput{
    TableName: aws.String(tableName),
    Item: attrs,
}); err != nil {
    panic(err)
}

这段代码不会引发错误,并且我可以在DynamoDB中看到该值。然而,当我从所有的dynamodbav标签中删除omitempty子句后,我注意到现在会出现以下错误:

ValidationException: Invalid attribute value type
        status code: 400, request id: 6a626232-fcd4-4999-afe4-3df5769ce1b2

经过进一步调查,我发现对象被序列化为:

map[account_id:{
  NULL: true
} address:{
  NULL: true,
} billing_title:{
  NULL: true
} id:{
  S: "A000"
} name:{
  S: "Some Record"
} language:{
  NULL: true
} org_id:{
  S: "O000"
} personal:{
  NULL: true
} phonenumber:{
  NULL: true
} postalcode:{
  NULL: true
} project_id:{
  NULL: true
} remarks:{
  NULL: true
}

当不包含omitempty时,但序列化为:

map[id:{
  S: "A000"
} name:{
  S: "Some Record"
} org_id:{
  S: "O000"
}

当包含omitempty时。

我希望得到前者的默认值,而不是NULL: true,但我没有看到任何序列化选项可以实现这一点。有没有一种方法可以在不实现DynamoDB marshaler接口的情况下实现这一点?

英文:

I've been using Protobuf for object definitions to be used when communicating with DynamoDB. Until now, such objects would look like this:

type Record struct {
	state         protoimpl.MessageState
	sizeCache     protoimpl.SizeCache
	unknownFields protoimpl.UnknownFields

	Id               string        `protobuf:"bytes,2,opt,name=id,json=id,proto3" dynamodbav:"id,omitempty" json:"id,omitempty"`
	Name             string        `protobuf:"bytes,3,opt,name=name,json=name,proto3" dynamodbav:"name,omitempty" json:"name,omitempty"`
	OrgId            string        `protobuf:"bytes,4,opt,name=org_id,json=orgId,proto3" dynamodbav:"org_id,omitempty" json:"org_id,omitempty"`
	AccountId        string        `protobuf:"bytes,7,opt,name=account_id,json=accountId,proto3" dynamodbav:"account_id,omitempty" json:"account_id,omitempty"`
	Address          string        `protobuf:"bytes,9,opt,name=address,proto3" dynamodbav:"address,omitempty" json:"address,omitempty"`
	BillingTitle     string        `protobuf:"bytes,10,opt,name=billing_title,json=billingTitle,proto3" dynamodbav:"billing_title,omitempty" json:"billing_title,omitempty"`
	Language         string        `protobuf:"bytes,11,opt,name=language,proto3" dynamodbav:"language,omitempty" json:"language,omitempty"`
	Personal         string        `protobuf:"bytes,12,opt,name=personal,proto3" dynamodbav:"personal,omitempty" json:"personal,omitempty"`
	Phonenumber      string        `protobuf:"bytes,13,opt,name=phonenumber,proto3" dynamodbav:"phonenumber,omitempty" json:"phonenumber,omitempty"`
	Postalcode       string        `protobuf:"bytes,14,opt,name=postalcode,proto3" dynamodbav:"postalcode,omitempty" json:"postalcode,omitempty"`
	ProjectId        string        `protobuf:"bytes,15,opt,name=project_id,json=projectId,proto3" dynamodbav:"project_id,omitempty" json:"project_id,omitempty"`
	Remarks          string        `protobuf:"bytes,16,opt,name=remarks,proto3" dynamodbav:"remarks,omitempty" json:"remarks,omitempty"`
}

The table I'm adding these objects to has a global secondary index with project_id as the sort-key but this field isn't always populated. So, when I went to do a PutObject operation on the table, like this:

record := Record {
    Id: "A000",
    Name: "Some Record",
    OrgId: "O000"
}

attrs, err := dynamodbattribute.MarshalMap(&record)
if err != nil {
    panic(err)
}

if _, err := conn.PutItem(&dynamodb.PutItemInput{
    TableName: aws.String(tableName),
	Item: attrs
}); err != nil {
    panic(err)
}

this code wouldn't panic and I'd see the value in DynamoDB. However, after removing the omitempty clause from all the dynamodbav tags, I notice that this now fails with the following error:

ValidationException: Invalid attribute value type
        status code: 400, request id: 6a626232-fcd4-4999-afe4-3df5769ce1b2

After some further investigation, I see that the object is serialized as:

map[account_id:{
  NULL: true
} address:{
  NULL: true,
} billing_title:{
  NULL: true
} id:{
  S: "A000"
} name:{
  S: "Some Record"
} language:{
  NULL: true
} org_id:{
  S: "O000"
} personal:{
  NULL: true
} phonenumber:{
  NULL: true
} postalcode:{
  NULL: true
} project_id:{
  NULL: true
} remarks:{
  NULL: true
}

when omitempty is not included but serializes to:

map[id:{
  S: "A000"
} name:{
  S: "Some Record"
} org_id:{
  S: "O000"
}

when omitempty is included.

I'd like to get the former with default values rather than NULL: true but I don't see any serialization options for this. Is there a way to do this without implementing the DynamoDB marshaler interface?

答案1

得分: 2

经过一些调查,我找到了Marshal函数的源代码,在这里。根据这个源代码,有一个叫做NullEmptyString的字段,我可以设置它来告诉编码器,我希望将空字符串作为字符串发送,而不是作为空属性发送。然而,如果我设置了这个字段,我就不能直接使用MarshalMarshalMapMarshalList函数。所以,我复制了它们并编写了自己的代码:

func MarshalMap(in interface{}) (map[string]*dynamodb.AttributeValue, error) {
	av, err := getEncoder().Encode(in)
	if err != nil || av == nil || av.M == nil {
		return map[string]*dynamodb.AttributeValue{}, err
	}

	return av.M, nil
}

func getEncoder() *dynamodbattribute.Encoder {
	encoder := dynamodbattribute.NewEncoder()
	encoder.NullEmptyString = false
	return encoder
}
英文:

After some investigation, I found the source code for the Marshal function, here. According to this, there's a field called NullEmptyString which I can set to tell the encoder that I want empty strings to be sent as strings, rather than as null attributes. However, if I set this I cannot use Marshal, MarshalMap and MarshalList directly. So, I copied them and wrote my own:

func MarshalMap(in interface{}) (map[string]*dynamodb.AttributeValue, error) {
	av, err := getEncoder().Encode(in)
	if err != nil || av == nil || av.M == nil {
		return map[string]*dynamodb.AttributeValue{}, err
	}

	return av.M, nil
}

func getEncoder() *dynamodbattribute.Encoder {
	encoder := dynamodbattribute.NewEncoder()
	encoder.NullEmptyString = false
	return encoder
}

huangapple
  • 本文由 发表于 2021年11月8日 14:14:50
  • 转载请务必保留本文链接:https://go.coder-hub.com/69879277.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定