DynamoDBEvent生成属性值JSON。

huangapple go评论79阅读模式
英文:

DynamoDBEvent produces attribute value JSON

问题

我正在编写一个lambda函数,该函数接收来自DynamoDB表的流式数据。在解析出正确的记录后,我试图将其转换为JSON格式。目前,我是这样做的:

func LambdaHandler(ctx context.Context, request events.DynamoDBEvent) error {

    // ...

    // 这只是演示,不是真正的代码
    record := request.Records[0]

    data, err := events.NewMapAttribute(record.Change.NewImage).MarshalJSON()
	if err != nil {
        return err
	}

    // ...
}

问题是,这样生成的JSON数据如下所示:

{
  "M": {
    "action": { "N": "0" },
    "expiration": { "N":"0" },
    "id": { "S": "trades|v4|2023-02-08" },
    "order": { "N":"22947407" },
    "price": { "N":"96.139" },
    "sort_key": { "S":"22947407" },
    "stop_limit": { "N":"0" },
    "stop_loss": { "N":"96.7" },
    "symbol": { "S":"CADJPY" },
    "take_profit": { "N":"94.83" },
    "type": { "N":"5" },
    "type_filling": { "N":"0" },
    "type_time": { "N":"0" },
    "volume": { "N":"1" }
  }
}

如你所见,这个JSON数据的结构模仿了DynamoDB属性值的结构,但这不是我想要的。相反,我想生成一个如下所示的JSON数据:

{
  "action": 0,
  "expiration": 0,
  "id": "trades|v4|2023-02-08",
  "order": 22947407, 
  "price": 96.139,
  "sort_key": "22947407",
  "stop_limit": 0,
  "stop_loss": 96.7,
  "symbol": "CADJPY",
  "take_profit": 94.83,
  "type": 5,
  "type_filling": 0,
  "type_time": 0,
  "volume": 1
}

现在,我可以想到几种方法来实现这个目标:将record.Change.NewImage中的值硬编码到map[interface{}]中,然后使用json.Marshal进行编组,但我接收到的数据类型可能是几种不同的类型之一。我也可以使用反射来做同样的事情,但我不想花时间调试反射代码。Amazon是否提供了可用于此目的的功能?似乎应该有,但我找不到相关的信息。

英文:

I'm writing a lambda that takes in streamed data from a DynamoDB table. After parsing out the proper record, I'm trying to convert it to JSON. Currently, I'm doing this:

func LambdaHandler(ctx context.Context, request events.DynamoDBEvent) error {

    // ...

    // Not actual code, just for demonstration
    record = request.Records[0]

    data, err := events.NewMapAttribute(record.Change.NewImage).MarshalJSON()
	if err != nil {
        return err
	}

    // ...
}

The problem is that this produces a JSON payload that looks like this:

{
  "M": {
    "action": { "N": "0" },
    "expiration": { "N":"0" },
    "id": { "S": "trades|v4|2023-02-08" },
    "order": { "N":"22947407" },
    "price": { "N":"96.139" },
    "sort_key": { "S":"22947407" },
    "stop_limit": { "N":"0" },
    "stop_loss": { "N":"96.7" },
    "symbol": { "S":"CADJPY" },
    "take_profit": { "N":"94.83" },
    "type": { "N":"5" },
    "type_filling": { "N":"0" },
    "type_time": { "N":"0" },
    "volume": { "N":"1" }
  }
}

As you can see, this mimics the structure of the DynamoDB attribute value but this isn't what I want. Instead, I'm trying to generate a JSON payload that looks like this:

{
  "action": 0,
  "expiration": 0,
  "id": "trades|v4|2023-02-08",
  "order": 22947407, 
  "price": 96.139,
  "sort_key": "22947407",
  "stop_limit": 0,
  "stop_loss": 96.7,
  "symbol": "CADJPY",
  "take_profit": 94.83,
  "type": 5,
  "type_filling": 0,
  "type_time": 0,
  "volume": 1
}

Now, I can think of a couple ways to do that: hardcoding the values from record.Change.NewImage into a map[interface{}] and then marshalling that using json.Marshal, but the type of the payload I receive could be one of several different types. I could also use reflection to do the same thing, but I'd rather not spend the time debugging reflection code. Is there functionality available from Amazon to do this? It seems like there should be but I can't find anything.

答案1

得分: 0

我最终编写了一个函数,基本上满足了我的需求。它将数值转换为字符串,并生成我需要的JSON负载:

// AttributesToJSON 尝试将DynamoDB属性值映射转换为格式正确的JSON字符串
func AttributesToJSON(attrs map[string]events.DynamoDBAttributeValue) ([]byte, error) {

	// 尝试将DynamoDB属性值映射转换为map[string]interface{}
	// 如果失败,则返回错误
	keys := make([]string, 0)
	mapping, err := toJSONInner(attrs, keys...)
	if err != nil {
		return nil, err
	}

	// 尝试将此映射转换为JSON并返回结果
	return json.Marshal(mapping)
}

// 将结构体转换为JSON字段映射的辅助函数
func toJSONInner(attrs map[string]events.DynamoDBAttributeValue, keys ...string) (map[string]interface{}, error) {
	jsonStr := make(map[string]interface{})
	for key, attr := range attrs {

		// 尝试将字段转换为JSON映射;如果失败,则返回错误
		casted, err := toJSONField(attr, append(keys, key)...)
		if err != nil {
			return nil, err
		}

		// 将字段设置为映射中的关联键
		jsonStr[key] = casted
	}

	return jsonStr, nil
}

// 将特定的DynamoDB属性值转换为其JSON值等效的辅助函数
func toJSONField(attr events.DynamoDBAttributeValue, keys ...string) (interface{}, error) {
	attrType := attr.DataType()
	switch attrType {
	case events.DataTypeBinary:
		return attr.Binary(), nil
	case events.DataTypeBinarySet:
		return attr.BinarySet(), nil
	case events.DataTypeBoolean:
		return attr.Boolean(), nil
	case events.DataTypeList:

		// 从属性值获取项目列表
		list := attr.List()

		// 尝试将列表中的每个项目转换为JSON映射
		data := make([]interface{}, len(list))
		for i, item := range list {

			// 尝试将字段映射到JSON映射;如果失败,则返回错误
			casted, err := toJSONField(item, keys...)
			if err != nil {
				return nil, err
			}

			// 将该索引处的值设置为我们生成的映射
			data[i] = casted
		}

		// 返回我们创建的列表
		return data, nil
	case events.DataTypeMap:
		return toJSONInner(attr.Map(), keys...)
	case events.DataTypeNull:
		return nil, nil
	case events.DataTypeNumber:
		return attr.Number(), nil
	case events.DataTypeNumberSet:
		return attr.NumberSet(), nil
	case events.DataTypeString:
		return attr.String(), nil
	case events.DataTypeStringSet:
		return attr.StringSet(), nil
	default:
		return nil, fmt.Errorf("位于%s的属性具有未知的属性类型%d",
			strings.Join(keys, "."), attrType)
	}
}

该代码通过迭代顶层映射中的每个键和值,并将值转换为interface{},然后将结果转换为JSON。在这种情况下,interface{}可以是[]byte[][]bytestring[]stringboolinterface{}map[string]interface{},具体取决于属性值的类型。

英文:

I ended up writing a function that does more or less what I need it to. This will write numeric values as strings, but otherwise will generate the JSON payload I'm looking for:

// AttributesToJSON attempts to convert a mapping of DynamoDB attribute values to a properly-formatted JSON string
func AttributesToJSON(attrs map[string]events.DynamoDBAttributeValue) ([]byte, error) {
// Attempt to map the DynamoDB attribute value mapping to a map[string]interface{}
// If this fails then return an error
keys := make([]string, 0)
mapping, err := toJSONInner(attrs, keys...)
if err != nil {
return nil, err
}
// Attempt to convert this mapping to JSON and return the result
return json.Marshal(mapping)
}
// Helper function that converts a struct to JSON field-mapping
func toJSONInner(attrs map[string]events.DynamoDBAttributeValue, keys ...string) (map[string]interface{}, error) {
jsonStr := make(map[string]interface{})
for key, attr := range attrs {
// Attempt to convert the field to a JSON mapping; if this fails then return an error
casted, err := toJSONField(attr, append(keys, key)...)
if err != nil {
return nil, err
}
// Set the field to its associated key in our mapping
jsonStr[key] = casted
}
return jsonStr, nil
}
// Helper function that converts a specific DynamoDB attribute value to its JSON value equivalent
func toJSONField(attr events.DynamoDBAttributeValue, keys ...string) (interface{}, error) {
attrType := attr.DataType()
switch attrType {
case events.DataTypeBinary:
return attr.Binary(), nil
case events.DataTypeBinarySet:
return attr.BinarySet(), nil
case events.DataTypeBoolean:
return attr.Boolean(), nil
case events.DataTypeList:
// Get the list of items from the attribute value
list := attr.List()
// Attempt to convert each item in the list to a JSON mapping
data := make([]interface{}, len(list))
for i, item := range list {
// Attempt to map the field to a JSON mapping; if this fails then return an error
casted, err := toJSONField(item, keys...)
if err != nil {
return nil, err
}
// Set the value at this index to the mapping we generated
data[i] = casted
}
// Return the list we created
return data, nil
case events.DataTypeMap:
return toJSONInner(attr.Map(), keys...)
case events.DataTypeNull:
return nil, nil
case events.DataTypeNumber:
return attr.Number(), nil
case events.DataTypeNumberSet:
return attr.NumberSet(), nil
case events.DataTypeString:
return attr.String(), nil
case events.DataTypeStringSet:
return attr.StringSet(), nil
default:
return nil, fmt.Errorf("Attribute at %s had unknown attribute type of %d",
strings.Join(keys, "."), attrType)
}
}

This code works by iterating over each key and value in the top-level mapping, and converting the value to an interface{} and then converting the result to JSON. In this case, the interface{} could be a []byte, [][]byte, string, []string, bool, interface{} or map[string]interface{} depending on the type of the attribute value.

huangapple
  • 本文由 发表于 2023年2月8日 11:18:43
  • 转载请务必保留本文链接:https://go.coder-hub.com/75381085.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定