strconv.Unquote在Go中不会删除转义的JSON字符,导致unmarshal失败。

huangapple go评论104阅读模式
英文:

strconv.Unquote does not remove escaped json characters, unmarshall fails in Go

问题

我正在解析一个相当简单的 JSON,其中日期格式字符串中的斜杠在响应中被转义。

然而,当尝试解组字符串时,它会失败并显示“无效语法”错误。
所以我搜索了一下,我们应该使用 strconv.Unquote 来首先替换转义字符。我这样做了,现在 unquote 函数在遇到“未知转义”错误时失败。然而,JSON RFC 8259 显示这是一个有效的情况。那么为什么有效的 JSON 会导致 Go 的解组器失败呢?

这是简单的 JSON:

{
   "created_at": "6\/30\/2022 21:51:49"
}
    var dst string
	err := json.Unmarshal([]byte("6\/30\/2022 21:51:49"), dst) // 这会导致无效引号错误
	fmt.Println(dst, err)

	s, err := strconv.Unquote(`"6\/30\/2022 21:51:49"`) // 这会导致无效语法错误
	fmt.Println(s, err)

当 JSON 是有效的,但是 Go 的内置函数都无法解析它时,应该如何处理这种情况?除了手动编写在 JSON []byte 中查找和替换模式之外,还有什么正确的方法吗?

编辑:
好吧,我想我应该提出一个更好的问题,或者提供一个具体的示例,而不仅仅是其中的一部分。垃圾进垃圾出,这是我的错。
这是问题。假设我们有一个特定的非标准时间格式,就像上面的 JSON 示例中一样。

{
   "created_at": "6\/30\/2022 21:51:49"
}

我需要将其解析为具有 Go 时间类型的结构体。因为标准的时间解析器无法处理该格式,所以我创建了一个自定义类型。

type Request struct {
    CreatedAt Time `json:"created_at"`
}

type Time time.Time

func (d *Time) UnmarshalJSON(b []byte) error {
    s := strings.Trim(string(b), "\"")
	parse, err := time.Parse("1/2/2006 15:04:05", s) // 这会失败,因为转义的反斜杠
	if err != nil {
		return err
	}
	*d = Time(parse)
	return nil
}

这是 Playground 中的示例:https://go.dev/play/p/bE3AWQeV-ug

panic: parsing time "6\\/30\\/2022 21:51:49" as "1/2/2006 15:04:05": cannot parse "\/30\/2022 21:51:49" as "/"

这就是我试图将 Unquote 放入自定义解组函数中的原因,但这不是正确的方法,正如之前指出的那样。默认的字符串 JSON 解组器是如何去除这些转义斜杠的?哦,我无法控制写入 JSON 的一方,它以单个转义斜杠的形式在 *http.Response[]byte 中传入。

英文:

I am parsing rather simple json where the slash in the date format string is escaped when it arrives in response.

however, when you try to unmarshall the string, it fails on "invalid syntax" error.
So I googled and we should use strconv.Unquote to replace the escaped characters first. Did that, and now the unquote function fails on the "unknown escape" error. However JSON RFC 8259 shows it is a valid case. So why valid JSON is causing failures in Go unmarshaller?

This is the simple JSON

{
   "created_at": "6\/30\/2022 21:51:49"
}
    var dst string
	err := json.Unmarshal([]byte("6\/30\/2022 21:51:49"), dst) // this fails on ivnalid quote
	fmt.Println(dst, err)

	s, err := strconv.Unquote(`"6\/30\/2022 21:51:49"`) // this fails on ivnalid syntax
	fmt.Println(s, err)

How does one proceed with such cases when the JSON is valid, but none of the built-in functions of Go actually parse it? What is the right way, apart from writing simple find and replace pattern in json []byte manually?

EDIT:
ok I think I should have asked a better question or provided a specific example from the scenario rather than just a portion of it. Garbage in garbage out, my fault.
Here is the thing. Let's say we have the specific, non standard time format like in the above json example.

{
   "created_at": "6\/30\/2022 21:51:49"
}

I need to parse it into the struct with go time type. Because the standard parser for time would fail for that format I create a custom type.

type Request struct {
    CreatedAt Time `json:"created_at"`
}

type Time time.Time

unc (d *Time) UnmarshalJSON(b []byte) error {
    s := strings.Trim(string(b), "\"")
	parse, err := time.Parse("1/2/2006 15:04:05", s) // this fails because of the escaped backslashes
	if err != nil {
		return err
	}
	*d = Time(parse)
	return nil
}

Here is an example in Playground: https://go.dev/play/p/bE3AWQeV-ug

panic: parsing time "6\\/30\\/2022 21:51:49" as "1/2/2006 15:04:05": cannot parse "\\/30\\/2022 21:51:49" as "/"

This is the reason I am trying to put the Unquote into the custom unmarshal function, but that is not the right approach as it was already pointed out. How does the default string json unmarshaller for example removes those escaped slashes? Oh and I have no control about the side that writes the json, it comes in with single escaped slashes like that in []byte from *http.Response

答案1

得分: 2

Go语言有一个很好的内置函数可以将JSON字符串转换为Go字符串:json.Unmarshal。以下是如何将其与自定义的UnmarshalJSON方法集成的示例代码:

func (d *Time) UnmarshalJSON(b []byte) error {
    var s string
    if err := json.Unmarshal(b, &s); err != nil {
        return err
    }
    // 移除这一行:s := strings.Trim(string(b), "\"")
    // 其余代码保持不变
}

请注意,我已经将代码中的HTML实体字符进行了转义,以确保显示正确。

英文:

Go has a great built-in function to convert a JSON string to a Go string: json.Unmarshal. Here is how you can integrate it with a custom UnmarshalJSON method:

func (d *Time) UnmarshalJSON(b []byte) error {
    var s string
    if err := json.Unmarshal(b, &s); err != nil {
        return err
    }
    // Remove this line: s := strings.Trim(string(b), "\"")
    // The rest of the code is unchanged

答案2

得分: 1

Unquote是为了处理双引号字符串的情况,例如"\"My name\"",在这种情况下你实际上不需要它。

该代码的问题在于你试图解组一个不是JSON的字节切片。你还试图解组成一个字符串,这是行不通的。你需要解组成具有JSON标签的结构体或解组成一个映射。

var dest map[string]string

someJsonString := `{
"created_at": "6\/30\/2022 21:51:49"
}`

err := json.Unmarshal([]byte(someJsonString), &dest)

if err != nil {		
    panic(err)
}

fmt.Println(dest)

如果你想要解组成一个结构体,可以这样做

type SomeStruct struct {
	CreatedAt string `json:"created_at"`
}

someJsonString := `{
   "created_at": "6\/30\/2022 21:51:49"
}`

var data SomeStruct

err := json.Unmarshal([]byte(someJsonString), &data)

if err != nil {
	panic(err)
}

fmt.Println(data)
英文:

Unquote is designed for cases when you have a double quoted string like "\"My name\"", you don't really need that in this case.

The problem with that code is that you're trying to unmarshal a byte slice that isn't JSON. You're also unmarshaling into an string which isn't going to work. You need to unmarshal into a struct that has json tags or unmarshal into a map.

var dest map[string]string

someJsonString := `{
"created_at": "6\/30\/2022 21:51:49"
}`


err := json.Unmarshal([]byte(someJsonString), &dest)

if err != nil {		
    panic(err)
}

fmt.Println(dest)

if you wanted to do Unmarshal into a struct, you can do it like so

type SomeStruct struct {
	CreatedAt string `json:"created_at"`
}

someJsonString := `{
   "created_at": "6\/30\/2022 21:51:49"
}`

var data SomeStruct

err := json.Unmarshal([]byte(someJsonString), &data)

if err != nil {
	panic(err)
}

fmt.Println(data)

huangapple
  • 本文由 发表于 2022年7月1日 07:36:52
  • 转载请务必保留本文链接:https://go.coder-hub.com/72823045.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定