英文:
Golang slow when looping over Collection
问题
我正在从Redis中提取一个JSON,并对其进行解析和循环遍历。
这是我从Redis中提取的JSON示例:
[
{
"titel": "test 1",
"event": "some value",
"pair": "some value",
"condition": [
"or",
[
"contains",
"url",
"/"
],
[
"notcontains",
"url",
"hello"
]
],
"actions": [
[
"option1",
"12",
"1"
],
[
"option2",
"3",
"1"
]
]
},
{
"titel": "test 2",
"event": "some value",
"pair": "some value",
"condition": [
"or",
[
"contains",
"url",
"/"
]
],
"actions": [
[
"option1",
"12",
"1"
],
[
"option2",
"3",
"1"
]
]
}
]
我用以下结构体来存储这个JSON:
type Trigger struct {
Event string json:"event"
Pair string json:"pair"
Actions [][]string json:"actions"
Condition Condition json:"condition"
}
type Condition []interface{}
func (c *Condition) Typ() string {
return (*c)[0].(string)
}
func (c *Condition) Val() []string {
xs := (*c)[1].([]interface{})
ys := make([]string, len(xs))
for i, x := range xs {
ys[i] = x.(string)
}
return ys
}
func (c *Condition) String() (string, string, string) {
return c.Val()[0], c.Val()[1], c.Val()[2]
}
type Triggers struct {
Collection []Trigger
}
然后我将其解析为结构体:
var triggers Triggers
err := json.Unmarshal([]byte(triggersJson), &triggers.Collection)
if err != nil {
fmt.Println("解析triggers json时出错:", err)
}
在使用循环遍历结构体时,性能变得很差,响应时间变长并出现超时。
这是我的循环遍历方式:
for _, element := range triggers.Collection {
if element.Event == "some value" {
fmt.Println("我们得到了:some value")
}
}
循环之外的性能是500个并发连接下的2毫秒。但是加上循环后,500个并发连接下的响应时间变为600毫秒,并出现了一些超时。
理想情况下,我希望更改JSON的结构,不包括"or",但是我无法控制JSON的结构,所以这是不可能的。
有什么想法是什么原因导致这种情况,并且如何解决?
编辑
我发现是什么导致整个过程变慢。
将结构体修改为:
type Trigger struct {
Event string json:"event"
Pair string json:"pair"
Actions [][]string json:"actions"
//Condition Condition json:"condition"
}
将响应时间从600毫秒降低到100毫秒。但是现在我无法解析Condition。
不确定如何以与当前方式不同的方式解析Condition,因为它有两种不同的格式。
英文:
Im feeding my app a json from Redis which I then unmarshal and loop through.
Here is what the json Im feeding from Redis looks like:
[
{
"titel": "test 1",
"event": "some value",
"pair": "some value",
"condition": [
"or",
[
"contains",
"url",
"/"
],[
"notcontains",
"url",
"hello"
]
],
"actions": [
[
"option1",
"12",
"1"
],
[
"option2",
"3",
"1"
]
]
}, {
"titel": "test 2",
"event": "some value",
"pair": "some value",
"condition": [
"or",
[
"contains",
"url",
"/"
]
],
"actions": [
[
"option1",
"12",
"1"
],
[
"option2",
"3",
"1"
]
]
}
]
My struct to store the json looks like this:
type Trigger struct {
Event string `json:"event"`
Pair string `json:"pair"`
Actions [][]string `json:"actions"`
Condition Condition `json:"condition"`
}
type Condition []interface{}
func (c *Condition) Typ() string {
return (*c)[0].(string)
}
func (c *Condition) Val() []string {
xs := (*c)[1].([]interface{})
ys := make([]string, len(xs))
for i, x := range xs {
ys[i] = x.(string)
}
return ys
}
func (c *Condition) String() (string, string, string) {
return c.Val()[0], c.Val()[1], c.Val()[2]
}
type Triggers struct {
Collection []Trigger
}
And Im unmarshaling it into the struct like so:
var triggers Triggers
err := json.Unmarshal([]byte(triggersJson), &triggers.Collection)
if err != nil {
fmt.Println("Error while unmarshaling triggers json: ", err)
}
This works and performs perfectly fine while testing it with siege.
However, as soon as I want to start loop over the struct Im starting to see long response times and timeouts.
This is how I loop over it:
for _,element := range triggers.Collection {
if element.Event == "some value" {
fmt.Println("We got: some value")
}
}
Performance without the loop is 2ms on 500 concurrent connections.
With the loop its 600ihs ms on 500 concurrent with a bunch of timeouts
Ideally Id like to change structure of the json to not include:
"or",
But Im not in control over the json so this is unfortunately impossible.
Any ideas whats causing this and how it could be resolved?
Edit
I found whats slowing the whole thing down.
Editing my struct like:
type Trigger struct {
Event string `json:"event"`
Pair string `json:"pair"`
Actions [][]string `json:"actions"`
//Condition Condition `json:"condition"`
}
Takes it from 600ms to 100ms. However now Im not able to parse the Condition.
Now sure how to parse the Condition in another way than Im currently doing due it having two different formats
答案1
得分: 4
尽量避免复制集合元素,可能听起来有些奇怪。可以尝试像这样写:
for i := range triggers.Collection {
if triggers.Collection[i].Event == "some value" {
fmt.Println("We got: some value")
}
}
英文:
Sounds weird, but try to avoid copying Collection elements. Something like this:
for i := range triggers.Collection {
if triggers.Collection[i].Event == "some value" {
fmt.Println("We got: some value")
}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。


评论