英文:
Golang slow when looping over Collection
问题
我正在从Redis中提取一个JSON,并对其进行解析和循环遍历。
这是我从Redis中提取的JSON示例:
[
{
"titel": "test 1",
"event": "some value",
"pair": "some value",
"condition": [
"or",
[
"contains",
"url",
"/"
],
[
"notcontains",
"url",
"hello"
]
],
"actions": [
[
"option1",
"12",
"1"
],
[
"option2",
"3",
"1"
]
]
},
{
"titel": "test 2",
"event": "some value",
"pair": "some value",
"condition": [
"or",
[
"contains",
"url",
"/"
]
],
"actions": [
[
"option1",
"12",
"1"
],
[
"option2",
"3",
"1"
]
]
}
]
我用以下结构体来存储这个JSON:
type Trigger struct {
Event string json:"event"
Pair string json:"pair"
Actions [][]string json:"actions"
Condition Condition json:"condition"
}
type Condition []interface{}
func (c *Condition) Typ() string {
return (*c)[0].(string)
}
func (c *Condition) Val() []string {
xs := (*c)[1].([]interface{})
ys := make([]string, len(xs))
for i, x := range xs {
ys[i] = x.(string)
}
return ys
}
func (c *Condition) String() (string, string, string) {
return c.Val()[0], c.Val()[1], c.Val()[2]
}
type Triggers struct {
Collection []Trigger
}
然后我将其解析为结构体:
var triggers Triggers
err := json.Unmarshal([]byte(triggersJson), &triggers.Collection)
if err != nil {
fmt.Println("解析triggers json时出错:", err)
}
在使用循环遍历结构体时,性能变得很差,响应时间变长并出现超时。
这是我的循环遍历方式:
for _, element := range triggers.Collection {
if element.Event == "some value" {
fmt.Println("我们得到了:some value")
}
}
循环之外的性能是500个并发连接下的2毫秒。但是加上循环后,500个并发连接下的响应时间变为600毫秒,并出现了一些超时。
理想情况下,我希望更改JSON的结构,不包括"or",但是我无法控制JSON的结构,所以这是不可能的。
有什么想法是什么原因导致这种情况,并且如何解决?
编辑
我发现是什么导致整个过程变慢。
将结构体修改为:
type Trigger struct {
Event string json:"event"
Pair string json:"pair"
Actions [][]string json:"actions"
//Condition Condition json:"condition"
}
将响应时间从600毫秒降低到100毫秒。但是现在我无法解析Condition
。
不确定如何以与当前方式不同的方式解析Condition
,因为它有两种不同的格式。
英文:
Im feeding my app a json from Redis which I then unmarshal and loop through.
Here is what the json Im feeding from Redis looks like:
[
{
"titel": "test 1",
"event": "some value",
"pair": "some value",
"condition": [
"or",
[
"contains",
"url",
"/"
],[
"notcontains",
"url",
"hello"
]
],
"actions": [
[
"option1",
"12",
"1"
],
[
"option2",
"3",
"1"
]
]
}, {
"titel": "test 2",
"event": "some value",
"pair": "some value",
"condition": [
"or",
[
"contains",
"url",
"/"
]
],
"actions": [
[
"option1",
"12",
"1"
],
[
"option2",
"3",
"1"
]
]
}
]
My struct to store the json looks like this:
type Trigger struct {
Event string `json:"event"`
Pair string `json:"pair"`
Actions [][]string `json:"actions"`
Condition Condition `json:"condition"`
}
type Condition []interface{}
func (c *Condition) Typ() string {
return (*c)[0].(string)
}
func (c *Condition) Val() []string {
xs := (*c)[1].([]interface{})
ys := make([]string, len(xs))
for i, x := range xs {
ys[i] = x.(string)
}
return ys
}
func (c *Condition) String() (string, string, string) {
return c.Val()[0], c.Val()[1], c.Val()[2]
}
type Triggers struct {
Collection []Trigger
}
And Im unmarshaling it into the struct like so:
var triggers Triggers
err := json.Unmarshal([]byte(triggersJson), &triggers.Collection)
if err != nil {
fmt.Println("Error while unmarshaling triggers json: ", err)
}
This works and performs perfectly fine while testing it with siege.
However, as soon as I want to start loop over the struct Im starting to see long response times and timeouts.
This is how I loop over it:
for _,element := range triggers.Collection {
if element.Event == "some value" {
fmt.Println("We got: some value")
}
}
Performance without the loop is 2ms on 500 concurrent connections.
With the loop its 600ihs ms on 500 concurrent with a bunch of timeouts
Ideally Id like to change structure of the json to not include:
"or",
But Im not in control over the json so this is unfortunately impossible.
Any ideas whats causing this and how it could be resolved?
Edit
I found whats slowing the whole thing down.
Editing my struct like:
type Trigger struct {
Event string `json:"event"`
Pair string `json:"pair"`
Actions [][]string `json:"actions"`
//Condition Condition `json:"condition"`
}
Takes it from 600ms to 100ms. However now Im not able to parse the Condition
.
Now sure how to parse the Condition in another way than Im currently doing due it having two different formats
答案1
得分: 4
尽量避免复制集合元素,可能听起来有些奇怪。可以尝试像这样写:
for i := range triggers.Collection {
if triggers.Collection[i].Event == "some value" {
fmt.Println("We got: some value")
}
}
英文:
Sounds weird, but try to avoid copying Collection elements. Something like this:
for i := range triggers.Collection {
if triggers.Collection[i].Event == "some value" {
fmt.Println("We got: some value")
}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论