Golang在循环遍历集合时速度较慢。

huangapple go评论71阅读模式
英文:

Golang slow when looping over Collection

问题

我正在从Redis中提取一个JSON,并对其进行解析和循环遍历。

这是我从Redis中提取的JSON示例:

[
{
"titel": "test 1",
"event": "some value",
"pair": "some value",
"condition": [
"or",
[
"contains",
"url",
"/"
],
[
"notcontains",
"url",
"hello"
]
],
"actions": [
[
"option1",
"12",
"1"
],
[
"option2",
"3",
"1"
]
]
},
{
"titel": "test 2",
"event": "some value",
"pair": "some value",
"condition": [
"or",
[
"contains",
"url",
"/"
]
],
"actions": [
[
"option1",
"12",
"1"
],
[
"option2",
"3",
"1"
]
]
}
]

我用以下结构体来存储这个JSON:

type Trigger struct {
Event string json:"event"
Pair string json:"pair"
Actions [][]string json:"actions"
Condition Condition json:"condition"
}

type Condition []interface{}

func (c *Condition) Typ() string {
return (*c)[0].(string)
}

func (c *Condition) Val() []string {
xs := (*c)[1].([]interface{})
ys := make([]string, len(xs))
for i, x := range xs {
ys[i] = x.(string)
}
return ys
}

func (c *Condition) String() (string, string, string) {
return c.Val()[0], c.Val()[1], c.Val()[2]
}

type Triggers struct {
Collection []Trigger
}

然后我将其解析为结构体:

var triggers Triggers
err := json.Unmarshal([]byte(triggersJson), &triggers.Collection)
if err != nil {
fmt.Println("解析triggers json时出错:", err)
}

在使用循环遍历结构体时,性能变得很差,响应时间变长并出现超时。

这是我的循环遍历方式:

for _, element := range triggers.Collection {
if element.Event == "some value" {
fmt.Println("我们得到了:some value")
}
}

循环之外的性能是500个并发连接下的2毫秒。但是加上循环后,500个并发连接下的响应时间变为600毫秒,并出现了一些超时。

理想情况下,我希望更改JSON的结构,不包括"or",但是我无法控制JSON的结构,所以这是不可能的。

有什么想法是什么原因导致这种情况,并且如何解决?

编辑

我发现是什么导致整个过程变慢。

将结构体修改为:

type Trigger struct {
Event string json:"event"
Pair string json:"pair"
Actions [][]string json:"actions"
//Condition Condition json:"condition"
}

将响应时间从600毫秒降低到100毫秒。但是现在我无法解析Condition

不确定如何以与当前方式不同的方式解析Condition,因为它有两种不同的格式。

英文:

Im feeding my app a json from Redis which I then unmarshal and loop through.

Here is what the json Im feeding from Redis looks like:

[
    {
        "titel": "test 1",
        "event": "some value",
        "pair": "some value",
        "condition": [
            "or",
            [
                "contains",
                "url",
                "/"
            ],[
                "notcontains",
                "url",
                "hello"
            ]
        ],
        "actions": [
            [
                "option1",
                "12",
                "1"
            ],
            [
                "option2",
                "3",
                "1"
            ]
        ]
    }, {
        "titel": "test 2",
        "event": "some value",
        "pair": "some value",
        "condition": [
            "or",
            [
                "contains",
                "url",
                "/"
            ]
        ],
        "actions": [
            [
                "option1",
                "12",
                "1"
            ],
            [
                "option2",
                "3",
                "1"
            ]
        ]
    }
]

My struct to store the json looks like this:

type Trigger struct {
	Event     string        `json:"event"`	
	Pair      string        `json:"pair"`	
	Actions   [][]string    `json:"actions"`
	Condition Condition     `json:"condition"`
}

type Condition []interface{}

func (c *Condition) Typ() string {
	return (*c)[0].(string)
}

func (c *Condition) Val() []string {
	xs := (*c)[1].([]interface{})
	ys := make([]string, len(xs))
	for i, x := range xs {
		ys[i] = x.(string)
	}
	return ys
}

func (c *Condition) String() (string, string, string) {
	return c.Val()[0], c.Val()[1], c.Val()[2]
}

type Triggers struct {
	Collection []Trigger
}

And Im unmarshaling it into the struct like so:

var triggers Triggers
err := json.Unmarshal([]byte(triggersJson), &triggers.Collection)
if err != nil {
	fmt.Println("Error while unmarshaling triggers json: ", err)
}

This works and performs perfectly fine while testing it with siege.
However, as soon as I want to start loop over the struct Im starting to see long response times and timeouts.

This is how I loop over it:

for _,element := range triggers.Collection {
	if element.Event == "some value" {
		fmt.Println("We got: some value")
    }
 }

Performance without the loop is 2ms on 500 concurrent connections.
With the loop its 600ihs ms on 500 concurrent with a bunch of timeouts

Ideally Id like to change structure of the json to not include:

"or",

But Im not in control over the json so this is unfortunately impossible.

Any ideas whats causing this and how it could be resolved?

Edit

I found whats slowing the whole thing down.

Editing my struct like:

type Trigger struct {
    Event     string        `json:"event"`  
    Pair      string        `json:"pair"`   
    Actions   [][]string    `json:"actions"`
    //Condition Condition     `json:"condition"`
}

Takes it from 600ms to 100ms. However now Im not able to parse the Condition.

Now sure how to parse the Condition in another way than Im currently doing due it having two different formats

答案1

得分: 4

尽量避免复制集合元素,可能听起来有些奇怪。可以尝试像这样写:

for i := range triggers.Collection {
    if triggers.Collection[i].Event == "some value" {
        fmt.Println("We got: some value")
    }
}
英文:

Sounds weird, but try to avoid copying Collection elements. Something like this:

for i := range triggers.Collection {
    if triggers.Collection[i].Event == "some value" {
        fmt.Println("We got: some value")
    }

huangapple
  • 本文由 发表于 2016年1月5日 00:00:25
  • 转载请务必保留本文链接:https://go.coder-hub.com/34595024.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定