如何使用Go在Elasticsearch中进行搜索,并通过时间范围对结果进行过滤。

huangapple go评论98阅读模式
英文:

How to search in elasticsearch with Go filtering results by time frames

问题

我对Go编程还比较新手。

我正在尝试创建一个简单的程序,只做一件事,通过elasticsearch API搜索字符串。
我的问题与我正在使用的"gopkg.in/olivere/elastic.v2"包有关。

这是一个代码示例:

package main

import (
	"fmt"
	"gopkg.in/olivere/elastic.v2"
	"log"
	"reflect"
)

type Syslog struct {
	Program   string
	Message   string
	Timestamp string
}


func main() {

	client, err := elastic.NewClient(elastic.SetURL("http://localhost:9200"))
	if err != nil {
		log.Fatal(err)
	}
	info, code, err := client.Ping().Do()
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("Elasticsearch returned with code %d and version  %s", code, info.Version.Number)

	termQuery := elastic.NewTermQuery("message", "configuration")

	searchResult, err := client.Search().
		//PostFilter(postFilter).
		Index("logstash-*").     // search in index "logstash-*"
		Query(&termQuery).       // specify the query
		Sort("timestamp", true). // sort by "message" field, ascending
		From(0).Size(10).        // take documents 0-9
		Pretty(true).            // pretty print request and response JSON
		Do()                     // execute
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf(" Query took %d milliseconds\n", searchResult.TookInMillis)

	var ttyp Syslog

	for _, item := range searchResult.Each(reflect.TypeOf(ttyp)) {
		t := item.(Syslog)
		fmt.Printf("Found %s %s %s\n", t.Timestamp, t.Program, t.Message)
	}

}

这个代码是可以工作的,但是我真正想要的是通过时间范围来限制搜索。

据我了解,我需要使用FilteredQuery函数。但是我在尝试弄清楚如何创建一个过滤查询时感到困惑。

有一个函数叫做"FilteredQuery",它需要一个查询类型作为输入参数,但是没有函数可以创建查询本身。我只能创建一个所需类型的变量,但它仍然需要像在elasticsearch中要查找的字段和要查找的字符串这样的字段。

对于func (FilteredQuery) Filter中的Filter变量也是同样的情况。

也许有人已经有一个示例,或者可以指导我正确的方向。

我有一个JSON的示例,如果传递给elastcisearch,可以得到我想要的结果:

"query": {
    "filtered": {
      "query": {
        "query_string": {
          "query": "message:\"configuration\"",
          "analyze_wildcard": true
        }
      },
      "filter": {
        "bool": {
          "must": [
            {
              "range": {
                "@timestamp": {
                  "gte": 1442100154219,
                  "lte": 1442704954219
                }
              }
            }
          ],
          "must_not": []
        }
      }
    }
  }

但是我不知道如何在Go中实现相同的功能。谢谢。

英文:

I am retentively new to programming in Go.

I am trying create a simple program which does only one thing searches string via elasticsearch API.
My question is specific to the "gopkg.in/olivere/elastic.v2" package I am using.

Here is a code sample:

package main
import (
"fmt"
"gopkg.in/olivere/elastic.v2"
"log"
"reflect"
)
type Syslog struct {
Program   string
Message   string
Timestamp string
}
func main() {
client, err := elastic.NewClient(elastic.SetURL("http://localhost:9200"))
if err != nil {
log.Fatal(err)
}
info, code, err := client.Ping().Do()
if err != nil {
log.Fatal(err)
}
fmt.Printf("Elasticsearch returned with code %d and version  %s", code, info.Version.Number)
termQuery := elastic.NewTermQuery("message", "configuration")
searchResult, err := client.Search().
//PostFilter(postFilter).
Index("logstash-*").     // search in index "logstash-*"
Query(&termQuery).       // specify the query
Sort("timestamp", true). // sort by "message" field, ascending
From(0).Size(10).        // take documents 0-9
Pretty(true).            // pretty print request and response JSON
Do()                     // execute
if err != nil {
log.Fatal(err)
}
fmt.Printf(" Query took %d milliseconds\n", searchResult.TookInMillis)
var ttyp Syslog
for _, item := range searchResult.Each(reflect.TypeOf(ttyp)) {
t := item.(Syslog)
fmt.Printf("Found %s %s %s\n", t.Timestamp, t.Program, t.Message)
}
}

This works, however what I really want is to limit search by time range.

As I understand I need to use FilteredQuery function. But I am lost in attempts to figure out how to create a filtered query.

There is function which created it "FilteredQuery" it requires Query Type as an input parameter but there is no function to create Query itself. I can just create a variable of a required type, still it need to have fields like which filed in elasticsearch to look for and a string to look for.

Same thing goes for the Filter variable in call for func (FilteredQuery) Filter.

May be someone has already got an example or can point me into the right direction.

I have an example of a JSON which if passed to elastcisearch gets me what I want:

"query": {
"filtered": {
"query": {
"query_string": {
"query": "message:\"configuration\"",
"analyze_wildcard": true
}
},
"filter": {
"bool": {
"must": [
{
"range": {
"@timestamp": {
"gte": 1442100154219,
"lte": 1442704954219
}
}
}
],
"must_not": []
}
}
}
},

But I do not know how to implement the same in Go. Thanks.

答案1

得分: 2

请看RangeFilterFilteredQuery

以下是一个产生你想要的结果的示例代码:

package main

import (
	"encoding/json"
	"fmt"

	elastic "gopkg.in/olivere/elastic.v2"
)

func main() {

	termQuery := elastic.NewTermQuery("message", "configuration")

	timeRangeFilter := elastic.NewRangeFilter("@timestamp").Gte(1442100154219).Lte(1442704954219)

	// 不确定这个是否真的有用,我只是把它放在这里以模仿你所期望的结果
	boolFilter := elastic.NewBoolFilter().Must(timeRangeFilter)

	query := elastic.NewFilteredQuery(termQuery).Filter(boolFilter)

	queryBytes, err := json.MarshalIndent(query.Source(), "", "\t")
	if err != nil {
		panic(err)
	}

	fmt.Println(string(queryBytes))
}

它产生的输出结果是:

{
	"filtered": {
		"filter": {
			"bool": {
				"must": {
					"range": {
						"@timestamp": {
							"from": 1442100154219,
							"include_lower": true,
							"include_upper": true,
							"to": 1442704954219
						}
					}
				}
			}
		},
		"query": {
			"term": {
				"message": "configuration"
			}
		}
	}
}
英文:

See RangeFilter and FilteredQuery

Here is an example producing the result you want :

package main
import (
"encoding/json"
"fmt"
elastic "gopkg.in/olivere/elastic.v2"
)
func main() {
termQuery := elastic.NewTermQuery("message", "configuration")
timeRangeFilter := elastic.NewRangeFilter("@timestamp").Gte(1442100154219).Lte(1442704954219)
// Not sure this one is really useful, I just put it here to mimic what you said is the expected result
boolFilter := elastic.NewBoolFilter().Must(timeRangeFilter)
query := elastic.NewFilteredQuery(termQuery).Filter(boolFilter)
queryBytes, err := json.MarshalIndent(query.Source(), "", "\t")
if err != nil {
panic(err)
}
fmt.Println(string(queryBytes))
}

The output it produces is:

{
"filtered": {
"filter": {
"bool": {
"must": {
"range": {
"@timestamp": {
"from": 1442100154219,
"include_lower": true,
"include_upper": true,
"to": 1442704954219
}
}
}
}
},
"query": {
"term": {
"message": "configuration"
}
}
}
}

huangapple
  • 本文由 发表于 2015年9月20日 08:48:45
  • 转载请务必保留本文链接:https://go.coder-hub.com/32674666.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定