ElasticSearch + Go：索引失败（没有名称的功能）

huangapple
go评论91阅读模式
英文:

ElasticSearch + Go: Index failures (No feature for name)

问题

我正在尝试使用Completion Suggesters在ElasticSearch的v1.4x版本中为我的自动完成服务建立索引。我正在遵循ElasticSearch - You Complete Me中的建议，并使用Go客户端olivere/elastic。

我的索引方法看起来像这样：

func IndexVehicle(client *elastic.Client, vehicle Vehicle) (bool, error) {
    // 检查是否已存在
    fetch, err := client.Get().
        Index(vehicleIndex).
        Type("vehicle").
        Id(vehicle.Id).
        Do()
    if err != nil || fetch.Found {
        return false, err
    }

    vehicleName := fmt.Sprintf("%s %s (%s) %s", vehicle.Make, vehicle.Model, vehicle.Model_year, vehicle.Primary_fuel)

    suggest := elastic.NewSuggestField()
    suggest.Input(vehicle.Make, vehicle.Model, vehicle.Primary_fuel, vehicle.Model_year).
        Output(vehicleName).
        Payload(vehicle)

    // 保存
    put, err := client.Index().
        Index(vehicleIndex).
        Type("vehicle").
        Id(vehicle.Id).
        Debug(true).Pretty(true).
        BodyJson(indexBody{Name: vehicleName, Suggest: suggest}).
        Do()
    if err != nil {
        return false, err
    }
    return put.Created, nil
}

奇怪的是，在一些测试中，这个方法可以正常工作并且项目将被索引。但在其他测试中，在清除和重建索引后，测试将失败。

之前的测试在调试中看起来像这样：

2014/12/15 14:11:37 PUT /vehicle/vehicle/369f96459b340507c4688740da3bfe1a?pretty=true HTTP/1.1
Host: localhost:9200
User-Agent: elastic/1.3.1 (darwin-amd64)
Transfer-Encoding: chunked
Accept: application/json
Content-Type: application/json
Accept-Encoding: gzip

{"name":"American Motors Corporation Eagle 4WD (1986) regular","suggest":{"input":["American Motors Corporation","Eagle 4WD","regular","1986"],"output":"American Motors Corporation Eagle 4WD (1986) regular","payload":{"make":"American Motors Corporation","model_year":"1986","model":"Eagle 4WD","primary_fuel":"regular","vehicle_class":"Special Purpose Vehicle 4WD","transmission":"Automatic 3-spd","displacement":"4.2","drive":"4-Wheel or All-Wheel Drive","city_mpg":"15.0","highway_mpg":"19.0","comb_mpg":"17.0"}}}

2014/12/15 14:11:37 HTTP/1.1 201 Created
Content-Length: 134
Content-Type: application/json; charset=UTF-8

{
  "_index" : "vehicle",
  "_type" : "vehicle",
  "_id" : "369f96459b340507c4688740da3bfe1a",
  "_version" : 1,
  "created" : true
}

但在后续的测试中，执行相同的操作会导致错误。在后续测试中，IndexVehicle()返回的err是：

Errors:

  * /Users/phil/go/src/github.com/ride/autocomplete/vehicle_test.go 
  Line 79: - elastic: Error 400: ElasticsearchIllegalArgumentException[No feature for name [vehicle]] 
  goroutine 245 [running]:
  ...

可能更重要的是这个回溯的一部分：

Error 400: ElasticsearchIllegalArgumentException[No feature for name [vehicle]]

所以，不确定出了什么问题。我在索引上设置了1秒的超时，因为这个客户端目前不支持“等待绿色”逻辑。

func ResetVehicleIndex(client *elastic.Client) (err error) {

    if _, err = client.DeleteIndex(vehicleIndex).Do(); err != nil {
        return
    }
    if _, err = EnsureVehicleIndex(client); err != nil {
        return
    }

    // TODO: 这很糟糕。当elastic客户端支持时，切换到“等待绿色”
    time.Sleep(time.Second * 1)

    return nil
}

这对于大多数其他测试都有效，因为elasticsearch似乎在一秒左右准备好了，但是这些测试无论在代码中等待多长时间都会出错。

有什么想法吗？我可能需要编辑问题以添加更多代码或更好的解释，但如果你在Twitter上@philsturgeon上提问，我会非常快速地回复。我真的被困住了。

英文:

I'm trying to get ElasticSearch to index content for my autocomplete service, using Completion Suggesters in v1.4x. I was following advice from ElasticSearch - You Complete Me and am using the Go Client olivere/elastic.

My index method looks a bit like this:

func IndexVehicle(client *elastic.Client, vehicle Vehicle) (bool, error) {
	// See if it exists already
	fetch, err := client.Get().
		Index(vehicleIndex).
		Type("vehicle").
		Id(vehicle.Id).
		Do()
	if err != nil || fetch.Found {
		return false, err
	}

	vehicleName := fmt.Sprintf("%s %s (%s) %s", vehicle.Make, vehicle.Model, vehicle.Model_year, vehicle.Primary_fuel)

	suggest := elastic.NewSuggestField()
	suggest.Input(vehicle.Make, vehicle.Model, vehicle.Primary_fuel, vehicle.Model_year).
		Output(vehicleName).
		Payload(vehicle)

	// Go forth and save
	put, err := client.Index().
		Index(vehicleIndex).
		Type("vehicle").
		Id(vehicle.Id).
		Debug(true).Pretty(true).
		BodyJson(indexBody{Name: vehicleName, Suggest: suggest}).
		Do()
	if err != nil {
		return false, err
	}
	return put.Created, nil
}

Now what is weird, is that in some of my tests this method works fine and items will be indexed. In other tests after wiping and rebuilding the index the tests will fail:

Earlier tests look like this in the debug:

2014/12/15 14:11:37 PUT /vehicle/vehicle/369f96459b340507c4688740da3bfe1a?pretty=true HTTP/1.1
Host: localhost:9200
User-Agent: elastic/1.3.1 (darwin-amd64)
Transfer-Encoding: chunked
Accept: application/json
Content-Type: application/json
Accept-Encoding: gzip

{"name":"American Motors Corporation Eagle 4WD (1986) regular","suggest":{"input":["American Motors Corporation","Eagle 4WD","regular","1986"],"output":"American Motors Corporation Eagle 4WD (1986) regular","payload":{"make":"American Motors Corporation","model_year":"1986","model":"Eagle 4WD","primary_fuel":"regular","vehicle_class":"Special Purpose Vehicle 4WD","transmission":"Automatic 3-spd","displacement":"4.2","drive":"4-Wheel or All-Wheel Drive","city_mpg":"15.0","highway_mpg":"19.0","comb_mpg":"17.0"}}}

2014/12/15 14:11:37 HTTP/1.1 201 Created
Content-Length: 134
Content-Type: application/json; charset=UTF-8

{
  "_index" : "vehicle",
  "_type" : "vehicle",
  "_id" : "369f96459b340507c4688740da3bfe1a",
  "_version" : 1,
  "created" : true
}

But in later tests, doing this same thing leads to errors. The err returned from IndexVehicle() in later tests is:

E
Errors:

  * /Users/phil/go/src/github.com/ride/autocomplete/vehicle_test.go 
  Line 79: - elastic: Error 400: ElasticsearchIllegalArgumentException[No feature for name [vehicle]] 
  goroutine 245 [running]:
  github.com/ride/autocomplete.func·033()
  	/Users/phil/go/src/github.com/ride/autocomplete/vehicle_test.go:79 +0x249
  github.com/ride/autocomplete.useIndex(0x499e98)
  	/Users/phil/go/src/github.com/ride/autocomplete/test_helper.go:18 +0x55
  github.com/ride/autocomplete.func·034()
  	/Users/phil/go/src/github.com/ride/autocomplete/vehicle_test.go:96 +0x2a
  github.com/jtolds/gls._m(0x0, 0xc2080ae9e0)
  	/Users/phil/go/src/github.com/jtolds/gls/stack_tags.go:70 +0x32
  github.com/jtolds/gls.markS(0x0, 0xc2080ae9e0)
  	/Users/phil/go/src/github.com/jtolds/gls/stack_tags.go:21 +0x32
  github.com/jtolds/gls.addStackTag(0x0, 0xc2080ae9e0)
  	/Users/phil/go/src/github.com/jtolds/gls/stack_tags.go:18 +0x3e
  github.com/jtolds/gls.(*ContextManager).SetValues(0xc20801e080, 0xc2080b31d0, 0xc2080ae9e0)
  	/Users/phil/go/src/github.com/jtolds/gls/context.go:98 +0x503
  github.com/ride/autocomplete.TestSearchForVehicles(0xc20806a480)
  	/Users/phil/go/src/github.com/ride/autocomplete/vehicle_test.go:97 +0x243
  testing.tRunner(0xc20806a480, 0x5be890)
  	/opt/boxen/homebrew/Cellar/go/1.4/libexec/src/testing/testing.go:447 +0xbf
  created by testing.RunTests
  	/opt/boxen/homebrew/Cellar/go/1.4/libexec/src/testing/testing.go:555 +0xa8b
  
  goroutine 1 [chan receive]:
  testing.RunTests(0x49a078, 0x5be800, 0x7, 0x7, 0x67c001)
  	/opt/boxen/homebrew/Cellar/go/1.4/libexec/src/testing/testing.go:556 +0xad6
  testing.(*M).Run(0xc2080463c0, 0x5c9b20)
  	/opt/boxen/homebrew/Cellar/go/1.4/libexec/src/testing/testing.go:485 +0x6c
  main.main()
  	github.com/ride/autocomplete/_test/_testmain.go:64 +0x1d5
  
  goroutine 208 [chan receive]:
  github.com/olivere/elastic.(*Client).pinger(0xc208106d40)
  	/Users/phil/go/src/github.com/olivere/elastic/client.go:79 +0x6b
  created by github.com/olivere/elastic.NewClient
  	/Users/phil/go/src/github.com/olivere/elastic/client.go:60 +0x26e
  
  goroutine 248 [runnable]:
  net/http.(*persistConn).readLoop(0xc20802e4d0)
  	/opt/boxen/homebrew/Cellar/go/1.4/libexec/src/net/http/transport.go:928 +0x9ce
  created by net/http.(*Transport).dialConn
  	/opt/boxen/homebrew/Cellar/go/1.4/libexec/src/net/http/transport.go:660 +0xc9f
  
  goroutine 98 [chan receive]:
  github.com/olivere/elastic.(*Client).pinger(0xc208033e00)
  	/Users/phil/go/src/github.com/olivere/elastic/client.go:79 +0x6b
  created by github.com/olivere/elastic.NewClient
  	/Users/phil/go/src/github.com/olivere/elastic/client.go:60 +0x26e
  
  goroutine 17 [syscall, locked to thread]:
  runtime.goexit()
  	/opt/boxen/homebrew/Cellar/go/1.4/libexec/src/runtime/asm_amd64.s:2232 +0x1
  
  goroutine 44 [chan receive]:
  github.com/olivere/elastic.(*Client).pinger(0xc2080332c0)
  	/Users/phil/go/src/github.com/olivere/elastic/client.go:79 +0x6b
  created by github.com/olivere/elastic.NewClient
  	/Users/phil/go/src/github.com/olivere/elastic/client.go:60 +0x26e
  
  goroutine 249 [select]:
  net/http.(*persistConn).writeLoop(0xc20802e4d0)
  	/opt/boxen/homebrew/Cellar/go/1.4/libexec/src/net/http/transport.go:945 +0x41d
  created by net/http.(*Transport).dialConn
  	/opt/boxen/homebrew/Cellar/go/1.4/libexec/src/net/http/transport.go:661 +0xcbc
  
  goroutine 54 [chan receive]:
  github.com/olivere/elastic.(*Client).pinger(0xc208032f80)
  	/Users/phil/go/src/github.com/olivere/elastic/client.go:79 +0x6b
  created by github.com/olivere/elastic.NewClient
  	/Users/phil/go/src/github.com/olivere/elastic/client.go:60 +0x26e
  
  goroutine 76 [chan receive]:
  github.com/olivere/elastic.(*Client).pinger(0xc208032e00)
  	/Users/phil/go/src/github.com/olivere/elastic/client.go:79 +0x6b
  created by github.com/olivere/elastic.NewClient
  	/Users/phil/go/src/github.com/olivere/elastic/client.go:60 +0x26e
  
  goroutine 250 [chan receive]:
  github.com/olivere/elastic.(*Client).pinger(0xc208106c40)
  	/Users/phil/go/src/github.com/olivere/elastic/client.go:79 +0x6b
  created by github.com/olivere/elastic.NewClient
  	/Users/phil/go/src/github.com/olivere/elastic/client.go:60 +0x26e
  
  goroutine 120 [chan receive]:
  github.com/olivere/elastic.(*Client).pinger(0xc208106b00)
  	/Users/phil/go/src/github.com/olivere/elastic/client.go:79 +0x6b
  created by github.com/olivere/elastic.NewClient
  	/Users/phil/go/src/github.com/olivere/elastic/client.go:60 +0x26e
  
  goroutine 142 [chan receive]:
  github.com/olivere/elastic.(*Client).pinger(0xc208106e00)
  	/Users/phil/go/src/github.com/olivere/elastic/client.go:79 +0x6b
  created by github.com/olivere/elastic.NewClient
  	/Users/phil/go/src/github.com/olivere/elastic/client.go:60 +0x26e
  
  goroutine 164 [chan receive]:
  github.com/olivere/elastic.(*Client).pinger(0xc208106b80)
  	/Users/phil/go/src/github.com/olivere/elastic/client.go:79 +0x6b
  created by github.com/olivere/elastic.NewClient
  	/Users/phil/go/src/github.com/olivere/elastic/client.go:60 +0x26e
  
  goroutine 186 [chan receive]:
  github.com/olivere/elastic.(*Client).pinger(0xc208106d00)
  	/Users/phil/go/src/github.com/olivere/elastic/client.go:79 +0x6b
  created by github.com/olivere/elastic.NewClient
  	/Users/phil/go/src/github.com/olivere/elastic/client.go:60 +0x26e
  
  goroutine 230 [chan receive]:
  github.com/olivere/elastic.(*Client).pinger(0xc208106dc0)
  	/Users/phil/go/src/github.com/olivere/elastic/client.go:79 +0x6b
  created by github.com/olivere/elastic.NewClient
  	/Users/phil/go/src/github.com/olivere/elastic/client.go:60 +0x26e

Possibly the more important part of that backtrace is this:

> Error 400: ElasticsearchIllegalArgumentException[No feature for name [vehicle]]

So, not sure what the heck is going wrong here. I have a 1 second timeout on the index, because this client doesn't support "wait for green" logic [yet].

func ResetVehicleIndex(client *elastic.Client) (err error) {

	if _, err = client.DeleteIndex(vehicleIndex).Do(); err != nil {
		return
	}
	if _, err = EnsureVehicleIndex(client); err != nil {
		return
	}

	// TODO: This is awful. Switch to "wait for green" when elastic client supports it
	time.Sleep(time.Second * 1)

	return nil
}

This has worked for most other tests, as elasticsearch seems to get ready in a second or so, but these tests are consistently erroring regardless of the wait time in that bit of code.

Any ideas? I might need to edit the question to add more code or better explanations, but I'll reply super quickly to any questions especially if you ping me on Twitter @philsturgeon. I'm really stuck on this.

答案1

得分: 10

这个异常只会在一个地方触发，即在进行 Get Index API 调用期间。这意味着你的车辆 ID 在这里必须为 null：

fetch, err := client.Get().
        Index(vehicleIndex).
        Type("vehicle").
        Id(vehicle.Id).       //<-- 这里
        Do()

你正在尝试进行 Get Document API，其格式为 GET /{index}/{type}/{id}。然而，你的客户端没有区分 Get Document 和 Get Index API 调用...并且它不验证你的参数是否为非空。

因此，如果将 null 的 vehicle.Id 传递给 Get 方法，你最终的 URL 实际上将变为 GET /{index}/{type}/

从 Elasticsearch 的角度来看，这不再是一个 Get Document API 调用...实际上是一个 Get Index 调用，其格式如下：GET /{index}/{feature}。Feature 可以是以下之一：_settings、_mappings、_aliases 或 _warmers。

因为 vehicle 不是这些 feature 之一，ES 抛出了一个异常并输出错误信息。你可以从控制台验证这一点：

curl -XGET localhost:9200/my_index/vehicle/
英文:

That exception is only triggered in one place, which is during a Get Index API call. Which means your vehicle ID must be null here:

fetch, err := client.Get().
        Index(vehicleIndex).
        Type(&quot;vehicle&quot;).
        Id(vehicle.Id).       //&lt;-- this
        Do()

You are trying to do a Get Document API, which follows the format of GET /{index}/{type}/{id}. However, your client doesn't make a distinction between Get Document and Get Index API calls...and it doesn't validate that your parameters are non-null.

So if a null vehicle.Id is passed to the Get method, your final URL will actually be GET /{index}/{type}/

From Elasticsearch's point of view, this is no longer a Get Document API call...it's actually a Get Index call, which has the following format: GET /{index}/{feature}. Feature can be one of: _settings, _mappings, _aliases or _warmers.

Because vehicle is not one of those features, ES is throwing an exception and spewing. You can verify this from the console:

curl -XGET localhost:9200/my_index/vehicle/

答案2

得分: 0

只是为了完整性：当我使用"post"而不是"POST"来初始化我的请求方法时，我遇到了相同的错误。

像这样：

req, _ := http.NewRequest("post", url, query)
// 返回：错误400：ElasticsearchIllegalArgumentException[没有名为[_bulk]的功能]
req, _ := http.NewRequest("POST", url, query)
// 运行得很好
英文:

Just for completeness: I had the same error when I initialized my request method with "post" instead of "POST"

like so:

req, _ := http.NewRequest(&quot;post&quot;, url, query)
// returns: Error 400: ElasticsearchIllegalArgumentException[No feature for name [_bulk]]
req, _ := http.NewRequest(&quot;POST&quot;, url, query)
// works just fine

huangapple
  • 本文由 发表于 2014年12月16日 03:36:05
  • 转载请务必保留本文链接：https://go.coder-hub.com/27491738.html
  • elasticsearch
  • go
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定