英文:
Can several esutil.BulkIndexer share an elasticsearch.Client?
问题
我有一个文档数组需要使用github.com/elastic/go-elasticsearch/v8库进行批量索引。
每个文档可能被添加到少数几个不同的索引中，所以我的代码如下：
var indexers map[string]esutil.BulkIndexer = make(map[string]esutil.BulkIndexer)
for i := range documents {
var bi esutil.BulkIndexer
var ok bool
var err error
if bi, ok = indexers[documents[i].index]; !ok {
bi, err = esutil.NewBulkIndexer(esutil.BulkIndexerConfig{
Index: documents[i].index, // 索引名称
Client: client, // Elasticsearch客户端*所有索引器共享同一个实例*
NumWorkers: BULK_NUM_WORKERS, // 工作goroutine的数量
FlushBytes: int(BULK_FLUSHBYTES), // 刷新阈值（以字节为单位）
FlushInterval: 5 * time.Second, // 定期刷新间隔
})
if err != nil {
fmt.Printf("创建索引器时出错：%s", err)
continue
}
indexers[documents[i].indexName] = bi
}
我的想法是我有一个esutil.BulkIndexer的映射，它们都共享同一个elasticsearch.Client实例。所以我的问题是，多个esutil.BulkIndexer共享同一个elasticsearch.Client实例是否安全和可行，还是它们每个都需要一个单独的elasticsearch.Client实例？
英文:
I have an array of documents to bulk indexusing the github.com/elastic/go-elasticsearch/v8 library.
Each document may be added to one of a handful of different indexes, so my code is something like:
var indexers map[string]esutil.BulkIndexer = make(map[string]esutil.BulkIndexer)
for i := range documents {
var bi esutil.BulkIndexer
var ok bool
var err error
if bi, ok = indexers[documents[i].index]; !ok {
bi, err = esutil.NewBulkIndexer(esutil.BulkIndexerConfig{
Index: documents[i].index, // The index name
Client: client, // The Elasticsearch client* Same isntance for all indexers*
NumWorkers: BULK_NUM_WORKERS, // The number of worker goroutines
FlushBytes: int(BULK_FLUSHBYTES), // The flush threshold in bytes
FlushInterval: 5 * time.Second, // The periodic flush interval
})
if err != nil {
fmt.Printf("Error creating the indexer: %s", err)
continue
}
indexers[documents[i].indexName] = bi
}
The idea is I have a map of esutil.BulkIndexer , an they all share the same elasticsearch.Client instance. So my question is if it's ok and safe for several
esutil.BulkIndexer to share the same elasticsearch.Client, or do they each need a separate instance of a elasticsearch.Client ?
答案1
得分: 0
无法在官方文档中找到详细信息。但是根据此链接，建议在并行请求中共享客户端连接。
客户端连接是在包初始化时创建的，后续以并行方式进行的请求共享同一个客户端。
来自上述链接的示例代码如下：
package httpexample
import (
"github.com/elastic/go-elasticsearch/v7"
)
var client *elasticsearch.Client
func init() {
var err error
... // 客户端配置
client, err = elasticsearch.NewClient(cfg)
if err != nil {
log.Fatalf("elasticsearch.NewClient: %v", err)
}
}
// 此方法将并行调用
func HttpExample(w http.ResponseWriter, r *http.Request) {
... // 使用客户端
}
注意：我建议使用多个bulkIndexer。
英文:
Could not find the details on official documentation. But Looking at this link it is recommended to share the client across the workers(go routines) for parallel requests
The client connection is created on package initialization, and subsequent requests happening in parallel fashion shares the same client.
from the above link
package httpexample
import (
"github.com/elastic/go-elasticsearch/v7"
)
var client *elasticsearch.Client
func init() {
var err error
... # Client configuration
client, err = elasticsearch.NewClient(cfg)
if err != nil {
log.Fatalf("elasticsearch.NewClient: %v", err)
}
}
//This method will be called in parallel
func HttpExample(w http.ResponseWriter, r *http.Request) {
... # Client usage
}
Note: I am an expert to give a recommendation on using multiple bulkIndexer.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论