英文:
ConnectionError ECONNREFUSED connecting to OpenSearch on localhost from asp.net core app
问题
我在Amazon OpenSearch Service上有一个OpenSearch域。我在我的asp.net核心应用程序中使用它。现在我想在开发过程中使用默认的docker-compose文件从快速入门文档中运行本地主机，而不是使用Amazon OpenSearch服务。我运行docker-compose文件一切正常。但当我运行我的asp.net应用程序时，在docker终端中开始看到以下内容：
> "error"，"opensearch"，"data"，"pid"：453，"message"："[ConnectionError]：> connect ECONNREFUSED 172.18.0.4:9200"
编辑：我也经常看到这个：
> 由io.netty.handler.ssl.NotSslRecordException引起：不是SSL/TLS记录
我相当确定我在我的asp.net应用程序中应该更改的唯一事物是appsettings.json中的这些连接字符串：
"elasticEndpoint": "https://search-xxxxxxxxxxxxxxxxx.ap-southeast-2.es.amazonaws.com",
"elasticIndexName": "vepo",
"elasticUsername": "xxx",
"elasticPassword": "xxx"
所以我将它更改为：
"elasticEndpoint": "http://localhost:9200",
"elasticIndexName": "vepo",
"elasticUsername": "admin",
"elasticPassword": "admin"
据我所知，这应该是所需的一切，有人知道我是否漏掉了什么或者做错了吗？
我现在将在下面提供大量信息，你可能不需要，但在找到ECONNREFUSED错误之前，我写了它，可能会有用，谁知道呢：
每次我启动asp.net应用程序时，它都会删除默认索引并重新创建它，然后从我的Postgres数据库重新创建每个其他索引并填充它。
这是我创建OpenSearch客户端的地方：
public class SearchService : ConfigurableSearchService, ISearchService
{
private readonly string _defaultIndex;
private readonly bool _refreshOnUpdate;
private OpenSearchClient _client;
public static readonly Dictionary<Type, string> SearchIndexMappings = new Dictionary<Type, string>
{
{typeof(VgnItmSearchDto), "vgnitmsearchdto"},
{typeof(VgnItmEstSearchDto), "vgnitmestsearchdto"},
};
public string DefaultIndex => _defaultIndex;
public SearchService(
string uri,
string username,
string password,
string defaultIndex,
bool forceInMemory = false,
bool refreshOnUpdate = false)
: base(SearchService.SearchIndexMappings)
{
ConnectionSettings connectionSettings;
_defaultIndex = defaultIndex;
_refreshOnUpdate = refreshOnUpdate;
var node = new Uri(uri);
IConnectionPool pool = new SingleNodeConnectionPool(node);
connectionSettings = forceInMemory
? new ConnectionSettings(pool, new InMemoryConnection())
: new ConnectionSettings(pool).BasicAuthentication(username, password);
connectionSettings = connectionSettings
.DefaultIndex(defaultIndex);
IConnectionSettingsValues settings = connectionSettings;
foreach (var mapping in SearchService.SearchIndexMappings)
{
settings.DefaultIndices.Add(mapping.Key, mapping.Value);
}
#if (DEBUG)
connectionSettings.DisableDirectStreaming();
#endif
_client = new OpenSearchClient(connectionSettings);
}
}
这在Startup.cs的ConfigureServices()中调用：
services.AddScoped<ISearchService, SearchService>(serviceProvider =>
{
return new SearchService(
Configuration["elasticEndpoint"],
Configuration["elasticUsername"],
Configuration["elasticPassword"],
Configuration["elasticIndexName"]);
});
Configuration从appsettings.json中使用这个，这是AWS的一个：
"elasticEndpoint": "https://search-xxxxxxxxxxxxxxxxx.ap-southeast-2.es.amazonaws.com",
"elasticIndexName": "xxx",
"elasticUsername": "xxx",
"elasticPassword": "xxx"
所以我将它更改为：
"elasticEndpoint": "http://localhost:9200",
"elasticIndexName": "vepo",
"elasticUsername": "admin",
"elasticPassword": "admin"
在Configure()的底部，我告诉searchService重新索引所有内容：
searchIndexService.ReIndex(context, SearchIndexes.All);
要在本地运行OpenSearch，我使用了快速入门文档中的默认docker-compose文件：
version: '3'
services:
opensearch-node1:
image: opensearchproject/opensearch:latest
container_name: opensearch-node1
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node1
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
hard: 65536
volumes:
- opensearch-data1:/usr/share/opensearch/data
ports:
- 9200:9200
- 9600:9600 # required for Performance Analyzer
networks:
- opensearch-net
opensearch-node2:
image: opensearchproject/opensearch:latest
container_name: opensearch-node2
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node2
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
volumes:
- opensearch-data2:/usr/share/opensearch/data
networks:
- opensearch-net
opensearch-dashboards:
image: opensearchproject/opensearch-dashboards:latest
container_name: opensearch-dashboards
ports:
- 5601
<details>
<summary>英文:</summary>
I have an OpenSearch domain on Amazon OpenSearch Service. I use it for my asp.net core app. I now want to start using localhost by running the default docker-compose file from the quickstart docs, instead of using the Amazon OpenSearch Service (while in development). I run the docker-compose file and it is all good. But when I run my asp.net app I start to see this come through in the docker terminal:
> "error","opensearch","data"],"pid":453,"message":"[ConnectionError]:
> connect ECONNREFUSED 172.18.0.4:9200"
**EDIT: I also see this a lot:**
> Caused by: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS
> record
I'm pretty sure the only thing I should be changing in my asp.net app is these connection strings in appsettings.json:
"elasticEndpoint": "https://search-xxxxxxxxxxxxxxxxx.ap-southeast-2.es.amazonaws.com",
"elasticIndexName": "vepo",
"elasticUsername": "xxx",
"elasticPassword": "xxx"
So I changed it to:
"elasticEndpoint": "http://localhost:9200",
"elasticIndexName": "vepo",
"elasticUsername": "admin",
"elasticPassword": "admin"
As far as I know that should be all that is needed, anyone know if I'm missing something or did it wrong?
I will now dump a massive amount of info below that you probably don't need but I wrote it before I found the ECONNREFUSED error and it may be useful, who knows:
Every time I start the asp.net app it deletes the default index and recreates it, then recreates every other index and populates it from my Postgres database.
This is where i create the OpenSearch client:
public class SearchService : ConfigurableSearchService, ISearchService
{
private readonly string _defaultIndex;
private readonly bool _refreshOnUpdate;
private OpenSearchClient _client;
public static readonly Dictionary<Type, string> SearchIndexMappings = new Dictionary<Type, string>
{
{typeof(VgnItmSearchDto), "vgnitmsearchdto"},
{typeof(VgnItmEstSearchDto), "vgnitmestsearchdto"},
};
public string DefaultIndex { get => _defaultIndex; }
public SearchService(
string uri,
string username,
string password,
string defaultIndex,
bool forceInMemory = false,
bool refreshOnUpdate = false)
: base(SearchService.SearchIndexMappings)
{
ConnectionSettings connectionSettings;
_defaultIndex = defaultIndex;
_refreshOnUpdate = refreshOnUpdate;
var node = new Uri(uri);
IConnectionPool pool = new SingleNodeConnectionPool(node);
connectionSettings = forceInMemory
? new ConnectionSettings(pool, new InMemoryConnection())
: new ConnectionSettings(pool).BasicAuthentication(username, password);
connectionSettings = connectionSettings
.DefaultIndex(defaultIndex);
IConnectionSettingsValues settings = connectionSettings;
foreach (var mapping in SearchService.SearchIndexMappings)
{
settings.DefaultIndices.Add(mapping.Key, mapping.Value);
}
#if (DEBUG)
connectionSettings.DisableDirectStreaming();
#endif
_client = new OpenSearchClient(connectionSettings);
}
And that is called in `Startup.cs`, `ConfigureServices()`:
services.AddScoped<ISearchService, SearchService>(serviceProvider =>
{
return new SearchService(
Configuration["elasticEndpoint"],
Configuration["elasticUsername"],
Configuration["elasticPassword"],
Configuration["elasticIndexName"]);
});
The Configuration is using this from appsettings.json, this was the AWS one:
"elasticEndpoint": "https://search-xxxxxxxxxxxxxxxxx.ap-southeast-2.es.amazonaws.com",
"elasticIndexName": "xxx",
"elasticUsername": "xxx",
"elasticPassword": "xxx"
So I changed it to:
"elasticEndpoint": "http://localhost:9200",
"elasticIndexName": "vepo",
"elasticUsername": "admin",
"elasticPassword": "admin"
At the bottom of `Configure()` I tell the searchService to reindex all:
`searchIndexService.ReIndex(context, SearchIndexes.All);`
To run OpenSearch locally, I have the default docker-compose file from the quick start docs:
version: '3'
services:
opensearch-node1:
image: opensearchproject/opensearch:latest
container_name: opensearch-node1
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node1
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
hard: 65536
volumes:
- opensearch-data1:/usr/share/opensearch/data
ports:
- 9200:9200
- 9600:9600 # required for Performance Analyzer
networks:
- opensearch-net
opensearch-node2:
image: opensearchproject/opensearch:latest
container_name: opensearch-node2
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node2
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
volumes:
- opensearch-data2:/usr/share/opensearch/data
networks:
- opensearch-net
opensearch-dashboards:
image: opensearchproject/opensearch-dashboards:latest
container_name: opensearch-dashboards
ports:
- 5601:5601
expose:
- "5601"
environment:
OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
networks:
- opensearch-net
volumes:
opensearch-data1:
opensearch-data2:
networks:
opensearch-net:
I thought that if I run OpenSearch locally with docker-compose up, then I run my asp.net app, that it would then create my indexes in my local version of OpenSearch viewable at `http://localhost:5601/app/home#/`. But it doesn't. The Create index code all runs without any exceptions.
But my indexes do not get added there. And when I perform a search, in my asp.net app I get:
> System.IO.IOException: The response ended prematurely. at
> System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage
> request, Boolean async, CancellationToken cancellationToken)
And the OpenSearch docker terminal shows:
> io.netty.handler.codec.DecoderException:
> io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record
What am I missing that seems to be not connecting my asp.net app to my localhost version of OpenSearch?
</details>
# 答案1
**得分**: 1
first: *如果您正在从Docker容器中运行应用程序，请不要使用localhost，而要使用主机机器的本地IP。*
---
我已经阅读了这个问题，并发现opensearch在所有连接上都使用**TLS证书**：节点之间和客户端与节点之间。
https://opensearch.org/docs/latest/security/configuration/tls/
因此，您可以自己创建并安装证书，以允许您的应用程序（客户端）连接到本地集群（节点）。但是，如果您认为在本地部署中不需要安装这些证书，我建议您避免SSL验证。
**首先**，如果您有curl（bash终端），请执行以下命令：
```bash
curl https://admin:admin@localhost:9200 -k
-k 选项要求curl不进行SSL验证，admin:admin是用户名：密码。如果该命令有效，这个解决方案将适用于您。
禁用Opensearch中的安全插件
在docker-composer.yaml文件中，将“DISABLE_SECURITY_PLUGIN=true”添加到环境变量中：
...
container_name: opensearch-node1
environment:
- cluster.name=opensearch-cluster # 为集群命名
- node.name=opensearch-node1 # 为此容器中运行的节点命名
- discovery.seed_hosts=opensearch-node1,opensearch-node2 # 在发现集群时要查找的节点
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2 # 有资格作为集群管理器的节点
- bootstrap.memory_lock=true # 禁用JVM堆内存交换
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # 将最小和最大JVM堆大小设置为系统RAM的至少50%
- "DISABLE_SECURITY_PLUGIN=true"
...
和
...
container_name: opensearch-node2
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node2
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
- "DISABLE_SECURITY_PLUGIN=true"
...
现在您可以使用http而不是https：
curl http://admin:admin@localhost:9200
禁用应用程序中的SSL
我不擅长ASP.NET Core，但我进行了一次禁用SSL验证的测试，它有效（使用HttpClient和您的composer.yaml文件）。
var handler = new HttpClientHandler();
handler.ClientCertificateOptions = ClientCertificateOption.Manual;
handler.ServerCertificateCustomValidationCallback =
(httpRequestMessage, cert, cetChain, policyErrors) =>
{
return true;
};
string url = "https://localhost:9200";
HttpClient client = new HttpClient(handler);
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", "YWRtaW46YWRtaW4=");
using HttpResponseMessage response = await client.GetAsync(url);
var jsonResponse = await response.Content.ReadAsStringAsync();
Console.WriteLine($"{jsonResponse}\n");
YWRtaW46YWRtaW4= 是从“user:password”（“admin:admin”）编码为base64字符串。
输出：
{
"name" : "opensearch-node1",
"cluster_name" : "opensearch-cluster",
"cluster_uuid" : "mHWmnO2ORguqfe4F-0nJrQ",
"version" : {
"distribution" : "opensearch",
"number" : "2.8.0",
"build_type" : "tar",
"build_hash" : "db90a415ff2fd428b4f7b3f800a51dc229287cb4",
"build_date" : "2023-06-03T06:24:25.112415503Z",
"build_snapshot" : false,
"lucene_version" : "9.6.0",
"minimum_wire_compatibility_version" : "7.10.0",
"minimum_index_compatibility_version" : "7.0.0"
},
"tagline" : "The OpenSearch Project: https://opensearch.org/"
}
正如您所看到的，它可以正常工作，因此我认为您应该查看OpenSearch.Client包，了解如何禁用SSL，如果可能的话，这是一个解决方案。
Nginx解决方案
此外，您可以从应用程序之外查看问题：您的应用程序需要一个端点，无需关心证书。因此，我们可以提供一个准备好的、无需SSL证书的端点供您使用。
如何？
使用Nginx容器作为“https://opensearch-node1:9200”的反向代理。默认情况下，Nginx代理不验证SSL证书，因此您可以向http://nginx-service:80发出请求，它将正常工作。
步骤：
1）在与docker-compose.yaml相同的文件夹中创建一个nginx.conf文件，并将其粘贴到其中。
server {
listen 80;
listen [::]:80;
location / {
proxy_pass https://opensearch-node1:9200;
}
}
2）将此服务添加到您的docker-composer.yaml中：
nginx-proxy:
image: nginx:latest
container_name: nginx-proxy
ports:
- 80:80
volumes:
- ./nginx.conf:/etc/nginx/conf.d/default.conf:ro
networks:
- opensearch-net
3）启动您的容器：
docker-compose up
4）使用curl（或其他工具如Postman）测试nginx是否工作：
curl http://admin:admin@localhost
5）在您的ASP.NET Core应用程序中更改端点，例如：
public class Student
{
public int Id { get; init; }
public string FirstName { get; init; }
public string LastName { get;
<details>
<summary>英文:</summary>
first: *If you are running your app from a docker container not use localhost, use the local IP of the host machine.*
---
I have read about the issue and I found that opensearch uses **tls certificates** for all connections: node-node, and client-node.
https://opensearch.org/docs/latest/security/configuration/tls/
So you could create and install the certificates by yourself to allow your app (client) to connect to the local cluster (node). But if you think is not necessary to install these certificates in the local deployment, I suggest you avoid the SSL verification.
---
**before all**, if you have curl (bash terminal) execute this command:
```bash
curl https://admin:admin@localhost:9200 -k
the -k asks curl not make ssl validation and admin:admin is username:password. And if the command works this solution will work for you.
Disabling the Security plugin in Opensearch
in docker-composer.yaml add "DISABLE_SECURITY_PLUGIN=true" to the environments:
...
container_name: opensearch-node1
environment:
- cluster.name=opensearch-cluster # Name the cluster
- node.name=opensearch-node1 # Name the node that will run in this container
- discovery.seed_hosts=opensearch-node1,opensearch-node2 # Nodes to look for when discovering the cluster
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2 # Nodes eligible to serve as cluster manager
- bootstrap.memory_lock=true # Disable JVM heap memory swapping
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # Set min and max JVM heap sizes to at least 50% of system RAM
- "DISABLE_SECURITY_PLUGIN=true"
...
and
...
container_name: opensearch-node2
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node2
- discovery.seed_hosts=opensearch-node1,opensearch-node2
- cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
- bootstrap.memory_lock=true
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
- "DISABLE_SECURITY_PLUGIN=true"
...
now you can use http instead https:
curl http://admin:admin@localhost:9200
Disabling SSL on your app
I'm not good at asp net core but I made a test disabling SSL validation and it worked(using HttpClient and your composer.yaml file).
var handler = new HttpClientHandler();
handler.ClientCertificateOptions = ClientCertificateOption.Manual;
handler.ServerCertificateCustomValidationCallback =
(httpRequestMessage, cert, cetChain, policyErrors) =>
{
return true;
};
string url = "https://localhost:9200";
HttpClient client = new HttpClient(handler);
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", "YWRtaW46YWRtaW4=");
using HttpResponseMessage response = await client.GetAsync(url);
var jsonResponse = await response.Content.ReadAsStringAsync();
Console.WriteLine($"{jsonResponse}\n");
YWRtaW46YWRtaW4= is a base64 string encode from "user:password" ("admin:admin")
the output:
{
"name" : "opensearch-node1",
"cluster_name" : "opensearch-cluster",
"cluster_uuid" : "mHWmnO2ORguqfe4F-0nJrQ",
"version" : {
"distribution" : "opensearch",
"number" : "2.8.0",
"build_type" : "tar",
"build_hash" : "db90a415ff2fd428b4f7b3f800a51dc229287cb4",
"build_date" : "2023-06-03T06:24:25.112415503Z",
"build_snapshot" : false,
"lucene_version" : "9.6.0",
"minimum_wire_compatibility_version" : "7.10.0",
"minimum_index_compatibility_version" : "7.0.0"
},
"tagline" : "The OpenSearch Project: https://opensearch.org/"
}
As you can see it works, so I think you should look into OpenSearch.Client package to know how to disable SSL and if it is possible, you have a solution.
Nginx solution
Also, you can see the issue from outside of your application: your app needs an endpoint and doesn't need to concern about certificates. So we can provide an endpoint ready to consume without SSL certificates.
How ?
Using an Nginx container as a reverse proxy for "https://opensearch-node1:9200". Nginx proxy by default does not validate SSL certificates, then you could make a request to http://nginx-service:80 and it will work.
steps:
- create a nginx.conf file in the same folder of docker-compose.yaml and paste it into it.
server {
listen 80;
listen [::]:80;
location / {
proxy_pass https://opensearch-node1:9200;
}
}
- add this service to your docker-composer.yaml
nginx-proxy:
image: nginx:latest
container_name: nginx-proxy
ports:
- 80:80
volumes:
- ./nginx.conf:/etc/nginx/conf.d/default.conf:ro
networks:
- opensearch-net
- start your containers:
docker-compose up
- test if the nginx works with curl (of another tool like Postman):
curl http://admin:admin@localhost
- change the endpoint in your asp net core app, for example:
public class Student
{
public int Id { get; init; }
public string FirstName { get; init; }
public string LastName { get; init; }
public int GradYear { get; init; }
public double Gpa { get; init; }
}
string url = "http://localhost";
Console.WriteLine(url);
var node = new Uri(url);
var config = new ConnectionSettings(node).BasicAuthentication("admin", "admin").DefaultIndex("students");
var client = new OpenSearchClient(config);
Console.WriteLine($"{config}\n");
var student = new Student { Id = 100, FirstName = "Paulo", LastName = "Santos", Gpa = 3.93, GradYear = 2021 };
var response = client.Index(student, i => i.Index("students"));
Console.WriteLine($"{response}\n");
Valid OpenSearch.Client response built from a successful (201) low level call on PUT: /students/_doc/100
- read indexes:
curl http://admin:admin@localhost/students/_doc/100
{"_index":"students","_id":"100","_version":2,"_seq_no":1,"_primary_term":2,"found":true,"_source":{"id":100,"firstName":"Paulo","lastName":"Santos","gradYear":2021,"gpa":3.93}}
I hope it helps you.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论