ElasticSearch通过重定向的URL进行REST HTTP请求

huangapple
go评论95阅读模式
英文:

ElesticSearch REST HTTP request through redirected URL

问题

以下是翻译好的部分：

首先，让我们从一个案例开始。

假设我有这个标准的 ElasticSearch URL：http://192.168.75.197:9200

根据 ElasticSearch 文档：https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/_changing_the_client_8217_s_initialization_code.html

我可以使用以下 Java 代码请求它：

  1. package com.acme.elasticsearch;
  3. import java.net.URL;
  4. import org.apache.http.HttpHost;
  5. import org.apache.http.auth.AuthScope;
  6. import org.apache.http.auth.UsernamePasswordCredentials;
  7. import org.apache.http.client.CredentialsProvider;
  8. import org.apache.http.impl.client.BasicCredentialsProvider;
  9. import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
  10. import org.elasticsearch.action.get.GetRequest;
  11. import org.elasticsearch.action.get.GetResponse;
  12. import org.elasticsearch.client.RequestOptions;
  13. import org.elasticsearch.client.RestClient;
  14. import org.elasticsearch.client.RestClientBuilder;
  15. import org.elasticsearch.client.RestClientBuilder.HttpClientConfigCallback;
  16. import org.elasticsearch.client.RestHighLevelClient;
  18. public class TestRedirect {
  19.     private static final String rawURL = "http://192.168.75.197:9200";
  20.     private static final String index = "demoindex02";
  21.     private static final String id = "Zw6AXnEB8ovVkXyeZ9wF";
  23.     public static void main(String[] args) throws Exception {
  24.         // ...
  25.     }
  26. }

现在我可以在控制台中看到一个好的结果：

接下来是问题：ElasticSearch 的 URL 被重定向了（例如，在 Apache、Nginx 或 F5 的前面）

新的 URL 变成：http://192.168.75.197:8060/v1/es/

这个新的 URL 是有效的：

  1. 仅更改 URL

我只调整了我的 Java 代码中的新 URL（客户端和请求保持不变）：

  1. private static final String rawURL = "http://192.168.75.197:8060/v1/es/";

问题是：404 Not Found

  1. 更改 URL 和客户端

我只调整了我的 Java 代码中的新 URL 并将路径添加到客户端（请求保持不变）：

  1. private static final String rawURL = "http://192.168.75.197:8060/v1/es/";
  2. RestClientBuilder restClientBuilder = RestClient.builder(new HttpHost(host + path, port, protocol));

问题是：Unknown host 192.168.75.197/v1/es/

  1. 更改 URL、客户端和请求

我只调整了我的 Java 代码中的新 URL，并在客户端和请求中添加了路径：

  1. private static final String rawURL = "http://192.168.75.197:8060/v1/es/";
  2. RestClientBuilder restClientBuilder = RestClient.builder(new HttpHost(host + path, port, protocol));
  3. GetRequest getRequest = new GetRequest(path + index, id);

问题与上述第2点完全相同：Unknown host 192.168.75.197/v1/es/

  1. 仅更改 URL 和请求

我只调整了我的 Java 代码中的新 URL，并在请求中添加了路径（客户端保持不变）：

  1. private static final String rawURL = "http://192.168.75.197:8060/v1/es/";
  2. RestClientBuilder restClientBuilder = RestClient.builder(new HttpHost(host, port, protocol));
  3. GetRequest getRequest = new GetRequest(path + index, id);

问题与上述第1点完全相同：404 Not Found

英文:

I am asking community help because I have some trouble requesting an ElasticSearch database with a special redirected URL.

Can you please give an hand on this ?

Thank you very much.

First, let's start with a fine case.

Let's assume that I have this standard ElasticSearch URL : http://192.168.75.197:9200

With ElasticSearch documentation : https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/_changing_the_client_8217_s_initialization_code.html

I can request it with this Java code :

<!-- begin snippet: js hide: false console: true babel: false -->

<!-- language: lang-java -->

  1. package com.acme.elasticsearch;
  3. import java.net.URL;
  5. import org.apache.http.HttpHost;
  6. import org.apache.http.auth.AuthScope;
  7. import org.apache.http.auth.UsernamePasswordCredentials;
  8. import org.apache.http.client.CredentialsProvider;
  9. import org.apache.http.impl.client.BasicCredentialsProvider;
  10. import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
  11. import org.elasticsearch.action.get.GetRequest;
  12. import org.elasticsearch.action.get.GetResponse;
  13. import org.elasticsearch.client.RequestOptions;
  14. import org.elasticsearch.client.RestClient;
  15. import org.elasticsearch.client.RestClientBuilder;
  16. import org.elasticsearch.client.RestClientBuilder.HttpClientConfigCallback;
  17. import org.elasticsearch.client.RestHighLevelClient;
  19. public class TestRedirect {
  20. 	private static final String rawURL = &quot;http://192.168.75.197:9200&quot;;
  21. 	private static final String index = &quot;demoindex02&quot;;
  22. 	private static final String id = &quot;Zw6AXnEB8ovVkXyeZ9wF&quot;;
  24. 	public static void main(String[] args) throws Exception {
  25. 		// parse URL
  26. 		System.out.println(&quot;URL = &quot; + rawURL);
  27. 		URL parsedHttpUrl = new URL(rawURL);
  28. 		String protocol = parsedHttpUrl.getProtocol();
  29. 		String host = parsedHttpUrl.getHost();
  30. 		int port = parsedHttpUrl.getPort();
  31. 		String path = parsedHttpUrl.getPath();
  32. 		System.out.println(&quot;protocol = &quot; + protocol);
  33. 		System.out.println(&quot;host = &quot; + host);
  34. 		System.out.println(&quot;port = &quot; + port);
  35. 		System.out.println(&quot;path = &quot; + path);
  36. 		// instantiate HTTP credentials
  37. 		final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
  38. 		credentialsProvider.setCredentials(AuthScope.ANY,
  39. 				new UsernamePasswordCredentials(Credential.user, Credential.password));
  40. 		// instantiate REST high level client
  41. 		RestClientBuilder restClientBuilder = RestClient.builder(new HttpHost(host, port, protocol));
  42. 		restClientBuilder.setHttpClientConfigCallback(new HttpClientConfigCallback() {
  43. 			@Override
  44. 			public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
  45. 				return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
  46. 			}
  47. 		});
  48. 		RestHighLevelClient restHighLevelClient = new RestHighLevelClient(restClientBuilder);
  49. 		// simple get request
  50. 		GetRequest getRequest = new GetRequest(index, id);
  51. 		GetResponse getResponse = restHighLevelClient.get(getRequest, RequestOptions.DEFAULT);
  52. 		System.out.println(&quot;getResponse = &quot; + getResponse);
  53. 	}
  55. }

<!-- end snippet -->
I can see in my console a fine result :

<!-- begin snippet: js hide: false console: true babel: false -->

  1. URL = http://192.168.75.197:9200
  2. protocol = http
  3. host = 192.168.75.197
  4. port = 9200
  5. path = 
  6. getResponse = {&quot;_index&quot;:&quot;demoindex02&quot;,&quot;_type&quot;:&quot;_doc&quot;,&quot;_id&quot;:&quot;Zw6AXnEB8ovVkXyeZ9wF&quot;,&quot;_version&quot;:1,&quot;_seq_no&quot;:16,&quot;_primary_term&quot;:1,&quot;found&quot;:true,&quot;_source&quot;:{&quot;id&quot;:&quot;1&quot;,&quot;firstName&quot;:&quot;Jason&quot;,&quot;lastName&quot;:&quot;GIBBS&quot;,&quot;title&quot;:&quot;Mister&quot;,&quot;company&quot;:&quot;Mister&quot;,&quot;phones&quot;:[&quot;(413)442-5250&quot;,&quot;(413)442-5252&quot;,&quot;(413)454-5663&quot;]}}

<!-- end snippet -->

Now comes the issue : the ElasticSearch URL is redirected (for example, in front of Apache, Nginx or F5 ...)

The URL becomes : http://192.168.75.197:8060/v1/es/

This new URL is valid :

ElasticSearch通过重定向的URL进行REST HTTP请求

1. change only URL

I adapt my java code with just new URL (client and request remain the same) :

<!-- begin snippet: js hide: false console: true babel: false -->

<!-- language: lang-java -->

  1. private static final String rawURL = &quot;http://192.168.75.197:8060/v1/es/&quot;;
  2. RestClientBuilder restClientBuilder = RestClient.builder(new HttpHost(host, port, protocol));
  3. GetRequest getRequest = new GetRequest(index, id);

<!-- end snippet -->

The issue is : 404 Not Found

<!-- begin snippet: js hide: false console: true babel: false -->

  1. Exception in thread &quot;main&quot; ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...
  2. ElasticsearchStatusException[Unable to parse response body]; nested: ResponseException[method [GET], host [http://192.168.75.197:8060], URI [/demoindex02/_doc/Zw6AXnEB8ovVkXyeZ9wF], status line [HTTP/1.1 404 Not Found]
  3. &lt;html&gt;
  4. &lt;head&gt;&lt;title&gt;404 Not Found&lt;/title&gt;&lt;/head&gt;
  5. &lt;body&gt;
  6. &lt;center&gt;&lt;h1&gt;404 Not Found&lt;/h1&gt;&lt;/center&gt;
  7. &lt;hr&gt;&lt;center&gt;nginx/1.16.1&lt;/center&gt;
  8. &lt;/body&gt;
  9. &lt;/html&gt;
  10. ];
  11. 	at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1686)
  12. 	at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1443)
  13. 	at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1403)
  14. 	at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1373)
  15. 	at org.elasticsearch.client.RestHighLevelClient.get(RestHighLevelClient.java:699)
  16. 	at com.acme.elasticsearch.TestRedirect.main(TestRedirect.java:52)
  17. 	Suppressed: java.lang.IllegalStateException: Unsupported Content-Type: text/html
  18. 		at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1703)
  19. 		at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1683)
  20. 		... 5 more
  21. Caused by: org.elasticsearch.client.ResponseException: method [GET], host [http://192.168.75.197:8060], URI [/demoindex02/_doc/Zw6AXnEB8ovVkXyeZ9wF], status line [HTTP/1.1 404 Not Found]
  22. &lt;html&gt;
  23. &lt;head&gt;&lt;title&gt;404 Not Found&lt;/title&gt;&lt;/head&gt;
  24. &lt;body&gt;
  25. &lt;center&gt;&lt;h1&gt;404 Not Found&lt;/h1&gt;&lt;/center&gt;
  26. &lt;hr&gt;&lt;center&gt;nginx/1.16.1&lt;/center&gt;
  27. &lt;/body&gt;
  28. &lt;/html&gt;
  30. 	at org.elasticsearch.client.RestClient.convertResponse(RestClient.java:260)
  31. 	at org.elasticsearch.client.RestClient.performRequest(RestClient.java:238)
  32. 	at org.elasticsearch.client.RestClient.performRequest(RestClient.java:212)
  33. 	at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1433)
  34. 	... 4 more

<!-- end snippet -->

2. change URL & client only

I adapt my java code with just new URL and add path to client (request remains the same) :

<!-- begin snippet: js hide: false console: true babel: false -->

<!-- language: lang-java -->

  1. private static final String rawURL = &quot;http://192.168.75.197:8060/v1/es/&quot;;
  2. RestClientBuilder restClientBuilder = RestClient.builder(new HttpHost(host + path, port, protocol));
  3. GetRequest getRequest = new GetRequest(index, id);

<!-- end snippet -->

The issue is : Unknown host 192.168.75.197/v1/es/

<!-- begin snippet: js hide: false console: true babel: false -->

  1. URL = http://192.168.75.197:8060/v1/es/
  2. protocol = http
  3. host = 192.168.75.197
  4. port = 8060
  5. path = /v1/es/
  6. Exception in thread &quot;main&quot; java.io.IOException: 192.168.75.197/v1/es/
  7. 	at org.elasticsearch.client.RestClient.extractAndWrapCause(RestClient.java:809)
  8. 	at org.elasticsearch.client.RestClient.performRequest(RestClient.java:225)
  9. 	at org.elasticsearch.client.RestClient.performRequest(RestClient.java:212)
  10. 	at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1433)
  11. 	at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1403)
  12. 	at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1373)
  13. 	at org.elasticsearch.client.RestHighLevelClient.get(RestHighLevelClient.java:699)
  14. 	at com.acme.elasticsearch.TestRedirect.main(TestRedirect.java:52)
  15. Caused by: java.net.UnknownHostException: 192.168.75.197/v1/es/
  16. 	at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
  17. 	at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929)
  18. 	at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324)
  19. 	at java.net.InetAddress.getAllByName0(InetAddress.java:1277)
  20. 	at java.net.InetAddress.getAllByName(InetAddress.java:1193)
  21. 	at java.net.InetAddress.getAllByName(InetAddress.java:1127)
  22. 	at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45)
  23. 	at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager$InternalAddressResolver.resolveRemoteAddress(PoolingNHttpClientConnectionManager.java:664)
  24. 	at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager$InternalAddressResolver.resolveRemoteAddress(PoolingNHttpClientConnectionManager.java:635)
  25. 	at org.apache.http.nio.pool.AbstractNIOConnPool.processPendingRequest(AbstractNIOConnPool.java:474)
  26. 	at org.apache.http.nio.pool.AbstractNIOConnPool.lease(AbstractNIOConnPool.java:280)
  27. 	at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.requestConnection(PoolingNHttpClientConnectionManager.java:295)
  28. 	at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.requestConnection(AbstractClientExchangeHandler.java:377)
  29. 	at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.start(DefaultClientExchangeHandlerImpl.java:129)
  30. 	at org.apache.http.impl.nio.client.InternalHttpAsyncClient.execute(InternalHttpAsyncClient.java:141)
  31. 	at org.elasticsearch.client.RestClient.performRequest(RestClient.java:221)
  32. 	... 6 more

<!-- end snippet -->

3. change URL, client and request

I adapt my java code with just new URL, add path to both client & request :

<!-- begin snippet: js hide: false console: true babel: false -->

<!-- language: lang-html -->

  1. private static final String rawURL = &quot;http://192.168.75.197:8060/v1/es/&quot;;
  2. RestClientBuilder restClientBuilder = RestClient.builder(new HttpHost(host + path, port, protocol));
  3. GetRequest getRequest = new GetRequest(path + index, id);

<!-- end snippet -->

The issue is the exact same as in point #2 above : Unknown host 192.168.75.197/v1/es/

4. change URL & request only

I adapt my java code with just new URL and add path to request (client remains the same) :

<!-- begin snippet: js hide: false console: true babel: false -->

<!-- language: lang-html -->

  1. private static final String rawURL = &quot;http://192.168.75.197:8060/v1/es/&quot;;
  2. RestClientBuilder restClientBuilder = RestClient.builder(new HttpHost(host, port, protocol));
  3. GetRequest getRequest = new GetRequest(path + index, id);

<!-- end snippet -->

The issue is the exact same as in point #1 above : 404 Not Found

答案1

得分: 0

  1. 最终，答案在这里：[RestHighLevelClient - 访问位于反向代理后面的 Elastic HTTP 终端][1]
  3. 因此，我按照以下方式修改了我的代码：
  5. &lt;!-- begin snippet: js hide: false console: true babel: false --&gt;
  7. &lt;!-- language: lang-java --&gt;
  9. // 实例化 REST 高级客户端
  10. RestClientBuilder restClientBuilder = RestClient.builder(new HttpHost(host, port, protocol));
  11. restClientBuilder.setHttpClientConfigCallback(new HttpClientConfigCallback() {
  12.     @Override
  13.     public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
  14.         return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
  15.     }
  16. });
  17. restClientBuilder.setPathPrefix(path);
  18. RestHighLevelClient restHighLevelClient = new RestHighLevelClient(restClientBuilder);
  20. &lt;!-- end snippet --&gt;
  22. [1]: https://discuss.elastic.co/t/resthighlevelclient-accessing-an-elastic-http-endpoint-behind-reverse-proxy/117306/2
英文:

Finally, answer is here : RestHighLevelClient - Accessing an elastic http endpoint behind reverse proxy

So I modified my code this way :

<!-- begin snippet: js hide: false console: true babel: false -->

<!-- language: lang-java -->

  1. 		// instantiate REST high level client
  2. 		RestClientBuilder restClientBuilder = RestClient.builder(new HttpHost(host, port, protocol));
  3. 		restClientBuilder.setHttpClientConfigCallback(new HttpClientConfigCallback() {
  4. 			@Override
  5. 			public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
  6. 				return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
  7. 			}
  8. 		});
  9. 		restClientBuilder.setPathPrefix(path);
  10. 		RestHighLevelClient restHighLevelClient = new RestHighLevelClient(restClientBuilder);

<!-- end snippet -->

huangapple
  • 本文由 发表于 2020年4月10日 18:17:09
  • 转载请务必保留本文链接：https://go.coder-hub.com/61138183.html
  • elasticsearch
  • java
  • redirect
  • rest
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定