英文:
Why is HttpUriRequest.getURI() returning null when the uri variable is not null?
问题
概要
我遇到了一个独特的情况，其中对httpcomponents的HttpUriRequest方法getURI（）的调用返回null，但实例中的变量不为null。在这种情况下，HttpUriRequest.getURI（）不可能返回null，但却发生了这种情况。
代码演练
这是代码演练。它演示了HttpUriRequest.getURI（）方法绝对不可能返回null，但却返回了null的情况。
使用的相关库是
- com.sun.jersey.contribs:jersey-apache-client4:jar:1.19.4
- org.apache.httpcomponents:httpclient:jar:4.5.12
我们的内部应用程序调用使用jersey-apache-client4进行通过apache httpcomponents httpclient向外部系统发出RESTful调用的第三方代码。
为了这个问题的目的，请求始于对ApacheHttpClient4Handler.handle（ClientRequest）的调用，源代码如下。
- 标有数字的内联注释将引导您进行演示。
类ApacheHttpClient4Handler v1.19.4的相关源代码
package com.sun.jersey.client.apache4;
// ... 删除了不相关的代码
public final class ApacheHttpClient4Handler extends TerminatingClientHandler {
private final HttpClient client;
private final boolean preemptiveBasicAuth;
// ... 删除了不相关的代码
public ClientResponse handle(final ClientRequest cr) throws ClientHandlerException {
// {1} 这是创建HttpUriRequest实例的地方。
// 请参阅下面的getUriHttpRequest（ClientRequest）源代码演练。
final HttpUriRequest request = getUriHttpRequest(cr);
// {8} - 在这里调用request.getURI（）返回非null值，记录产生预期的“https://{serverpath}”toString（）。
writeOutBoundHeaders(cr.getHeaders(), request);
// {11} - 在这里调用request.getURI（）返回非null值，记录产生预期的“https://{serverpath}”toString（）。
try {
HttpResponse response;
if(preemptiveBasicAuth) {
// 在这个代码块中，永远不会进入，因为对于这种情况，preemptiveBasicAuth始终为false
// ... 删除了不相关的代码
} else {
// {12} - 在这里调用request.getURI（）返回非null值，记录产生预期的“https://{serverpath}”toString（）。
// 下面的行引发了意外的NullPointerException，源自于对getHost（request）的调用。
// 请参阅下面的getHost（HttpUriRequest）源代码演练。
response = getHttpClient().execute(getHost(request), request);
}
// 从这里到catch之间永远不会被执行，因为上面两行会引发NPE。
ClientResponse r = new ClientResponse(response.getStatusLine().getStatusCode(),
getInBoundHeaders(response),
new HttpClientResponseInputStream(response),
getMessageBodyWorkers());
if (!r.hasEntity()) {
r.bufferEntity();
r.close();
}
return r;
} catch (Exception e) {
// {15} - 在这里捕获并重新抛出NPE。
throw a ClientHandlerException(e);
}
}
// ... 删除了不相关的代码
// 这是意外NPE抛出的方法。
private HttpHost getHost(final HttpUriRequest request) {
// {13} - 在这里调用request.getURI（）返回null。
// 这不可能发生。
// {14} - 由于getURI（）返回null，将引发NPE，因为不能在null返回上调用.getHost（）。
// 为什么getURI（）返回null？？？！！！
return new HttpHost(request.getURI().getHost(), request.getURI().getPort(), request.getURI().getScheme());
}
// {2} - 这是由handle（ClientRequest）调用的
private HttpUriRequest getUriHttpRequest(final ClientRequest cr) {
final String strMethod = cr.getMethod();
final URI uri = cr.getURI();
// {3} - uri不为null，并且记录产生预期的“https://{serverpath}”toString（）。
final Boolean bufferingEnabled = isBufferingEnabled(cr);
final HttpEntity entity = getHttpEntity(cr, bufferingEnabled);
final HttpUriRequest request;
// {4} - uri不为null，并且记录产生预期的“https://{serverpath}”toString（）。
// 对于HttpGet、HttpPost、HttpPut、HttpDelete，有一半的机会；HttpHead和HttpOptions不会被这种情况引用。
// 请求是哪个类，uri最终会变成null，无论是哪个类的请求，uri最终都会变成null。
if (strMethod.equals("GET")) {
request = new HttpGet(uri);
} else if (strMethod.equals("POST")) {
request = new HttpPost(uri);
} else if (strMethod.equals("PUT")) {
request = new HttpPut(uri);
} else if (strMethod.equals("DELETE")) {
request = new HttpDelete(uri);
} else if (strMethod.equals("HEAD")) {
request = new HttpHead(uri);
} else if (strMethod.equals("OPTIONS")) {
request = new HttpOptions(uri);
} else {
// 这段代码永远不会被这种情况的请求命中。
request = new HttpEntityEnclosingRequestBase() {
@Override
public String getMethod() {
return strMethod;
}
@Override
public URI getURI() {
return uri;
}
};
}
// {5} - 检查uri显示它不为null，并且记录产生预期的“https://{serverpath}”toString（）。
// 在这里调用request.getURI（）返回非null值，并且记录产生预期的“https://{serverpath}”toString（）。
// {6} - 有时会调用它，有时不会；无论如何，错误情况仍会发生。
if(entity
<details>
<summary>英文:</summary>
SUMMARY
=======
I am encountering a unique scenario where a call to an instance of httpcomponents `HttpUriRequest` method `getURI()` is returning `null`, when the variable in the instance is not `null`. It is impossible in this scenario for `HttpUriRequest.getURI()` to return a `null`, yet that is what is happening.
CODE WALKTHROUGH
================
This is the code walkthrough. It demonstrates how the `HttpUriRequest.getURI()` method cannot possibly return a `null`, yet it is returning `null`.
The relevant libraries used are
* com.sun.jersey.contribs:jersey-apache-client4:jar:1.19.4
* org.apache.httpcomponents:httpclient:jar:4.5.12
Our in-house application calls third-party code that uses `jersey-apache-client4` to make RESTful calls to an external system via `apache httpcomponents httpclient`.
For the purposes of this question, requests begin at the call to `ApacheHttpClient4Handler.handle(ClientRequest)`, source below.
* **Numbered** in-line comments will guide the walk-through.
Relevant source for class `ApacheHttpClient4Handler` v1.19.4
``` lang-java
package com.sun.jersey.client.apache4;
// ... irrelevant code removed
public final class ApacheHttpClient4Handler extends TerminatingClientHandler {
private final HttpClient client;
private final boolean preemptiveBasicAuth;
// ... irrelevant code removed
public ClientResponse handle(final ClientRequest cr) throws ClientHandlerException {
// {1} this is where the HttpUriRequest instance is created.
// See source walkthrough of getUriHttpRequest(ClientRequest) below.
final HttpUriRequest request = getUriHttpRequest(cr);
// {8} - calling request.getURI() here returns not null, and logging produces the expected "https://{serverpath}" on toString()
writeOutBoundHeaders(cr.getHeaders(), request);
// {11} - calling request.getURI() here returns not null, and logging produces the expected "https://{serverpath}" on toString()
try {
HttpResponse response;
if(preemptiveBasicAuth) {
// this code block is never entered as preemptiveBasicAuth is always false for the scenario
// ... irrelevant code removed
} else {
// {12} - calling request.getURI() here returns not null, and logging produces the expected "https://{serverpath}" on toString()
// the following line throws the unexpected NullPointerException originating from the call to getHost(request).
// See source walkthrough of getHost(HttpUriRequest) below.
response = getHttpClient().execute(getHost(request), request);
}
// from here to the catch is never reached due to the NPE two lines above.
ClientResponse r = new ClientResponse(response.getStatusLine().getStatusCode(),
getInBoundHeaders(response),
new HttpClientResponseInputStream(response),
getMessageBodyWorkers());
if (!r.hasEntity()) {
r.bufferEntity();
r.close();
}
return r;
} catch (Exception e) {
// {15} - NPE is caught here and rethrown.
throw new ClientHandlerException(e);
}
}
// ... irrelevant code removed
// this is the method where the unexpected NPE is thrown.
private HttpHost getHost(final HttpUriRequest request) {
// {13} - calling request.getURI() here returns null.
// It is not possible for this to be happening.
// {14} - since getURI() returns null, a NPE will be thrown because .getHost() can't be called on a null return.
// WHY IS getURI() returning null ???!!!???
return new HttpHost(request.getURI().getHost(), request.getURI().getPort(), request.getURI().getScheme());
}
// {2} - this is called by handle(ClientRequest)
private HttpUriRequest getUriHttpRequest(final ClientRequest cr) {
final String strMethod = cr.getMethod();
final URI uri = cr.getURI();
// {3} - uri is not null, and logging produces the expected "https://{serverpath}" on toString()
final Boolean bufferingEnabled = isBufferingEnabled(cr);
final HttpEntity entity = getHttpEntity(cr, bufferingEnabled);
final HttpUriRequest request;
// {4} - uri is not null, and logging produces the expected "https://{serverpath}" on toString()
// even odds for a HttpGet, HttpPost, HttpPut, HttpDelete; HttpHead and HttpOptions do not get referenced by this scenario.
// it does not matter which class the request is, the uri will end up being null.
if (strMethod.equals("GET")) {
request = new HttpGet(uri);
} else if (strMethod.equals("POST")) {
request = new HttpPost(uri);
} else if (strMethod.equals("PUT")) {
request = new HttpPut(uri);
} else if (strMethod.equals("DELETE")) {
request = new HttpDelete(uri);
} else if (strMethod.equals("HEAD")) {
request = new HttpHead(uri);
} else if (strMethod.equals("OPTIONS")) {
request = new HttpOptions(uri);
} else {
// this block of code is never hit by the requests for this scenario.
request = new HttpEntityEnclosingRequestBase() {
@Override
public String getMethod() {
return strMethod;
}
@Override
public URI getURI() {
return uri;
}
};
}
// {5} - checking uri shows it is not null, and logging produces the expected "https://{serverpath}" on toString()
// calling request.getURI() here returns not null, and logging produces the expected "https://{serverpath}" on toString()
// {6} - sometimes this is called, sometimes it is not; regardless, the error scenario still happens
if(entity != null && request instanceof HttpEntityEnclosingRequestBase) {
((HttpEntityEnclosingRequestBase) request).setEntity(entity);
} else if (entity != null) {
throw new ClientHandlerException("Adding entity to http method " + cr.getMethod() + " is not supported.");
}
// ... irrelevant code removed
// {7} - calling request.getURI() here returns not null, and logging produces the expected "https://{serverpath}" on toString()
return request;
}
// ... irrelevant code removed
private void writeOutBoundHeaders(final MultivaluedMap<String, Object> headers, final HttpUriRequest request) {
// {9} - none of this code is relevant; it does not change the value of the uri
for (Map.Entry<String, List<Object>> e : headers.entrySet()) {
List<Object> vs = e.getValue();
if (vs.size() == 1) {
request.addHeader(e.getKey(), ClientRequest.getHeaderValue(vs.get(0)));
} else {
StringBuilder b = new StringBuilder();
for (Object v : e.getValue()) {
if (b.length() > 0) {
b.append(',');
}
b.append(ClientRequest.getHeaderValue(v));
}
request.addHeader(e.getKey(), b.toString());
}
}
// {10} - calling request.getURI() here returns not null, and logging produces the expected "https://{serverpath}" on toString()
}
// ... irrelevant code removed
}
Relevant source for interface HttpUriRequest v4.5.12
package org.apache.http.client.methods;
import java.net.URI;
import org.apache.http.HttpRequest;
public interface HttpUriRequest extends HttpRequest {
// ... irrelevant code removed
URI getURI();
// ... irrelevant code removed
}
Relevant source for abstract class HttpRequestBase v4.5.12 (HttpGet, HttpPost, etc. extend this)
package org.apache.http.client.methods;
import java.net.URI;
// ... irrelevant code removed
public abstract class HttpRequestBase extends AbstractExecutionAwareRequest
implements HttpUriRequest, Configurable {
// ... irrelevant code removed
private URI uri;
// ... irrelevant code removed
@Override
public URI getURI() {
return this.uri;
}
// ... irrelevant code removed
public void setURI(final URI uri) {
this.uri = uri;
}
}
Relevant source for class HttpGet v4.5.12 (HttpPost, etc. are implemented similarly)
package org.apache.http.client.methods;
import java.net.URI;
public class HttpGet extends HttpRequestBase {
public final static String METHOD_NAME = "GET";
public HttpGet() {
super();
}
public HttpGet(final URI uri) {
super();
setURI(uri);
}
/**
* @throws IllegalArgumentException if the uri is invalid.
*/
public HttpGet(final String uri) {
super();
setURI(URI.create(uri));
}
@Override
public String getMethod() {
return METHOD_NAME;
}
}
Given the source walkthrough and the supporting comments proving the viability of the state of the Object instance, how is it possible the call to request.getURI() is returning null?
ENVIRONMENT
- This code runs under IBM Liberty on Linux servers (JVM 8.0-5.40).
- This code also runs under Wildfly on Windows servers (JVM 1.7).
- The code is identical on both system types.
- The code never has the above stated problem on the Windows servers.
- The code does not have the above stated problem on Liberty servers where the Liberty management administration console is installed.
OBSERVATIONS
- The first 2-3 pass-throughs of
ApacheHttpClient4Hander.handle(ClientRequest)succeed withoutrequest.getURI()returningnull; it is only on the 4th and subsequent requests that it returnsnull.
INVESTIGATION
Here are some investigation steps I have taken so far:
1 - I created this case on SO for group-mind insight.
2 - I opened a case with IBM.
3 - Customize ApacheHttpClient4Handler
- I add debug logging to all methods via SL4J.
RESULT
- the log output proves the call to
HttpUriRequest.getURI()only starts returningnullafter enteringApacheHttpClient4Handler.getHost(final HttpUriRequest request). - any call immediately before
getHost(HttpUriRequest)showsgetURI()to return not-null. - after adding the debug logging, instead of after the 2-3 and subsequent requests (per OBSERVATIONS),
HttpUriRequest.getURI()now returns null after the 1-2 (and subsequent) requests. I suspect this is due to debug logging proactively callingHttpUriRequest.getURI(), but I cannot confirm this and do not know why the scenario is now shorter.
4 - Customize ApacheHttpClient4Handler further
- change all references of
HttpUriRequesttoHttpRequestBase
RESULT
- doing this did not solve the problem;
getURI()still returnsnull.
5 - Customize HttpRequestBase
- add debug logging to
getURI()/setURI(...)methods
RESULT
- Application started working as designed, except with lots of debug logging!!!
- see section MAJOR UPDATE below.
6 - Customize HttpGet, HttpPut, etc.
- add debug logging to constructors.
RESULT
- see section MAJOR UPDATE below.
7 - Update IBM JVM from 8.0-5.40 to 8.0-6.11
- List of NPE-related bugfixes from the IBM release notes
- IJ17563 | NPE WHEN CALLING JAVA.LANG.STRING.INDEXOF API WITH A LARGE OFFSET
- IJ19669 | NPE OCCURS WHEN READING AN EMPTY PKCS12 KEYSTORE USING JAVA 8
- IJ20528 | NPE OR SEGFAULT FOR METHOD CALL TO OBJECT OF A PRIMITIVE WRAPPER TYPE
- IJ23079 | UNEXPECTED NPE CAUSED BY AN INCORRECT JIT COMPILER OPTIMIZATION
- TODO: results to be posted in update
CONCLUSION
What is going on here?
Is the IBM implementation of the JVM used by Liberty mistreating this code, or is there something else?
Based on the code, and the walkthrough, there is no way HttpUriRequest.getURI() should be returning null.
MAJOR UPDATE
Per Step 5 in INVESTIGATION, I customized httpclient with debug logging using source from the global distribution. After adding the JAR to my project, it started working. I rolled back my changes, rebuilt the JAR from unmodified source, added it to my project, and the application started working.
I redeployed again using the globally-distributed httpclient-4.5.12.jar My application failed.
Again, I did not make any changes to the code; I took the httpclient-4.5.12-sources.jar, compiled it in Eclipse and redeployed with my newly-named JAR, httpclient-4.5.12.joshdm.jar. And it works.
I decompiled both JARs using JAD to examine the code. There were minimal differences, only in labels, which means the compilers were different. I compared MANIFEST.MF for both JARs; the Build-Jdk entries were different. I compiled using Jdk 1.8.0_191. The global one was compiled using Jdk 1.8.0_181.
I have reported this strange discrepancy to IBM, provided both compiled JARs and the source JAR, and am waiting to hear back as to why mine works with their JVM and why the global one does not. I am now 99% certain this is an IBM JVM or server configuration issue, and has nothing to do with the deployed application.
UPDATE #2
Adding the following configuration and rolling back to the global JAR works (also works with the recompiled JAR).
-Xshareclasses:none - disables shared class caching and AOT compilation.
-Xnojit - disables JIT compilation
Disabling shared class caching, disabling AOT compilation and disabling JIT compilation on Liberty seems to have resolved the issue. I will write-up an actual answer once I hear back from IBM why the above settings were needed.
答案1
得分: 0
IBM Liberty 运行在版本 ibm-java-sdk-8.0-5.40 上。
将其更新到 ibm-java-sdk-8.0-6.11 解决了问题。
IBM 回应说，"自从 Java 8 SR5 FP40 以来已经修复了一些 JIT 问题，很可能您在工作环境中没有遇到相同的代码路径。"
英文:
IBM Liberty was running on version ibm-java-sdk-8.0-5.40
Updating it to ibm-java-sdk-8.0-6.11 solved it.
IBM responded, "several JIT issues {...} were fixed since Java 8 SR5 FP40 {...} {i}t is very likely that you did not hit the same code path in the working environment."
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论