英文:
too much waiting thread cause java heap dump in websocket client in java 8
问题
今天我的Java应用程序堆转储了,我从服务器上拷贝了转储文件,然后使用VisualVM进行分析,日志看起来像这样:
"WebSocketClient-SecureIO-1" 守护线程优先级=5 tid=888 等待
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
...
看起来似乎有太多的线程一直在等待,我的内存配置为500MB,但我不知道为什么会发生这种情况。这是我的WebSocket连接代码:
public WebsocketClientEndpoint robotNewConnect(Long roomTypeId, String token, String userMark) {
WebsocketClientEndpoint clientEndPoint = null;
String websocketConnUrl = websocketUrl + "?token=" + token + "&roomTypeId=" + roomTypeId + "&robotFlag=1";
try {
String appMark = SessionUtil.getThreadLocal("appMark");
clientEndPoint = new WebsocketClientEndpoint(new URI(websocketConnUrl));
...
} catch (Exception e) {
log.error("Websocket", e);
}
return clientEndPoint;
}
我已经尝试了一些方法,但问题仍然没有解决。这个问题可能是由什么引起的,我应该怎么做才能解决?
我已经尝试了以下方法:
- 我查看了
tomcat-embed-websocket-9.0.30
源代码,其中包含WebsocketClientEndpoint
类。连接成功,但卡在了这行代码上:
WsFrameClient wsFrameClient = new WsFrameClient(response, channel, wsSession, transformation);
我进入了这个类并发现代码卡在了死锁代码处:
private void doResumeProcessing(boolean checkOpenOnError) {
while (true) {
switch (getReadState()) {
case PROCESSING:
if (!changeReadState(ReadState.PROCESSING, ReadState.WAITING)) {
continue;
}
resumeProcessing(checkOpenOnError);
return;
case SUSPENDING_PROCESS:
if (!changeReadState(ReadState.SUSPENDING_PROCESS, ReadState.SUSPENDED)) {
continue;
}
return;
default:
throw new IllegalStateException(sm.getString("wsFrame.illegalReadState", getReadState()));
}
}
}
读取状态getReadState
始终是PROCESSING
,代码陷入了无限循环,这就是为什么在转储文件中有这么多等待线程的原因。但现在我不知道为什么读取状态是PROCESSING
,以及如何解决它。有人可以帮助我吗?
英文:
Today my java application heap dump, and I copy the dump file from server analysis using visualVM, the log look like this:
"WebSocketClient-SecureIO-1" daemon prio=5 tid=888 WAITING
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
local variable: java.util.concurrent.locks.AbstractQueuedSynchronizer$Node#184
local variable: java.util.concurrent.locks.AbstractQueuedSynchronizer$Node#185
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
local variable: java.util.concurrent.CountDownLatch$Sync#36
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
local variable: java.util.concurrent.CountDownLatch#35
at sun.nio.ch.PendingFuture.get(PendingFuture.java:180)
at org.apache.tomcat.websocket.AsyncChannelWrapperSecure$ReadTask.run(AsyncChannelWrapperSecure.java:269)
local variable: sun.nio.ch.PendingFuture#47
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
local variable: java.util.concurrent.ThreadPoolExecutor#1
local variable: org.apache.tomcat.websocket.AsyncChannelWrapperSecure$ReadTask#6
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
local variable: java.util.concurrent.ThreadPoolExecutor$Worker#1
at java.lang.Thread.run(Thread.java:748)
"WebSocketClient-SecureIO-2" daemon prio=5 tid=889 WAITING
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
local variable: java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#5
local variable: java.util.concurrent.locks.AbstractQueuedSynchronizer$Node#114
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
local variable: java.util.concurrent.LinkedBlockingQueue#1
local variable: java.util.concurrent.atomic.AtomicInteger#56
local variable: java.util.concurrent.locks.ReentrantLock#9
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
local variable: java.util.concurrent.ThreadPoolExecutor$Worker#2
at java.lang.Thread.run(Thread.java:748)
"pool-87-thread-1" prio=5 tid=890 TIMED_WAITING
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
local variable: java.util.concurrent.locks.AbstractQueuedSynchronizer$Node#183
local variable: java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#558
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
local variable: java.util.concurrent.locks.ReentrantLock#3654
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
local variable: java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue#1
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
local variable: java.util.concurrent.ScheduledThreadPoolExecutor#129
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
local variable: java.util.concurrent.ThreadPoolExecutor$Worker#3
at java.lang.Thread.run(Thread.java:748)
seems too many thread(maybe thousands) is waithing all the time, my memory now config to 500MB, and I now I have no idea why would this happen.This my websocket connection code:
public WebsocketClientEndpoint robotNewConnect(Long roomTypeId, String token, String userMark) {
WebsocketClientEndpoint clientEndPoint = null;
String websocketConnUrl = websocketUrl + "?token=" + token + "&roomTypeId=" + roomTypeId + "&robotFlag=1";
try {
String appMark = SessionUtil.getThreadLocal("appMark");
clientEndPoint = new WebsocketClientEndpoint(new URI(websocketConnUrl));
clientEndPoint.userSession.getUserProperties().put("userIdentity", userMark + "-" + appMark + "-" + roomTypeId);
clientEndPoint.addMessageHandler(message -> {
log.info("addMessageHandler:", message);
});
} catch (Exception e) {
log.error("Websocket", e);
}
return clientEndPoint;
}
I am searhing from internet and try to incrase my memory but problem still not resolve. what may cause this problem and what should I do to fix this?
what I have tried:
- I follow to the souce code of
tomcat-embed-websocket-9.0.30
where classWebsocketClientEndpoint
belong. the connect was success, but stuck on this line code:
WsFrameClient wsFrameClient = new WsFrameClient(response, channel,
wsSession, transformation);
and I step into the class and find the code was stuck in the dead lock code:
private void doResumeProcessing(boolean checkOpenOnError) {
while (true) {
switch (getReadState()) {
case PROCESSING:
if (!changeReadState(ReadState.PROCESSING, ReadState.WAITING)) {
continue;
}
resumeProcessing(checkOpenOnError);
return;
case SUSPENDING_PROCESS:
if (!changeReadState(ReadState.SUSPENDING_PROCESS, ReadState.SUSPENDED)) {
continue;
}
return;
default:
throw new IllegalStateException(
sm.getString("wsFrame.illegalReadState", getReadState()));
}
}
}
the read state getReadState
is always PROCESSING
, and the code loop forever, this is why so much waiting thread in dump file.
But now I do not know why the read state is PROCESSING
and how to solve it? any one could help me?
答案1
得分: 1
我曾经在使用WebSocket时遇到类似的问题,当我没有正确关闭中断的WebSocket连接并在重试时创建新的连接时。我采取了以下措施来解决这些问题:
- 确保WebSocketClient对象在由于错误而关闭连接时被垃圾回收。我正在使用Spring,因此我将WebSocketClient注册为具有原型范围的bean。这样当连接关闭时,Spring会进行清理。
@Bean
@Scope("prototype")
public StandardWebSocketClient webSocketClient() throws Exception {
StandardWebSocketClient standardWebSocketClient = new StandardWebSocketClient(clientContainer());
standardWebSocketClient.setTaskExecutor(webSocketTaskExecutor());
return standardWebSocketClient;
}
@Bean
public AsyncListenableTaskExecutor webSocketTaskExecutor() {
SimpleAsyncTaskExecutor executor = new SimpleAsyncTaskExecutor("webSocketTaskExecutor-");
executor.setConcurrencyLimit(20);
return executor;
}
- 在注解中声明了具有destroyMethod的ClientContainer bean。
@Bean(destroyMethod = "doStop")
public ClientContainer clientContainer() throws Exception {
WebSocketPolicy webSocketPolicy = WebSocketPolicy.newClientPolicy();
webSocketPolicy.setMaxTextMessageSize(1024000); // 1MB
ClientContainer clientContainer = new ClientContainer(new SimpleContainerScope(webSocketPolicy));
clientContainer.start();
return clientContainer;
}
FYI,我当时在使用spring-boot-starter-websocket和jetty-server。
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-websocket</artifactId>
</dependency>
这些更改显著减少了线程数量。
英文:
I had faced similar problems with WebSocket
when I was not properly closing the interrupted WebSocket
connections and creating new one while retrying.
Steps I took to fix them:
- Ensure that the
WebSocketClient
objects are garbage collected when connection is closed due to error. I was usingSpring
so I registeredWebSocketClient
as bean withprototype
scope. So that when connection is closedSpring
will do the cleanup.
@Bean
@Scope("prototype")
public StandardWebSocketClient webSocketClient() throws Exception {
StandardWebSocketClient standardWebSocketClient = new StandardWebSocketClient(clientContainer());
standardWebSocketClient.setTaskExecutor(webSocketTaskExecutor());
return standardWebSocketClient;
}
@Bean
public AsyncListenableTaskExecutor webSocketTaskExecutor() {
SimpleAsyncTaskExecutor executor = new SimpleAsyncTaskExecutor("webSocketTaskExecutor-");
executor.setConcurrencyLimit(20);
return executor;
}
- Declared
ClientContainer
bean withdestroyMethod
in annotation.
@Bean(destroyMethod = "doStop")
public ClientContainer clientContainer() throws Exception {
WebSocketPolicy webSocketPolicy = WebSocketPolicy.newClientPolicy();
webSocketPolicy.setMaxTextMessageSize(1024000); //1MB
ClientContainer clientContainer = new ClientContainer(new SimpleContainerScope(webSocketPolicy));
clientContainer.start();
return clientContainer;
}
FYI I was using spring-boot-starter-websocket
with jetty-server
.
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-websocket</artifactId>
</dependency>
These changes reduced the number of threads drastically.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论