太多等待线程在Java 8中的WebSocket客户端中导致Java堆转储。

huangapple
go评论90阅读模式
英文:

too much waiting thread cause java heap dump in websocket client in java 8

问题

今天我的Java应用程序堆转储了，我从服务器上拷贝了转储文件，然后使用VisualVM进行分析，日志看起来像这样：

  1. "WebSocketClient-SecureIO-1" 守护线程优先级=5 tid=888 等待
  2.     at sun.misc.Unsafe.park(Native Method)
  3.     at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
  4.     ...

看起来似乎有太多的线程一直在等待，我的内存配置为500MB，但我不知道为什么会发生这种情况。这是我的WebSocket连接代码：

  1. public WebsocketClientEndpoint robotNewConnect(Long roomTypeId, String token, String userMark) {
  2.     WebsocketClientEndpoint clientEndPoint = null;
  3.     String websocketConnUrl = websocketUrl + "?token=" + token + "&roomTypeId=" + roomTypeId + "&robotFlag=1";
  4.     try {
  5.         String appMark = SessionUtil.getThreadLocal("appMark");
  6.         clientEndPoint = new WebsocketClientEndpoint(new URI(websocketConnUrl));
  7.         ...
  8.     } catch (Exception e) {
  9.         log.error("Websocket", e);
  10.     }
  11.     return clientEndPoint;
  12. }

我已经尝试了一些方法，但问题仍然没有解决。这个问题可能是由什么引起的，我应该怎么做才能解决？

我已经尝试了以下方法：

  1. 我查看了tomcat-embed-websocket-9.0.30源代码，其中包含WebsocketClientEndpoint类。连接成功，但卡在了这行代码上：
  1. WsFrameClient wsFrameClient = new WsFrameClient(response, channel, wsSession, transformation);

我进入了这个类并发现代码卡在了死锁代码处：

  1. private void doResumeProcessing(boolean checkOpenOnError) {
  2.     while (true) {
  3.         switch (getReadState()) {
  4.             case PROCESSING:
  5.                 if (!changeReadState(ReadState.PROCESSING, ReadState.WAITING)) {
  6.                     continue;
  7.                 }
  8.                 resumeProcessing(checkOpenOnError);
  9.                 return;
  10.             case SUSPENDING_PROCESS:
  11.                 if (!changeReadState(ReadState.SUSPENDING_PROCESS, ReadState.SUSPENDED)) {
  12.                     continue;
  13.                 }
  14.                 return;
  15.             default:
  16.                 throw new IllegalStateException(sm.getString("wsFrame.illegalReadState", getReadState()));
  17.         }
  18.     }
  19. }

读取状态getReadState始终是PROCESSING，代码陷入了无限循环，这就是为什么在转储文件中有这么多等待线程的原因。但现在我不知道为什么读取状态是PROCESSING，以及如何解决它。有人可以帮助我吗？

英文:

Today my java application heap dump, and I copy the dump file from server analysis using visualVM, the log look like this:

  1. "WebSocketClient-SecureIO-1" daemon prio=5 tid=888 WAITING
  2.     at sun.misc.Unsafe.park(Native Method)
  3.     at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
  4.     at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
  5.     at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
  6.        local variable: java.util.concurrent.locks.AbstractQueuedSynchronizer$Node#184
  7.        local variable: java.util.concurrent.locks.AbstractQueuedSynchronizer$Node#185
  8.     at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
  9.        local variable: java.util.concurrent.CountDownLatch$Sync#36
  10.     at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
  11.        local variable: java.util.concurrent.CountDownLatch#35
  12.     at sun.nio.ch.PendingFuture.get(PendingFuture.java:180)
  13.     at org.apache.tomcat.websocket.AsyncChannelWrapperSecure$ReadTask.run(AsyncChannelWrapperSecure.java:269)
  14.        local variable: sun.nio.ch.PendingFuture#47
  15.     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  16.        local variable: java.util.concurrent.ThreadPoolExecutor#1
  17.        local variable: org.apache.tomcat.websocket.AsyncChannelWrapperSecure$ReadTask#6
  18.     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  19.        local variable: java.util.concurrent.ThreadPoolExecutor$Worker#1
  20.     at java.lang.Thread.run(Thread.java:748)
  22. "WebSocketClient-SecureIO-2" daemon prio=5 tid=889 WAITING
  23.     at sun.misc.Unsafe.park(Native Method)
  24.     at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
  25.     at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
  26.        local variable: java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#5
  27.        local variable: java.util.concurrent.locks.AbstractQueuedSynchronizer$Node#114
  28.     at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  29.        local variable: java.util.concurrent.LinkedBlockingQueue#1
  30.        local variable: java.util.concurrent.atomic.AtomicInteger#56
  31.        local variable: java.util.concurrent.locks.ReentrantLock#9
  32.     at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
  33.     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
  34.     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  35.        local variable: java.util.concurrent.ThreadPoolExecutor$Worker#2
  36.     at java.lang.Thread.run(Thread.java:748)
  38. "pool-87-thread-1" prio=5 tid=890 TIMED_WAITING
  39.     at sun.misc.Unsafe.park(Native Method)
  40.     at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
  41.     at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
  42.        local variable: java.util.concurrent.locks.AbstractQueuedSynchronizer$Node#183
  43.        local variable: java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#558
  44.     at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
  45.        local variable: java.util.concurrent.locks.ReentrantLock#3654
  46.     at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
  47.        local variable: java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue#1
  48.     at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
  49.     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
  50.        local variable: java.util.concurrent.ScheduledThreadPoolExecutor#129
  51.     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  52.        local variable: java.util.concurrent.ThreadPoolExecutor$Worker#3
  53.     at java.lang.Thread.run(Thread.java:748)

seems too many thread(maybe thousands) is waithing all the time, my memory now config to 500MB, and I now I have no idea why would this happen.This my websocket connection code:

  1.  public WebsocketClientEndpoint robotNewConnect(Long roomTypeId, String token, String userMark) {
  2.         WebsocketClientEndpoint clientEndPoint = null;
  3.         String websocketConnUrl = websocketUrl + "?token=" + token + "&roomTypeId=" + roomTypeId + "&robotFlag=1";
  4.         try {
  5.             String appMark = SessionUtil.getThreadLocal("appMark");
  6.            
  7.             clientEndPoint = new WebsocketClientEndpoint(new URI(websocketConnUrl));
  8.             clientEndPoint.userSession.getUserProperties().put("userIdentity", userMark + "-" + appMark + "-" + roomTypeId);
  9.             clientEndPoint.addMessageHandler(message -> {
  10.                 log.info("addMessageHandler:", message);
  11.             });
  12.         } catch (Exception e) {
  13.             log.error("Websocket", e);
  14.         }
  15.         return clientEndPoint;
  16.     }

I am searhing from internet and try to incrase my memory but problem still not resolve. what may cause this problem and what should I do to fix this?

what I have tried:

  1. I follow to the souce code of tomcat-embed-websocket-9.0.30 where class WebsocketClientEndpoint belong. the connect was success, but stuck on this line code:

WsFrameClient wsFrameClient = new WsFrameClient(response, channel,
wsSession, transformation);

and I step into the class and find the code was stuck in the dead lock code:

  1. private void doResumeProcessing(boolean checkOpenOnError) {
  2.             while (true) {
  3.                 switch (getReadState()) {
  4.                 case PROCESSING:
  5.                     if (!changeReadState(ReadState.PROCESSING, ReadState.WAITING)) {
  6.                         continue;
  7.                     }
  8.                     resumeProcessing(checkOpenOnError);
  9.                     return;
  10.                 case SUSPENDING_PROCESS:
  11.                     if (!changeReadState(ReadState.SUSPENDING_PROCESS, ReadState.SUSPENDED)) {
  12.                         continue;
  13.                     }
  14.                     return;
  15.                 default:
  16.                     throw new IllegalStateException(
  17.                             sm.getString("wsFrame.illegalReadState", getReadState()));
  18.                 }
  19.             }
  20.         }

the read state getReadState is always PROCESSING, and the code loop forever, this is why so much waiting thread in dump file.

But now I do not know why the read state is PROCESSING and how to solve it? any one could help me?

答案1

得分: 1

我曾经在使用WebSocket时遇到类似的问题，当我没有正确关闭中断的WebSocket连接并在重试时创建新的连接时。我采取了以下措施来解决这些问题：

  1. 确保WebSocketClient对象在由于错误而关闭连接时被垃圾回收。我正在使用Spring，因此我将WebSocketClient注册为具有原型范围的bean。这样当连接关闭时，Spring会进行清理。
  1. @Bean
  2. @Scope("prototype")
  3. public StandardWebSocketClient webSocketClient() throws Exception {
  4.     StandardWebSocketClient standardWebSocketClient = new StandardWebSocketClient(clientContainer());
  5.     standardWebSocketClient.setTaskExecutor(webSocketTaskExecutor());
  6.     return standardWebSocketClient;
  7. }
  9. @Bean
  10. public AsyncListenableTaskExecutor webSocketTaskExecutor() {
  11.     SimpleAsyncTaskExecutor executor = new SimpleAsyncTaskExecutor("webSocketTaskExecutor-");
  12.     executor.setConcurrencyLimit(20);
  13.     return executor;
  14. }
  1. 在注解中声明了具有destroyMethod的ClientContainer bean。
  1. @Bean(destroyMethod = "doStop")
  2. public ClientContainer clientContainer() throws Exception {
  3.     WebSocketPolicy webSocketPolicy = WebSocketPolicy.newClientPolicy();
  4.     webSocketPolicy.setMaxTextMessageSize(1024000); // 1MB
  5.     ClientContainer clientContainer = new ClientContainer(new SimpleContainerScope(webSocketPolicy));
  6.     clientContainer.start();
  7.     return clientContainer;
  8. }

FYI，我当时在使用spring-boot-starter-websocket和jetty-server。

  1. <dependency>
  2.     <groupId>org.springframework.boot</groupId>
  3.     <artifactId>spring-boot-starter-websocket</artifactId>
  4. </dependency>

这些更改显著减少了线程数量。

英文:

I had faced similar problems with WebSocket when I was not properly closing the interrupted WebSocket connections and creating new one while retrying.

Steps I took to fix them:

  1. Ensure that the WebSocketClient objects are garbage collected when connection is closed due to error. I was using Spring so I registered WebSocketClient as bean with prototype scope. So that when connection is closed Spring will do the cleanup.
  1.     @Bean
  2.     @Scope(&quot;prototype&quot;)
  3.     public StandardWebSocketClient webSocketClient() throws Exception {
  4.         StandardWebSocketClient standardWebSocketClient = new StandardWebSocketClient(clientContainer());
  5.         standardWebSocketClient.setTaskExecutor(webSocketTaskExecutor());
  6.         return standardWebSocketClient;
  7.     }
  9.     @Bean
  10.     public AsyncListenableTaskExecutor webSocketTaskExecutor() {
  11.         SimpleAsyncTaskExecutor executor = new SimpleAsyncTaskExecutor(&quot;webSocketTaskExecutor-&quot;);
  12.         executor.setConcurrencyLimit(20);
  13.         return executor;
  14.     }
  1. Declared ClientContainer bean with destroyMethod in annotation.
  1.     @Bean(destroyMethod = &quot;doStop&quot;)
  2.     public ClientContainer clientContainer() throws Exception {
  3.         WebSocketPolicy webSocketPolicy = WebSocketPolicy.newClientPolicy();
  4.         webSocketPolicy.setMaxTextMessageSize(1024000); //1MB
  5.         ClientContainer clientContainer = new ClientContainer(new SimpleContainerScope(webSocketPolicy));
  6.         clientContainer.start();
  7.         return clientContainer;
  8.     }

FYI I was using spring-boot-starter-websocket with jetty-server.

  1.         &lt;dependency&gt;
  2.             &lt;groupId&gt;org.springframework.boot&lt;/groupId&gt;
  3.             &lt;artifactId&gt;spring-boot-starter-websocket&lt;/artifactId&gt;
  4.         &lt;/dependency&gt;

These changes reduced the number of threads drastically.

huangapple
  • 本文由 发表于 2020年8月12日 19:32:42
  • 转载请务必保留本文链接：https://go.coder-hub.com/63375622.html
  • java
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定