太多等待线程在Java 8中的WebSocket客户端中导致Java堆转储。

huangapple
go评论65阅读模式
英文:

too much waiting thread cause java heap dump in websocket client in java 8

问题

今天我的Java应用程序堆转储了，我从服务器上拷贝了转储文件，然后使用VisualVM进行分析，日志看起来像这样：

"WebSocketClient-SecureIO-1" 守护线程优先级=5 tid=888 等待
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    ...

看起来似乎有太多的线程一直在等待，我的内存配置为500MB，但我不知道为什么会发生这种情况。这是我的WebSocket连接代码：

public WebsocketClientEndpoint robotNewConnect(Long roomTypeId, String token, String userMark) {
    WebsocketClientEndpoint clientEndPoint = null;
    String websocketConnUrl = websocketUrl + "?token=" + token + "&roomTypeId=" + roomTypeId + "&robotFlag=1";
    try {
        String appMark = SessionUtil.getThreadLocal("appMark");
        clientEndPoint = new WebsocketClientEndpoint(new URI(websocketConnUrl));
        ...
    } catch (Exception e) {
        log.error("Websocket", e);
    }
    return clientEndPoint;
}

我已经尝试了一些方法，但问题仍然没有解决。这个问题可能是由什么引起的，我应该怎么做才能解决？

我已经尝试了以下方法：

  1. 我查看了tomcat-embed-websocket-9.0.30源代码，其中包含WebsocketClientEndpoint类。连接成功，但卡在了这行代码上：
WsFrameClient wsFrameClient = new WsFrameClient(response, channel, wsSession, transformation);

我进入了这个类并发现代码卡在了死锁代码处：

private void doResumeProcessing(boolean checkOpenOnError) {
    while (true) {
        switch (getReadState()) {
            case PROCESSING:
                if (!changeReadState(ReadState.PROCESSING, ReadState.WAITING)) {
                    continue;
                }
                resumeProcessing(checkOpenOnError);
                return;
            case SUSPENDING_PROCESS:
                if (!changeReadState(ReadState.SUSPENDING_PROCESS, ReadState.SUSPENDED)) {
                    continue;
                }
                return;
            default:
                throw new IllegalStateException(sm.getString("wsFrame.illegalReadState", getReadState()));
        }
    }
}

读取状态getReadState始终是PROCESSING，代码陷入了无限循环，这就是为什么在转储文件中有这么多等待线程的原因。但现在我不知道为什么读取状态是PROCESSING，以及如何解决它。有人可以帮助我吗？

英文:

Today my java application heap dump, and I copy the dump file from server analysis using visualVM, the log look like this:

"WebSocketClient-SecureIO-1" daemon prio=5 tid=888 WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
       local variable: java.util.concurrent.locks.AbstractQueuedSynchronizer$Node#184
       local variable: java.util.concurrent.locks.AbstractQueuedSynchronizer$Node#185
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
       local variable: java.util.concurrent.CountDownLatch$Sync#36
    at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
       local variable: java.util.concurrent.CountDownLatch#35
    at sun.nio.ch.PendingFuture.get(PendingFuture.java:180)
    at org.apache.tomcat.websocket.AsyncChannelWrapperSecure$ReadTask.run(AsyncChannelWrapperSecure.java:269)
       local variable: sun.nio.ch.PendingFuture#47
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
       local variable: java.util.concurrent.ThreadPoolExecutor#1
       local variable: org.apache.tomcat.websocket.AsyncChannelWrapperSecure$ReadTask#6
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
       local variable: java.util.concurrent.ThreadPoolExecutor$Worker#1
    at java.lang.Thread.run(Thread.java:748)

"WebSocketClient-SecureIO-2" daemon prio=5 tid=889 WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
       local variable: java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#5
       local variable: java.util.concurrent.locks.AbstractQueuedSynchronizer$Node#114
    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
       local variable: java.util.concurrent.LinkedBlockingQueue#1
       local variable: java.util.concurrent.atomic.AtomicInteger#56
       local variable: java.util.concurrent.locks.ReentrantLock#9
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
       local variable: java.util.concurrent.ThreadPoolExecutor$Worker#2
    at java.lang.Thread.run(Thread.java:748)

"pool-87-thread-1" prio=5 tid=890 TIMED_WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
       local variable: java.util.concurrent.locks.AbstractQueuedSynchronizer$Node#183
       local variable: java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#558
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
       local variable: java.util.concurrent.locks.ReentrantLock#3654
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
       local variable: java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue#1
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
       local variable: java.util.concurrent.ScheduledThreadPoolExecutor#129
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
       local variable: java.util.concurrent.ThreadPoolExecutor$Worker#3
    at java.lang.Thread.run(Thread.java:748)

seems too many thread(maybe thousands) is waithing all the time, my memory now config to 500MB, and I now I have no idea why would this happen.This my websocket connection code:

 public WebsocketClientEndpoint robotNewConnect(Long roomTypeId, String token, String userMark) {
        WebsocketClientEndpoint clientEndPoint = null;
        String websocketConnUrl = websocketUrl + "?token=" + token + "&roomTypeId=" + roomTypeId + "&robotFlag=1";
        try {
            String appMark = SessionUtil.getThreadLocal("appMark");
           
            clientEndPoint = new WebsocketClientEndpoint(new URI(websocketConnUrl));
            clientEndPoint.userSession.getUserProperties().put("userIdentity", userMark + "-" + appMark + "-" + roomTypeId);
            clientEndPoint.addMessageHandler(message -> {
                log.info("addMessageHandler:", message);
            });
        } catch (Exception e) {
            log.error("Websocket", e);
        }
        return clientEndPoint;
    }

I am searhing from internet and try to incrase my memory but problem still not resolve. what may cause this problem and what should I do to fix this?

what I have tried:

  1. I follow to the souce code of tomcat-embed-websocket-9.0.30 where class WebsocketClientEndpoint belong. the connect was success, but stuck on this line code:

WsFrameClient wsFrameClient = new WsFrameClient(response, channel,
wsSession, transformation);

and I step into the class and find the code was stuck in the dead lock code:

private void doResumeProcessing(boolean checkOpenOnError) {
            while (true) {
                switch (getReadState()) {
                case PROCESSING:
                    if (!changeReadState(ReadState.PROCESSING, ReadState.WAITING)) {
                        continue;
                    }
                    resumeProcessing(checkOpenOnError);
                    return;
                case SUSPENDING_PROCESS:
                    if (!changeReadState(ReadState.SUSPENDING_PROCESS, ReadState.SUSPENDED)) {
                        continue;
                    }
                    return;
                default:
                    throw new IllegalStateException(
                            sm.getString("wsFrame.illegalReadState", getReadState()));
                }
            }
        }

the read state getReadState is always PROCESSING, and the code loop forever, this is why so much waiting thread in dump file.

But now I do not know why the read state is PROCESSING and how to solve it? any one could help me?

答案1

得分: 1

我曾经在使用WebSocket时遇到类似的问题，当我没有正确关闭中断的WebSocket连接并在重试时创建新的连接时。我采取了以下措施来解决这些问题：

  1. 确保WebSocketClient对象在由于错误而关闭连接时被垃圾回收。我正在使用Spring，因此我将WebSocketClient注册为具有原型范围的bean。这样当连接关闭时，Spring会进行清理。
@Bean
@Scope("prototype")
public StandardWebSocketClient webSocketClient() throws Exception {
    StandardWebSocketClient standardWebSocketClient = new StandardWebSocketClient(clientContainer());
    standardWebSocketClient.setTaskExecutor(webSocketTaskExecutor());
    return standardWebSocketClient;
}

@Bean
public AsyncListenableTaskExecutor webSocketTaskExecutor() {
    SimpleAsyncTaskExecutor executor = new SimpleAsyncTaskExecutor("webSocketTaskExecutor-");
    executor.setConcurrencyLimit(20);
    return executor;
}
  1. 在注解中声明了具有destroyMethod的ClientContainer bean。
@Bean(destroyMethod = "doStop")
public ClientContainer clientContainer() throws Exception {
    WebSocketPolicy webSocketPolicy = WebSocketPolicy.newClientPolicy();
    webSocketPolicy.setMaxTextMessageSize(1024000); // 1MB
    ClientContainer clientContainer = new ClientContainer(new SimpleContainerScope(webSocketPolicy));
    clientContainer.start();
    return clientContainer;
}

FYI，我当时在使用spring-boot-starter-websocket和jetty-server。

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-websocket</artifactId>
</dependency>

这些更改显著减少了线程数量。

英文:

I had faced similar problems with WebSocket when I was not properly closing the interrupted WebSocket connections and creating new one while retrying.

Steps I took to fix them:

  1. Ensure that the WebSocketClient objects are garbage collected when connection is closed due to error. I was using Spring so I registered WebSocketClient as bean with prototype scope. So that when connection is closed Spring will do the cleanup.
    @Bean
    @Scope(&quot;prototype&quot;)
    public StandardWebSocketClient webSocketClient() throws Exception {
        StandardWebSocketClient standardWebSocketClient = new StandardWebSocketClient(clientContainer());
        standardWebSocketClient.setTaskExecutor(webSocketTaskExecutor());
        return standardWebSocketClient;
    }

    @Bean
    public AsyncListenableTaskExecutor webSocketTaskExecutor() {
        SimpleAsyncTaskExecutor executor = new SimpleAsyncTaskExecutor(&quot;webSocketTaskExecutor-&quot;);
        executor.setConcurrencyLimit(20);
        return executor;
    }
  1. Declared ClientContainer bean with destroyMethod in annotation.
    @Bean(destroyMethod = &quot;doStop&quot;)
    public ClientContainer clientContainer() throws Exception {
        WebSocketPolicy webSocketPolicy = WebSocketPolicy.newClientPolicy();
        webSocketPolicy.setMaxTextMessageSize(1024000); //1MB
        ClientContainer clientContainer = new ClientContainer(new SimpleContainerScope(webSocketPolicy));
        clientContainer.start();
        return clientContainer;
    }

FYI I was using spring-boot-starter-websocket with jetty-server.

        &lt;dependency&gt;
            &lt;groupId&gt;org.springframework.boot&lt;/groupId&gt;
            &lt;artifactId&gt;spring-boot-starter-websocket&lt;/artifactId&gt;
        &lt;/dependency&gt;

These changes reduced the number of threads drastically.

huangapple
  • 本文由 发表于 2020年8月12日 19:32:42
  • 转载请务必保留本文链接：https://go.coder-hub.com/63375622.html
  • java
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定