问题

I encountered the following problem when I used the big data tool to test spark running in a k8s cluster：

java.lang.IllegalArgumentException: Too large frame: 5135603447296520289
        at org.sparkproject.guava.base.Preconditions.checkArgument(Preconditions.java:119)
        at org.apache.spark.network.util.TransportFrameDecoder.decodeNext(TransportFrameDecoder.java:148)
        at org.apache.spark.network.util.TransportFrameDecoder.channelRead(TransportFrameDecoder.java:98)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:833)```

I exposed spark 7077 using nodeport, which can be connected externally using telnet. The log displays that spark runs normally. I used the same nodeport way to expose sparkUI to be accessible normally
[enter image description here](https://i.stack.imgur.com/Oz5Nb.png)

The following is the error message for the big data tool：

```com.jetbrains.bigdatatools.common.connection.exception.BdtRestExecutionException: Error during GET: /api/v1/version request.
NoHttpResponseException: 172.31.88.129:30210 failed to respond
	at com.jetbrains.bigdatatools.common.connection.BdtRestExceptionHelper.wrapRest(BdtRestExceptionHelper.kt:27)
	at com.jetbrains.spark.monitoring.connection.SparkMonitoringRestClient.getVersion(SparkMonitoringRestClient.kt:127)
	at com.jetbrains.spark.monitoring.connection.SparkMonitoringRestClient.checkConnectionInner(SparkMonitoringRestClient.kt:20)
	at com.jetbrains.spark.monitoring.connection.SparkMonitoringRestClient.connectInner(SparkMonitoringRestClient.kt:30)
	at com.jetbrains.bigdatatools.common.monitoring.connection.MonitoringClient$connect$1.invoke(MonitoringClient.kt:32)
	at com.jetbrains.bigdatatools.common.monitoring.connection.MonitoringClient$connect$1.invoke(MonitoringClient.kt:31)
	at com.jetbrains.bigdatatools.common.monitoring.connection.MonitoringClient.connectWrapper(MonitoringClient.kt:77)
	at com.jetbrains.bigdatatools.common.monitoring.connection.MonitoringClient.connect(MonitoringClient.kt:31)
	at com.jetbrains.bigdatatools.common.monitoring.connection.MonitoringClient.connectWithThrowable(MonitoringClient.kt:36)
	at com.jetbrains.spark.monitoring.connection.SparkMonitoringConnectionChecker.checkConnection(SparkMonitoringConnectionChecker.kt:21)
	at com.jetbrains.spark.monitoring.settings.SparkConnectionTestingBase.checkConnection(SparkConnectionTestingBase.kt:21)
	at com.jetbrains.spark.monitoring.settings.SparkConnectionTestingBase.checkConnection(SparkConnectionTestingBase.kt:13)
	at com.jetbrains.bigdatatools.common.rfs.settings.RfsConnectionTestingBase.testConnection(RfsConnectionTestingBase.kt:22)
	at com.jetbrains.bigdatatools.common.settings.connections.ConnectionTesting.testConnection(ConnectionData.kt:119)
	at com.jetbrains.bigdatatools.common.settings.defaultui.TestConnectionPanelWrapper$runTesting$1.invoke(TestConnectionPanelWrapper.kt:105)
	at com.jetbrains.bigdatatools.common.settings.defaultui.TestConnectionPanelWrapper$runTesting$1.invoke(TestConnectionPanelWrapper.kt:84)
	at com.jetbrains.bigdatatools.common.util.ThreadUtilsKt.executeOnPooledThread$lambda$3(ThreadUtils.kt:31)
	at com.intellij.openapi.application.impl.ApplicationImpl$1.run(ApplicationImpl.java:252)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:702)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:699)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1.run(Executors.java:699)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: com.jetbrains.bigdatatools.common.connection.exception.impl.RestExecutionException: Error during GET: /api/v1/version request.```

so how can i solve this problem? Thanks

i tried to use other port to connect to the spark master but failed.

<details>
<summary>英文:</summary>

I encountered the following problem when I used the big data tool to test spark running in a k8s cluster：

23/05/10 11:38:13 WARN TransportChannelHandler: Exception in connection from /172.31.88.129:32691
java.lang.IllegalArgumentException: Too large frame: 5135603447296520289
at org.sparkproject.guava.base.Preconditions.checkArgument(Preconditions.java:119)
at org.apache.spark.network.util.TransportFrameDecoder.decodeNext(TransportFrameDecoder.java:148)
at org.apache.spark.network.util.TransportFrameDecoder.channelRead(TransportFrameDecoder.java:98)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:833)

I exposed spark 7077 using nodeport, which can be connected externally using telnet. The log displays that spark runs normally. I used the same nodeport way to expose sparkUI to be accessible normally
[enter image description here](https://i.stack.imgur.com/Oz5Nb.png)
The following is the error message for the big data tool：

com.jetbrains.bigdatatools.common.connection.exception.BdtRestExecutionException: Error during 'GET: /api/v1/version' request.
NoHttpResponseException: 172.31.88.129:30210 failed to respond
at com.jetbrains.bigdatatools.common.connection.BdtRestExceptionHelper.wrapRest(BdtRestExceptionHelper.kt:27)
at com.jetbrains.spark.monitoring.connection.SparkMonitoringRestClient.getVersion(SparkMonitoringRestClient.kt:127)
at com.jetbrains.spark.monitoring.connection.SparkMonitoringRestClient.checkConnectionInner(SparkMonitoringRestClient.kt:20)
at com.jetbrains.spark.monitoring.connection.SparkMonitoringRestClient.connectInner(SparkMonitoringRestClient.kt:30)
at com.jetbrains.bigdatatools.common.monitoring.connection.MonitoringClient$connect$1.invoke(MonitoringClient.kt:32)
at com.jetbrains.bigdatatools.common.monitoring.connection.MonitoringClient$connect$1.invoke(MonitoringClient.kt:31)
at com.jetbrains.bigdatatools.common.monitoring.connection.MonitoringClient.connectWrapper(MonitoringClient.kt:77)
at com.jetbrains.bigdatatools.common.monitoring.connection.MonitoringClient.connect(MonitoringClient.kt:31)
at com.jetbrains.bigdatatools.common.monitoring.connection.MonitoringClient.connectWithThrowable(MonitoringClient.kt:36)
at com.jetbrains.spark.monitoring.connection.SparkMonitoringConnectionChecker.checkConnection(SparkMonitoringConnectionChecker.kt:21)
at com.jetbrains.spark.monitoring.settings.SparkConnectionTestingBase.checkConnection(SparkConnectionTestingBase.kt:21)
at com.jetbrains.spark.monitoring.settings.SparkConnectionTestingBase.checkConnection(SparkConnectionTestingBase.kt:13)
at com.jetbrains.bigdatatools.common.rfs.settings.RfsConnectionTestingBase.testConnection(RfsConnectionTestingBase.kt:22)
at com.jetbrains.bigdatatools.common.settings.connections.ConnectionTesting.testConnection(ConnectionData.kt:119)
at com.jetbrains.bigdatatools.common.settings.defaultui.TestConnectionPanelWrapper$runTesting$1.invoke(TestConnectionPanelWrapper.kt:105)
at com.jetbrains.bigdatatools.common.settings.defaultui.TestConnectionPanelWrapper$runTesting$1.invoke(TestConnectionPanelWrapper.kt:84)
at com.jetbrains.bigdatatools.common.util.ThreadUtilsKt.executeOnPooledThread$lambda$3(ThreadUtils.kt:31)
at com.intellij.openapi.application.impl.ApplicationImpl$1.run(ApplicationImpl.java:252)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:702)
at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:699)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1.run(Executors.java:699)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: com.jetbrains.bigdatatools.common.connection.exception.impl.RestExecutionException: Error during 'GET: /api/v1/version' request.

so how can i solve this problem? Thanks


i tried to use other port to connect to the spark master but failed.


</details>


# 答案1
**得分**: 0

这可能与Spark版本冲突有关。

您可以检查版本的兼容性，如果需要的话进行修改，并重新测试。

<details>
<summary>英文:</summary>

This can be related to a spark version conflict.

You can check the version&#39;s compatibility, apply modification if needed and retest.

</details>