2020年10月18日 13:07:31go评论71阅读模式

英文:

phoenix table select join return Failed after attempts=11

问题

我在执行以下语句时遇到了错误:

SELECT r.*, 
       l.productid, 
       l.prodname, 
       l.projnum, 
       l.projname, 
       l.capitalnumb, 
       l.fundname, 
       l.channelno, 
       l.channelname, 
       l.merchantno, 
       l.merchantname, 
       l.staff_name, 
       l.busipers 
FROM   lake.lake_retu_record r 
       LEFT JOIN lake.lake_loan l 
              ON r.loanacno = l.loanacno 
WHERE  r.acdate = '20200912000000' 
LIMIT  1000

表信息:

lake.lake_retu_record 大小:40MB 数据：44000 条数据
lake.lake_loan 大小:1.5GB 数据: 1700000 条数据

版本:

HBase 版本: 2.1.0
Phoenix 核心: 5.0.0-cdh-6.2.0

我在网上搜索了很长时间，但没有解决这个问题。请帮忙或者尝试提供一些解决方法。
我尝试修改了这些参数 phoenix.query.queueSize phoenix.query.threadPoolSize phoenix.query.targetConcurrency phoenix.query.maxConcurrency

异常:

20/10/18 10:35:50 WARN client.SyncCoprocessorRpcChannel: IOException 上的调用失败
    org.apache.hadoop.hbase.client.RetriesExhaustedException: 尝试后失败=11次，异常:
    Sun Oct 18 10:35:08 CST 2020, RpcRetryingCaller{globalStartTime=1602988506012, pause=100, maxAttempts=11}, java.io.IOException: 调用到 CQ-HM-ALIFC-test/10.10.10.10:16020 失败，本地异常: java.io.IOException: 连接被对等方重置
    Sun Oct 18 10:35:08 CST 2020, RpcRetryingCaller{globalStartTime=1602988506012, pause=100, maxAttempts=11}, org.apache.hadoop.hbase.exceptions.ConnectionClosedException: 调用到 CQ-HM-ALIFC-test/10.10.10.10:16020 失败，本地异常: org.apache.hadoop.hbase.exceptions.ConnectionClosedException: 连接已关闭
    Sun Oct 18 10:35:09 CST 2020, RpcRetryingCaller{globalStartTime=1602988506012, pause=100, maxAttempts=11}, org.apache.hadoop.hbase.exceptions.ConnectionClosedException: 调用到 CQ-HM-ALIFC-test/10.10.10.10:16020 失败，本地异常: org.apache.hadoop.hbase.exceptions.ConnectionClosedException: 连接已关闭
    ... 更多异常行...

有人可以帮帮我吗..!

英文:

I get an error when I execute the following statement:

SELECT r.*, 
       l.productid, 
       l.prodname, 
       l.projnum, 
       l.projname, 
       l.capitalnumb, 
       l.fundname, 
       l.channelno, 
       l.channelname, 
       l.merchantno, 
       l.merchantname, 
       l.staff_name, 
       l.busipers 
FROM   lake.lake_retu_record r 
       LEFT JOIN lake.lake_loan l 
              ON r.loanacno = l.loanacno 
WHERE  r.acdate = &#39;20200912000000&#39; 
LIMIT  1000

table information:

lake.lake_retu_record size:40MB data：44000 pieces of data 
lake.lake_loan size:1.5GB data: 1700000 pieces of data

version:

HBase-Version : 2.1.0 
Phoenix-core: 5.0.0-cdh-6.2.0

I am searching for a long time on net. But it doesn't solve the problem. Please help or try to give some ideas how to solve this. 
I tried to modify these parameters phoenix.query.queueSize phoenix.query.threadPoolSize phoenix.query.targetConcurrency phoenix.query.maxConcurrency

Exception:

    20/10/18 10:35:50 WARN client.SyncCoprocessorRpcChannel: Call failed on IOException
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=11, exceptions:
Sun Oct 18 10:35:08 CST 2020, RpcRetryingCaller{globalStartTime=1602988506012, pause=100, maxAttempts=11}, java.io.IOException: Call to CQ-HM-ALIFC-test/10.10.10.10:16020 failed on local exception: java.io.IOException: Connection reset by peer
Sun Oct 18 10:35:08 CST 2020, RpcRetryingCaller{globalStartTime=1602988506012, pause=100, maxAttempts=11}, org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Call to CQ-HM-ALIFC-test/10.10.10.10:16020 failed on local exception: org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Connection closed
Sun Oct 18 10:35:09 CST 2020, RpcRetryingCaller{globalStartTime=1602988506012, pause=100, maxAttempts=11}, org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Call to CQ-HM-ALIFC-test/10.10.10.10:16020 failed on local exception: org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Connection closed
Sun Oct 18 10:35:10 CST 2020, RpcRetryingCaller{globalStartTime=1602988506012, pause=100, maxAttempts=11}, org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Call to CQ-HM-ALIFC-test/10.10.10.10:16020 failed on local exception: org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Connection closed
Sun Oct 18 10:35:11 CST 2020, RpcRetryingCaller{globalStartTime=1602988506012, pause=100, maxAttempts=11}, org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Call to CQ-HM-ALIFC-test/10.10.10.10:16020 failed on local exception: org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Connection closed
Sun Oct 18 10:35:12 CST 2020, RpcRetryingCaller{globalStartTime=1602988506012, pause=100, maxAttempts=11}, org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Call to CQ-HM-ALIFC-test/10.10.10.10:16020 failed on local exception: org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Connection closed
Sun Oct 18 10:35:15 CST 2020, RpcRetryingCaller{globalStartTime=1602988506012, pause=100, maxAttempts=11}, org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Call to CQ-HM-ALIFC-test/10.10.10.10:16020 failed on local exception: org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Connection closed
Sun Oct 18 10:35:19 CST 2020, RpcRetryingCaller{globalStartTime=1602988506012, pause=100, maxAttempts=11}, org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Call to CQ-HM-ALIFC-test/10.10.10.10:16020 failed on local exception: org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Connection closed
Sun Oct 18 10:35:29 CST 2020, RpcRetryingCaller{globalStartTime=1602988506012, pause=100, maxAttempts=11}, org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Call to CQ-HM-ALIFC-test/10.10.10.10:16020 failed on local exception: org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Connection closed
Sun Oct 18 10:35:40 CST 2020, RpcRetryingCaller{globalStartTime=1602988506012, pause=100, maxAttempts=11}, org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Call to CQ-HM-ALIFC-test/10.10.10.10:16020 failed on local exception: org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Connection closed
Sun Oct 18 10:35:50 CST 2020, RpcRetryingCaller{globalStartTime=1602988506012, pause=100, maxAttempts=11}, org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Call to CQ-HM-ALIFC-test/10.10.10.10:16020 failed on local exception: org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Connection closed
at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:145)
at org.apache.hadoop.hbase.client.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:91)
at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callMethod(SyncCoprocessorRpcChannel.java:52)
at org.apache.phoenix.coprocessor.generated.ServerCachingProtos$ServerCachingService$Stub.addServerCache(ServerCachingProtos.java:8484)
at org.apache.phoenix.cache.ServerCacheClient$3.call(ServerCacheClient.java:494)
at org.apache.phoenix.cache.ServerCacheClient$3.call(ServerCacheClient.java:462)
at org.apache.hadoop.hbase.client.HTable$12.call(HTable.java:997)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Call to CQ-HM-ALIFC-test/10.10.10.10:16020 failed on local exception: org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Connection closed
at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:202)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:390)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:95)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:410)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:406)
at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:103)
at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:118)
at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.cleanupCalls(NettyRpcDuplexHandler.java:203)
at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelInactive(NettyRpcDuplexHandler.java:211)
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:377)
at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:342)
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75)
at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelInactive(IdleStateHandler.java:277)
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1354)
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:917)
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:822)
at org.apache.hbase.thirdparty.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
... 1 more
Caused by: org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Connection closed
... 25 more

Can some one help me out..!

答案1

得分: 0

谢谢大家，我解决了这个问题！

解决方法如下： 
我找到了异常的主要原因

org.apache.hadoop.hbase.client.RetriesExhaustedException: 尝试11次后失败，异常：
2020年10月18日，中国标准时间上午10:35:08，RpcRetryingCaller{globalStartTime=1602988506012，暂停=100，最多尝试11次}，java.io.IOException：对CQ-HM-ALIFC-test/10.10.10.10:16020的调用失败，本地异常：java.io.IOException：连接被对等方重置

初步判断是RPC出了问题，所以在执行SQL语句时，我打开了报告错误的服务器的RegionServer日志。我发现RegionServer生成了以下日志：

警告 org.apache.hadoop.hbase.ipc.NettyRpcServer：从10.10.10.10接收的RPC数据长度为353247227，超过了允许的最大值268435456。在服务器上设置“hbase.ipc.max.request.size”以覆盖此限制（不建议这样做）

我增加了这个参数，问题得以解决！

英文:

thx everyone,I solved the problem!

The solutions are as follows: 
I found the main reason for the abnormality

org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=11, exceptions:
Sun Oct 18 10:35:08 CST 2020, RpcRetryingCaller{globalStartTime=1602988506012, pause=100, maxAttempts=11}, java.io.IOException: Call to CQ-HM-ALIFC-test/10.10.10.10:16020 failed on local exception: java.io.IOException: Connection reset by peer

The preliminary judgment is that the RPC has problems,So when I execute the SQL statement, I open the RegionServer log of the server where the error is reported.I found that RegionServer generates the following logs：

WARN org.apache.hadoop.hbase.ipc.NettyRpcServer: RPC data length of 353247227 received from 10.10.10.10 is greater than max allowed 268435456. Set &quot;hbase.ipc.max.request.size&quot; on server to override this limit (not recommended)

I increased the parameter and solved the problem!

通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。

凤凰表选择连接返回失败，尝试次数为11次。

问题

我在执行以下语句时遇到了错误:

表信息:

版本:

异常:

I get an error when I execute the following statement:

table information:<br/>

version:<br/>

Exception:<br/>

Can some one help me out..!

答案1

谢谢大家，我解决了这个问题！<br/>

thx everyone,I solved the problem!<br/>

练习JUnit时比较字符串时出现奇怪行为

我遇到了一个Java空指针异常。

从IntelliJ部署到Heroku的战争部分

“jms.queue/”和”jms.topic/”前缀的来源是什么？

What's the correct way to type hint an empty list as a literal in python?

如何在Highcharts Gantt中更改本地化的星期名称

如何在同一个流中使用多个过滤器和映射函数？

如何使用Map/Set来将代码优化到O(n)？

.NET MAUI Android在GitHub Actions上构建失败，错误代码为1。

如何在Playwright视觉比较中屏蔽多个定位器？

在C++中，可以使用可变模板参数来检索类型的内部类型。

selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: stale element not found

Creating and opening a URL to log in to Website via Basic Auth with Robot Framework/Selenium (Python)

AG Grid 在上下文菜单中以大文本形式打开

发表评论