Caused by: java.lang.ClassNotFoundException: org.apache.orc.storage.ql.exec.vector.VectorizedRowBatch appears in flink query hive

huangapple
go评论95阅读模式
英文:

Caused by: java.lang.ClassNotFoundException: org.apache.orc.storage.ql.exec.vector.VectorizedRowBatch appears in flink query hive

问题

  1. 1. 软件版本：
  2. * flink 1.11
  3. * hive 1.2.1
  4. * hadoop 2.7.1
  6. 2. 使用flink运行jar来运行提交程序时出现以下异常
  8. ```text
  9. org.apache.flink.runtime.JobException: 由 NoRestartBackoffTimeStrategy 抑制了恢复
  10. 	at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:116)
  11. 	...
  12. Caused by: java.lang.NoClassDefFoundError: org/apache/orc/storage/ql/exec/vector/VectorizedRowBatch
  13. 	...
  14. Caused by: java.lang.ClassNotFoundException: org.apache.orc.storage.ql.exec.vector.VectorizedRowBatch
  15. 	...
  1. 我在Maven中添加了以下依赖
  1. <dependency>
  2.   <groupId>org.apache.hive</groupId>
  3.   <artifactId>hive-storage-api</artifactId>
  4.   <version>2.0.0</version>
  5. </dependency>

但程序仍然报错，我不太清楚是什么原因导致的！

  1. 我的POM配置如下
  1. <!-- 你提供的POM配置 -->
  1. 我在flink的lib目录下添加了以下jar包
  1. flink-sql-connector-hive-1.2.2
  2. hive-metastore-1.2.1.jar
  3. hive-exec-1.2.1.jar
  4. libfb303-0.9.2.jar 
  5. orc-core-1.4.3-nohive.jar
  6. aircompressor-0.8.jar
  2. <details>
  3. <summary>英文:</summary>
  5. 1. Software version:
  6. * flink 1.11
  7. * hive1.2.1
  8. * hadoop2.7.1
  10. 2. Use flink run jar to run the submission program with the following exceptions
  12. ```text
  13. org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy
  14. 	at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:116)
  15. 	at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:78)
  16. 	at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:192)
  17. 	at org.apache.flink.runtime.scheduler.DefaultScheduler.maybeHandleTaskFailure(DefaultScheduler.java:185)
  18. 	at org.apache.flink.runtime.scheduler.DefaultScheduler.updateTaskExecutionStateInternal(DefaultScheduler.java:179)
  19. 	at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:503)
  20. 	at org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:386)
  21. 	at sun.reflect.GeneratedMethodAccessor70.invoke(Unknown Source)
  22. 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  23. 	at java.lang.reflect.Method.invoke(Method.java:497)
  24. 	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:284)
  25. 	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:199)
  26. 	at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74)
  27. 	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:152)
  28. 	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$$Lambda$98/618592213.apply(Unknown Source)
  29. 	at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26)
  30. 	at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21)
  31. 	at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
  32. 	at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21)
  33. 	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170)
  34. 	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
  35. 	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
  36. 	at akka.actor.Actor$class.aroundReceive(Actor.scala:517)
  37. 	at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225)
  38. 	at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592)
  39. 	at akka.actor.ActorCell.invoke(ActorCell.scala:561)
  40. 	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258)
  41. 	at akka.dispatch.Mailbox.run(Mailbox.scala:225)
  42. 	at akka.dispatch.Mailbox.exec(Mailbox.scala:235)
  43. 	at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  44. 	at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  45. 	at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  46. 	at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
  47. Caused by: java.lang.NoClassDefFoundError: org/apache/orc/storage/ql/exec/vector/VectorizedRowBatch
  48. 	at org.apache.flink.orc.nohive.OrcNoHiveSplitReaderUtil.genPartColumnarRowReader(OrcNoHiveSplitReaderUtil.java:67)
  49. 	at org.apache.flink.connectors.hive.read.HiveVectorizedOrcSplitReader.&lt;init&gt;(HiveVectorizedOrcSplitReader.java:67)
  50. 	at org.apache.flink.connectors.hive.read.HiveTableInputFormat.open(HiveTableInputFormat.java:137)
  51. 	at org.apache.flink.connectors.hive.read.HiveTableInputFormat.open(HiveTableInputFormat.java:66)
  52. 	at org.apache.flink.streaming.api.functions.source.InputFormatSourceFunction.run(InputFormatSourceFunction.java:85)
  53. 	at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:100)
  54. 	at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:63)
  55. 	at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:213)
  56. Caused by: java.lang.ClassNotFoundException: org.apache.orc.storage.ql.exec.vector.VectorizedRowBatch
  57. 	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
  58. 	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
  59. 	at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:61)
  60. 	at org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:74)
  61. 	at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)
  62. 	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
  63. 	... 8 more
  1. I added the following dependencies in maven
  1. &lt;dependency&gt;
  2.   &lt;groupId&gt;org.apache.hive&lt;/groupId&gt;
  3.   &lt;artifactId&gt;hive-storage-api&lt;/artifactId&gt;
  4.   &lt;version&gt;2.0.0&lt;/version&gt;
  5. &lt;/dependency&gt;

The program still reports an error, I don’t quite clear what caused it!

  1. My pom configuration is as follows
  2. &lt;project xmlns=&quot;http://maven.apache.org/POM/4.0.0&quot; xmlns:xsi=&quot;http://www.w3.org/2001/XMLSchema-instance&quot;
  3. 	xsi:schemaLocation=&quot;http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd&quot;&gt;
  4. 	&lt;modelVersion&gt;4.0.0&lt;/modelVersion&gt;
  6. 	&lt;groupId&gt;com.credit.analyze&lt;/groupId&gt;
  7. 	&lt;artifactId&gt;fast-analyze&lt;/artifactId&gt;
  8. 	&lt;version&gt;0.1&lt;/version&gt;
  9. 	&lt;packaging&gt;jar&lt;/packaging&gt;
  11. 	&lt;name&gt;Fast Analyze Job&lt;/name&gt;
  13. 	&lt;properties&gt;
  14. 		&lt;project.build.sourceEncoding&gt;UTF-8&lt;/project.build.sourceEncoding&gt;
  15. 		&lt;flink.version&gt;1.11.0&lt;/flink.version&gt;
  16. 		&lt;java.version&gt;1.8&lt;/java.version&gt;
  17. 		&lt;scala.binary.version&gt;2.11&lt;/scala.binary.version&gt;
  18. 		&lt;maven.compiler.source&gt;${java.version}&lt;/maven.compiler.source&gt;
  19. 		&lt;maven.compiler.target&gt;${java.version}&lt;/maven.compiler.target&gt;
  20. 		&lt;log4j.version&gt;2.12.1&lt;/log4j.version&gt;
  21. 		&lt;hive.version&gt;1.2.1&lt;/hive.version&gt;
  22. 		&lt;hadoop.version&gt;2.7.1&lt;/hadoop.version&gt;
  23. 		&lt;mysql.version&gt;5.1.16&lt;/mysql.version&gt;
  24.         &lt;junit.version&gt;4.12&lt;/junit.version&gt;
  25. 	&lt;/properties&gt;
  27. 	&lt;repositories&gt;
  28. 		&lt;repository&gt;
  29. 			&lt;id&gt;apache.snapshots&lt;/id&gt;
  30. 			&lt;name&gt;Apache Development Snapshot Repository&lt;/name&gt;
  31. 			&lt;url&gt;https://repository.apache.org/content/repositories/snapshots/&lt;/url&gt;
  32. 			&lt;releases&gt;
  33. 				&lt;enabled&gt;false&lt;/enabled&gt;
  34. 			&lt;/releases&gt;
  35. 			&lt;snapshots&gt;
  36. 				&lt;enabled&gt;true&lt;/enabled&gt;
  37. 			&lt;/snapshots&gt;
  38. 		&lt;/repository&gt;
  39. 	&lt;/repositories&gt;
  41. 	&lt;dependencies&gt;
  42. 		&lt;!-- Apache Flink dependencies --&gt;
  43. 		&lt;!-- These dependencies are provided, because they should not be packaged into the JAR file. --&gt;
  44. 		&lt;dependency&gt;
  45. 			&lt;groupId&gt;org.apache.flink&lt;/groupId&gt;
  46. 			&lt;artifactId&gt;flink-java&lt;/artifactId&gt;
  47. 			&lt;version&gt;${flink.version}&lt;/version&gt;
  48. 			&lt;!--&lt;scope&gt;provided&lt;/scope&gt;--&gt;
  49. 		&lt;/dependency&gt;
  50. 		&lt;dependency&gt;
  51. 			&lt;groupId&gt;org.apache.flink&lt;/groupId&gt;
  52. 			&lt;artifactId&gt;flink-streaming-java_${scala.binary.version}&lt;/artifactId&gt;
  53. 			&lt;version&gt;${flink.version}&lt;/version&gt;
  54. 			&lt;!--&lt;scope&gt;provided&lt;/scope&gt;--&gt;
  55. 		&lt;/dependency&gt;
  56. 		&lt;dependency&gt;
  57. 			&lt;groupId&gt;org.apache.flink&lt;/groupId&gt;
  58. 			&lt;artifactId&gt;flink-clients_${scala.binary.version}&lt;/artifactId&gt;
  59. 			&lt;version&gt;${flink.version}&lt;/version&gt;
  60. 			&lt;!--&lt;scope&gt;provided&lt;/scope&gt;--&gt;
  61. 		&lt;/dependency&gt;
  63. 		&lt;!-- Add connector dependencies here. They must be in the default scope (compile). --&gt;
  65. 		&lt;dependency&gt;
  66. 			&lt;groupId&gt;org.apache.flink&lt;/groupId&gt;
  67. 			&lt;artifactId&gt;flink-table-api-java-bridge_${scala.binary.version}&lt;/artifactId&gt;
  68. 			&lt;version&gt;${flink.version}&lt;/version&gt;
  69. 			&lt;!--&lt;scope&gt;provided&lt;/scope&gt;--&gt;
  70. 		&lt;/dependency&gt;
  72. 		&lt;dependency&gt;
  73. 			&lt;groupId&gt;org.apache.flink&lt;/groupId&gt;
  74. 			&lt;artifactId&gt;flink-table-planner_${scala.binary.version}&lt;/artifactId&gt;
  75. 			&lt;version&gt;${flink.version}&lt;/version&gt;
  76. 			&lt;scope&gt;provided&lt;/scope&gt;
  77. 		&lt;/dependency&gt;
  78. 		&lt;!-- or.. (for the new Blink planner) --&gt;
  79. 		&lt;dependency&gt;
  80. 			&lt;groupId&gt;org.apache.flink&lt;/groupId&gt;
  81. 			&lt;artifactId&gt;flink-table-planner-blink_${scala.binary.version}&lt;/artifactId&gt;
  82. 			&lt;version&gt;${flink.version}&lt;/version&gt;
  83. 			&lt;scope&gt;provided&lt;/scope&gt;
  84. 		&lt;/dependency&gt;
  87. 		&lt;dependency&gt;
  88. 			&lt;groupId&gt;org.apache.flink&lt;/groupId&gt;
  89. 			&lt;artifactId&gt;flink-streaming-scala_${scala.binary.version}&lt;/artifactId&gt;
  90. 			&lt;version&gt;${flink.version}&lt;/version&gt;
  91. 			&lt;!--&lt;scope&gt;provided&lt;/scope&gt;--&gt;
  92. 		&lt;/dependency&gt;
  94. 		&lt;dependency&gt;
  95. 			&lt;groupId&gt;org.apache.flink&lt;/groupId&gt;
  96. 			&lt;artifactId&gt;flink-connector-hive_${scala.binary.version}&lt;/artifactId&gt;
  97. 			&lt;version&gt;${flink.version}&lt;/version&gt;
  98. 			&lt;!--&lt;scope&gt;provided&lt;/scope&gt;--&gt;
  99. 		&lt;/dependency&gt;
  101. 		&lt;dependency&gt;
  102. 			&lt;groupId&gt;org.apache.flink&lt;/groupId&gt;
  103. 			&lt;artifactId&gt;flink-table-api-java-bridge_${scala.binary.version}&lt;/artifactId&gt;
  104. 			&lt;version&gt;${flink.version}&lt;/version&gt;
  105. 			&lt;!--&lt;scope&gt;provided&lt;/scope&gt;--&gt;
  106. 		&lt;/dependency&gt;
  108. 		&lt;dependency&gt;
  109. 			&lt;groupId&gt;org.apache.hive&lt;/groupId&gt;
  110. 			&lt;artifactId&gt;hive-exec&lt;/artifactId&gt;
  111. 			&lt;version&gt;${hive.version}&lt;/version&gt;
  112. 			&lt;scope&gt;provided&lt;/scope&gt;
  113. 		&lt;/dependency&gt;
  115. 		&lt;dependency&gt;
  116. 			&lt;groupId&gt;org.apache.hadoop&lt;/groupId&gt;
  117. 			&lt;artifactId&gt;hadoop-client&lt;/artifactId&gt;
  118. 			&lt;version&gt;${hadoop.version}&lt;/version&gt;
  119. 		&lt;/dependency&gt;
  121. 		&lt;dependency&gt;
  122. 			&lt;groupId&gt;org.apache.flink&lt;/groupId&gt;
  123. 			&lt;artifactId&gt;flink-connector-jdbc_${scala.binary.version}&lt;/artifactId&gt;
  124. 			&lt;version&gt;${flink.version}&lt;/version&gt;
  125. 		&lt;/dependency&gt;
  127. 		&lt;dependency&gt;
  128. 			&lt;groupId&gt;mysql&lt;/groupId&gt;
  129. 			&lt;artifactId&gt;mysql-connector-java&lt;/artifactId&gt;
  130. 			&lt;version&gt;${mysql.version}&lt;/version&gt;
  131. 		&lt;/dependency&gt;
  134. 		&lt;dependency&gt;
  135. 			&lt;groupId&gt;junit&lt;/groupId&gt;
  136. 			&lt;artifactId&gt;junit&lt;/artifactId&gt;
  137. 			&lt;version&gt;${junit.version}&lt;/version&gt;
  138. 		&lt;/dependency&gt;
  142. 		&lt;dependency&gt;
  143. 			&lt;groupId&gt;org.apache.orc&lt;/groupId&gt;
  144. 			&lt;artifactId&gt;orc-core&lt;/artifactId&gt;
  145. 			&lt;version&gt;1.6.5&lt;/version&gt;
  146. 		&lt;/dependency&gt;
  154. 		&lt;!-- Add logging framework, to produce console output when running in the IDE. --&gt;
  155. 		&lt;!-- These dependencies are excluded from the application JAR by default. --&gt;
  156. 		&lt;dependency&gt;
  157. 			&lt;groupId&gt;org.apache.logging.log4j&lt;/groupId&gt;
  158. 			&lt;artifactId&gt;log4j-slf4j-impl&lt;/artifactId&gt;
  159. 			&lt;version&gt;${log4j.version}&lt;/version&gt;
  160. 			&lt;scope&gt;runtime&lt;/scope&gt;
  161. 		&lt;/dependency&gt;
  162. 		&lt;dependency&gt;
  163. 			&lt;groupId&gt;org.apache.logging.log4j&lt;/groupId&gt;
  164. 			&lt;artifactId&gt;log4j-api&lt;/artifactId&gt;
  165. 			&lt;version&gt;${log4j.version}&lt;/version&gt;
  166. 			&lt;scope&gt;runtime&lt;/scope&gt;
  167. 		&lt;/dependency&gt;
  168. 		&lt;dependency&gt;
  169. 			&lt;groupId&gt;org.apache.logging.log4j&lt;/groupId&gt;
  170. 			&lt;artifactId&gt;log4j-core&lt;/artifactId&gt;
  171. 			&lt;version&gt;${log4j.version}&lt;/version&gt;
  172. 			&lt;scope&gt;runtime&lt;/scope&gt;
  173. 		&lt;/dependency&gt;
  174. 	&lt;/dependencies&gt;
  176. 	&lt;build&gt;
  177. 		&lt;plugins&gt;
  179. 			&lt;!-- Java Compiler --&gt;
  180. 			&lt;plugin&gt;
  181. 				&lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
  182. 				&lt;artifactId&gt;maven-compiler-plugin&lt;/artifactId&gt;
  183. 				&lt;version&gt;3.1&lt;/version&gt;
  184. 				&lt;configuration&gt;
  185. 					&lt;source&gt;${java.version}&lt;/source&gt;
  186. 					&lt;target&gt;${java.version}&lt;/target&gt;
  187. 				&lt;/configuration&gt;
  188. 			&lt;/plugin&gt;
  190. 			&lt;!-- We use the maven-shade plugin to create a fat jar that contains all necessary dependencies. --&gt;
  191. 			&lt;!-- Change the value of &lt;mainClass&gt;...&lt;/mainClass&gt; if your program entry point changes. --&gt;
  192. 			&lt;plugin&gt;
  193. 				&lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
  194. 				&lt;artifactId&gt;maven-shade-plugin&lt;/artifactId&gt;
  195. 				&lt;version&gt;3.1.1&lt;/version&gt;
  196. 				&lt;executions&gt;
  197. 					&lt;!-- Run shade goal on package phase --&gt;
  198. 					&lt;execution&gt;
  199. 						&lt;phase&gt;package&lt;/phase&gt;
  200. 						&lt;goals&gt;
  201. 							&lt;goal&gt;shade&lt;/goal&gt;
  202. 						&lt;/goals&gt;
  203. 						&lt;configuration&gt;
  204. 							&lt;artifactSet&gt;
  205. 								&lt;excludes&gt;
  206. 									&lt;exclude&gt;org.apache.flink:force-shading&lt;/exclude&gt;
  207. 									&lt;exclude&gt;com.google.code.findbugs:jsr305&lt;/exclude&gt;
  208. 									&lt;exclude&gt;org.slf4j:*&lt;/exclude&gt;
  209. 									&lt;exclude&gt;org.apache.logging.log4j:*&lt;/exclude&gt;
  210. 								&lt;/excludes&gt;
  211. 							&lt;/artifactSet&gt;
  212. 							&lt;filters&gt;
  213. 								&lt;filter&gt;
  214. 									&lt;!-- Do not copy the signatures in the META-INF folder.
  215. 									Otherwise, this might cause SecurityExceptions when using the JAR. --&gt;
  216. 									&lt;artifact&gt;*:*&lt;/artifact&gt;
  217. 									&lt;excludes&gt;
  218. 										&lt;exclude&gt;META-INF/*.SF&lt;/exclude&gt;
  219. 										&lt;exclude&gt;META-INF/*.DSA&lt;/exclude&gt;
  220. 										&lt;exclude&gt;META-INF/*.RSA&lt;/exclude&gt;
  221. 									&lt;/excludes&gt;
  222. 								&lt;/filter&gt;
  223. 							&lt;/filters&gt;
  224. 							&lt;transformers&gt;
  225. 								&lt;transformer implementation=&quot;org.apache.maven.plugins.shade.resource.ManifestResourceTransformer&quot;&gt;
  226. 									&lt;mainClass&gt;com.credit.analyze.job.batch.market.channel.MarketMonitorHiveJob&lt;/mainClass&gt;
  227. 								&lt;/transformer&gt;
  228. 							&lt;/transformers&gt;
  229. 						&lt;/configuration&gt;
  230. 					&lt;/execution&gt;
  231. 				&lt;/executions&gt;
  232. 			&lt;/plugin&gt;
  233. 		&lt;/plugins&gt;
  235. 		&lt;pluginManagement&gt;
  236. 			&lt;plugins&gt;
  238. 				&lt;!-- This improves the out-of-the-box experience in Eclipse by resolving some warnings. --&gt;
  239. 				&lt;plugin&gt;
  240. 					&lt;groupId&gt;org.eclipse.m2e&lt;/groupId&gt;
  241. 					&lt;artifactId&gt;lifecycle-mapping&lt;/artifactId&gt;
  242. 					&lt;version&gt;1.0.0&lt;/version&gt;
  243. 					&lt;configuration&gt;
  244. 						&lt;lifecycleMappingMetadata&gt;
  245. 							&lt;pluginExecutions&gt;
  246. 								&lt;pluginExecution&gt;
  247. 									&lt;pluginExecutionFilter&gt;
  248. 										&lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
  249. 										&lt;artifactId&gt;maven-shade-plugin&lt;/artifactId&gt;
  250. 										&lt;versionRange&gt;[3.1.1,)&lt;/versionRange&gt;
  251. 										&lt;goals&gt;
  252. 											&lt;goal&gt;shade&lt;/goal&gt;
  253. 										&lt;/goals&gt;
  254. 									&lt;/pluginExecutionFilter&gt;
  255. 									&lt;action&gt;
  256. 										&lt;ignore/&gt;
  257. 									&lt;/action&gt;
  258. 								&lt;/pluginExecution&gt;
  259. 								&lt;pluginExecution&gt;
  260. 									&lt;pluginExecutionFilter&gt;
  261. 										&lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
  262. 										&lt;artifactId&gt;maven-compiler-plugin&lt;/artifactId&gt;
  263. 										&lt;versionRange&gt;[3.1,)&lt;/versionRange&gt;
  264. 										&lt;goals&gt;
  265. 											&lt;goal&gt;testCompile&lt;/goal&gt;
  266. 											&lt;goal&gt;compile&lt;/goal&gt;
  267. 										&lt;/goals&gt;
  268. 									&lt;/pluginExecutionFilter&gt;
  269. 									&lt;action&gt;
  270. 										&lt;ignore/&gt;
  271. 									&lt;/action&gt;
  272. 								&lt;/pluginExecution&gt;
  273. 							&lt;/pluginExecutions&gt;
  274. 						&lt;/lifecycleMappingMetadata&gt;
  275. 					&lt;/configuration&gt;
  276. 				&lt;/plugin&gt;
  277. 			&lt;/plugins&gt;
  278. 		&lt;/pluginManagement&gt;
  279. 	&lt;/build&gt;
  280. &lt;/project&gt;
  1. I have added the following jar package in the flink lib directory
  1. flink-sql-connector-hive-1.2.2
  2. hive-metastore-1.2.1.jar
  3. hive-exec-1.2.1.jar
  4. libfb303-0.9.2.jar 
  5. orc-core-1.4.3-nohive.jar
  6. aircompressor-0.8.jar

答案1

得分: 1

我已解决这个问题。需要在 pom.xml 中添加以下依赖项：

  1. <!-- flink orc nohive -->
  2. <dependency>
  3.     <groupId>org.apache.flink</groupId>
  4.     <artifactId>flink-orc-nohive_2.12</artifactId>
  5.     <version>${flink.version}</version>
  6. </dependency>
英文:

I have solved the problem. Need to add the following dependencies in pom.xml

  1. &lt;!-- flink orc nohive --&gt;
  2. 		&lt;dependency&gt;
  3. 			&lt;groupId&gt;org.apache.flink&lt;/groupId&gt;
  4. 			&lt;artifactId&gt;flink-orc-nohive_2.12&lt;/artifactId&gt;
  5. 			&lt;version&gt;${flink.version}&lt;/version&gt;
  6. 		&lt;/dependency&gt;

答案2

得分: 0

你的错误明确指出，确保你在依赖中导入了正确的 orc-core jar 包，如下所示，

  1. Caused by: java.lang.ClassNotFoundException: org.apache.orc.storage.ql.exec.vector.VectorizedRowBatch

你应该添加以下依赖项，

  1. <dependency>
  2. <groupId>org.apache.orc</groupId>
  3. <artifactId>orc-core</artifactId>
  4. <version>1.6.5</version>
  5. </dependency>
英文:

Your error clearly indicates, make sure that your have proper orc-core jar imported as dependency in case,

  1. Caused by: java.lang.ClassNotFoundException: org.apache.orc.storage.ql.exec.vector.VectorizedRowBatch

you should add the following artifact,

  1. &lt;dependency&gt;
  2. &lt;groupId&gt;org.apache.orc&lt;/groupId&gt;
  3. &lt;artifactId&gt;orc-core&lt;/artifactId&gt;
  4. &lt;version&gt;1.6.5&lt;/version&gt;
  5. &lt;/dependency&gt;

huangapple
  • 本文由 发表于 2020年10月13日 19:52:26
  • 转载请务必保留本文链接：https://go.coder-hub.com/64334718.html
  • apache-flink
  • flink-sql
  • hive
  • java
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定