英文:
java.lang.NoClassDefFoundError: org/apache/flink/streaming/connectors/rabbitmq/common/RMQConnectionConfig in flink job
问题
以下是您提供的代码部分的翻译:
我正在将 Flink(版本 v1.10.0)作业提交到我的集群,但出现以下错误:
System.out:(无)
System.err:(无)
at org.apache.flink.client.program.OptimizerPlanEnvironment.generateException(OptimizerPlanEnvironment.java:149)
at org.apache.flink.client.program.OptimizerPlanEnvironment.getPipeline(OptimizerPlanEnvironment.java:89)
at org.apache.flink.client.program.PackagedProgramUtils.getPipelineFromProgram(PackagedProgramUtils.java:101)
at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:56)
at org.apache.flink.runtime.webmonitor.handlers.utils.JarHandlerUtils$JarHandlerContext.toJobGraph(JarHandlerUtils.java:128)
at org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$getJobGraphAsync$6(JarRunHandler.java:138)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError: org/apache/flink/streaming/connectors/rabbitmq/common/RMQConnectionConfig$Builder
at org.myorg.quickstart.StreamingJob.i(StreamingJob.java:123)
at org.myorg.quickstart.StreamingJob.main(StreamingJob.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
at org.apache.flink.client.program.OptimizerPlanEnvironment.getPipeline(OptimizerPlanEnvironment.java:79)
... 8 more
Caused by: java.lang.ClassNotFoundException: org.apache.flink.streaming.connectors.rabbitmq.common.RMQConnectionConfig$Builder
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
at org.apache.flink.util.ChildFirstClassLoader.loadClass(ChildFirstClassLoader.java:60)
at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
... 17 more
似乎在运行时找不到 RMQConnectionConfig,如何在系统范围内添加该 JAR 并使我的任务 JAR 变得更小更薄?这是我的 gradle.build 依赖项:
dependencies {
// --------------------------------------------------------------
// 在 Flink 的 lib 文件夹中提供的不应包含在 shadow JAR 中的编译时依赖项
// --------------------------------------------------------------
compile "org.apache.flink:flink-java:${flinkVersion}"
compile "org.apache.flink:flink-streaming-java_${scalaBinaryVersion}:${flinkVersion}"
// --------------------------------------------------------------
// 应包含在 shadow JAR 中的依赖项,例如连接器。这些必须在 flinkShadowJar 配置中!
// --------------------------------------------------------------
//flinkShadowJar "org.apache.flink:flink-connector-kafka-0.11_${scalaBinaryVersion}:${flinkVersion}"
compile "log4j:log4j:${log4jVersion}"
compile "org.slf4j:slf4j-log4j12:${slf4jVersion}"
compile "com.rabbitmq:amqp-client:4.2.0"
compile "com.alibaba:fastjson:1.2.57"
compile "org.apache.flink:flink-connector-rabbitmq_${scalaBinaryVersion}:1.10.0"
}
英文:
I am commit flink(v1.10.0) job to my cluster, but shows this error:
System.out: (none)
System.err: (none)
at org.apache.flink.client.program.OptimizerPlanEnvironment.generateException(OptimizerPlanEnvironment.java:149)
at org.apache.flink.client.program.OptimizerPlanEnvironment.getPipeline(OptimizerPlanEnvironment.java:89)
at org.apache.flink.client.program.PackagedProgramUtils.getPipelineFromProgram(PackagedProgramUtils.java:101)
at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:56)
at org.apache.flink.runtime.webmonitor.handlers.utils.JarHandlerUtils$JarHandlerContext.toJobGraph(JarHandlerUtils.java:128)
at org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$getJobGraphAsync$6(JarRunHandler.java:138)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError: org/apache/flink/streaming/connectors/rabbitmq/common/RMQConnectionConfig$Builder
at org.myorg.quickstart.StreamingJob.i(StreamingJob.java:123)
at org.myorg.quickstart.StreamingJob.main(StreamingJob.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
at org.apache.flink.client.program.OptimizerPlanEnvironment.getPipeline(OptimizerPlanEnvironment.java:79)
... 8 more
Caused by: java.lang.ClassNotFoundException: org.apache.flink.streaming.connectors.rabbitmq.common.RMQConnectionConfig$Builder
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
at org.apache.flink.util.ChildFirstClassLoader.loadClass(ChildFirstClassLoader.java:60)
at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
... 17 more
it seems like could not find RMQConnectionConfig in runtime, how to add the jar in system wide and make my task jar small and thin? This is my gradle.build dependencies:
dependencies {
// --------------------------------------------------------------
// Compile-time dependencies that should NOT be part of the
// shadow jar and are provided in the lib folder of Flink
// --------------------------------------------------------------
compile "org.apache.flink:flink-java:${flinkVersion}"
compile "org.apache.flink:flink-streaming-java_${scalaBinaryVersion}:${flinkVersion}"
// --------------------------------------------------------------
// Dependencies that should be part of the shadow jar, e.g.
// connectors. These must be in the flinkShadowJar configuration!
// --------------------------------------------------------------
//flinkShadowJar "org.apache.flink:flink-connector-kafka-0.11_${scalaBinaryVersion}:${flinkVersion}"
compile "log4j:log4j:${log4jVersion}"
compile "org.slf4j:slf4j-log4j12:${slf4jVersion}"
compile "com.rabbitmq:amqp-client:4.2.0"
compile "com.alibaba:fastjson:1.2.57"
compile "org.apache.flink:flink-connector-rabbitmq_${scalaBinaryVersion}:1.10.0"
}
答案1
得分: 0
如果您希望将 jar 文件中包含到 dist.jar 文件中,您应该像这样编写依赖项:
flinkShadowJar "com.rabbitmq:amqp-client:4.2.0"
flinkShadowJar "com.alibaba:fastjson:1.2.57"
flinkShadowJar "org.apache.flink:flink-connector-rabbitmq_${scalaBinaryVersion}:1.10.0"
这将会将依赖项打包到 dist jar 文件中,并在运行时使用。
英文:
if you want the jar contains to dist.jar file,you should write dependencies like this:
flinkShadowJar "com.rabbitmq:amqp-client:4.2.0"
flinkShadowJar "com.alibaba:fastjson:1.2.57"
flinkShadowJar "org.apache.flink:flink-connector-rabbitmq_${scalaBinaryVersion}:1.10.0"
it will package the dependencies to dist jar file and using at runtime.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论