java.lang.NoClassDefFoundError: org/apache/flink/streaming/connectors/rabbitmq/common/RMQConnectionConfig in flink job

huangapple go评论97阅读模式
英文:

java.lang.NoClassDefFoundError: org/apache/flink/streaming/connectors/rabbitmq/common/RMQConnectionConfig in flink job

问题

以下是您提供的代码部分的翻译:

我正在将 Flink版本 v1.10.0作业提交到我的集群但出现以下错误

System.out:
System.err:
	at org.apache.flink.client.program.OptimizerPlanEnvironment.generateException(OptimizerPlanEnvironment.java:149)
	at org.apache.flink.client.program.OptimizerPlanEnvironment.getPipeline(OptimizerPlanEnvironment.java:89)
	at org.apache.flink.client.program.PackagedProgramUtils.getPipelineFromProgram(PackagedProgramUtils.java:101)
	at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:56)
	at org.apache.flink.runtime.webmonitor.handlers.utils.JarHandlerUtils$JarHandlerContext.toJobGraph(JarHandlerUtils.java:128)
	at org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$getJobGraphAsync$6(JarRunHandler.java:138)
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError: org/apache/flink/streaming/connectors/rabbitmq/common/RMQConnectionConfig$Builder
	at org.myorg.quickstart.StreamingJob.i(StreamingJob.java:123)
	at org.myorg.quickstart.StreamingJob.main(StreamingJob.java:55)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
	at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
	at org.apache.flink.client.program.OptimizerPlanEnvironment.getPipeline(OptimizerPlanEnvironment.java:79)
	... 8 more
Caused by: java.lang.ClassNotFoundException: org.apache.flink.streaming.connectors.rabbitmq.common.RMQConnectionConfig$Builder
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
	at org.apache.flink.util.ChildFirstClassLoader.loadClass(ChildFirstClassLoader.java:60)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
	... 17 more

似乎在运行时找不到 RMQConnectionConfig如何在系统范围内添加该 JAR 并使我的任务 JAR 变得更小更薄这是我的 gradle.build 依赖项

dependencies {

    // --------------------------------------------------------------
    // 在 Flink 的 lib 文件夹中提供的不应包含在 shadow JAR 中的编译时依赖项
    // --------------------------------------------------------------
    compile "org.apache.flink:flink-java:${flinkVersion}"
    compile "org.apache.flink:flink-streaming-java_${scalaBinaryVersion}:${flinkVersion}"

    // --------------------------------------------------------------
    // 应包含在 shadow JAR 中的依赖项,例如连接器。这些必须在 flinkShadowJar 配置中!
    // --------------------------------------------------------------
    //flinkShadowJar "org.apache.flink:flink-connector-kafka-0.11_${scalaBinaryVersion}:${flinkVersion}"

    compile "log4j:log4j:${log4jVersion}"
    compile "org.slf4j:slf4j-log4j12:${slf4jVersion}"

    compile "com.rabbitmq:amqp-client:4.2.0"
    compile "com.alibaba:fastjson:1.2.57"
    compile "org.apache.flink:flink-connector-rabbitmq_${scalaBinaryVersion}:1.10.0"

}
英文:

I am commit flink(v1.10.0) job to my cluster, but shows this error:

 System.out: (none)
System.err: (none)
at org.apache.flink.client.program.OptimizerPlanEnvironment.generateException(OptimizerPlanEnvironment.java:149)
at org.apache.flink.client.program.OptimizerPlanEnvironment.getPipeline(OptimizerPlanEnvironment.java:89)
at org.apache.flink.client.program.PackagedProgramUtils.getPipelineFromProgram(PackagedProgramUtils.java:101)
at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:56)
at org.apache.flink.runtime.webmonitor.handlers.utils.JarHandlerUtils$JarHandlerContext.toJobGraph(JarHandlerUtils.java:128)
at org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$getJobGraphAsync$6(JarRunHandler.java:138)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError: org/apache/flink/streaming/connectors/rabbitmq/common/RMQConnectionConfig$Builder
at org.myorg.quickstart.StreamingJob.i(StreamingJob.java:123)
at org.myorg.quickstart.StreamingJob.main(StreamingJob.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
at org.apache.flink.client.program.OptimizerPlanEnvironment.getPipeline(OptimizerPlanEnvironment.java:79)
... 8 more
Caused by: java.lang.ClassNotFoundException: org.apache.flink.streaming.connectors.rabbitmq.common.RMQConnectionConfig$Builder
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
at org.apache.flink.util.ChildFirstClassLoader.loadClass(ChildFirstClassLoader.java:60)
at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
... 17 more

it seems like could not find RMQConnectionConfig in runtime, how to add the jar in system wide and make my task jar small and thin? This is my gradle.build dependencies:

dependencies {
// --------------------------------------------------------------
// Compile-time dependencies that should NOT be part of the
// shadow jar and are provided in the lib folder of Flink
// --------------------------------------------------------------
compile "org.apache.flink:flink-java:${flinkVersion}"
compile "org.apache.flink:flink-streaming-java_${scalaBinaryVersion}:${flinkVersion}"
// --------------------------------------------------------------
// Dependencies that should be part of the shadow jar, e.g.
// connectors. These must be in the flinkShadowJar configuration!
// --------------------------------------------------------------
//flinkShadowJar "org.apache.flink:flink-connector-kafka-0.11_${scalaBinaryVersion}:${flinkVersion}"
compile "log4j:log4j:${log4jVersion}"
compile "org.slf4j:slf4j-log4j12:${slf4jVersion}"
compile "com.rabbitmq:amqp-client:4.2.0"
compile "com.alibaba:fastjson:1.2.57"
compile "org.apache.flink:flink-connector-rabbitmq_${scalaBinaryVersion}:1.10.0"
}

答案1

得分: 0

如果您希望将 jar 文件中包含到 dist.jar 文件中,您应该像这样编写依赖项:

flinkShadowJar "com.rabbitmq:amqp-client:4.2.0"
flinkShadowJar "com.alibaba:fastjson:1.2.57"
flinkShadowJar "org.apache.flink:flink-connector-rabbitmq_${scalaBinaryVersion}:1.10.0"

这将会将依赖项打包到 dist jar 文件中,并在运行时使用。

英文:

if you want the jar contains to dist.jar file,you should write dependencies like this:

 flinkShadowJar "com.rabbitmq:amqp-client:4.2.0"
flinkShadowJar "com.alibaba:fastjson:1.2.57"
flinkShadowJar "org.apache.flink:flink-connector-rabbitmq_${scalaBinaryVersion}:1.10.0"

it will package the dependencies to dist jar file and using at runtime.

huangapple
  • 本文由 发表于 2020年4月5日 15:47:37
  • 转载请务必保留本文链接:https://go.coder-hub.com/61039550.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定