英文:
java.lang.NoSuchMethodError: 'void org.apache.logging.slf4j.Log4jLoggerFactory.<init>(org.apache.logging.slf4j.Log4jMarkerFactory)'
问题
I'm currently on an Apache Flink job that provides log4j2 as the default logging implementation at runtime (see here). For flexibility, SLF4J should be used. Running the entire job in Flink logs properly, but during unit tests using JUnit 5, there are no logs shown in IntelliJ.
For debugging purposes, it would be necessary to have the log statements also during unit test execution.
I have created a minimal example project.
The Problem:
I have traced it down to the inclusion of org.apache.flink:flink-test-utils:1.16.1, which results in the following error:
Exception in thread "main" java.lang.NoSuchMethodError: 'void org.apache.logging.slf4j.Log4jLoggerFactory.<init>(org.apache.logging.slf4j.Log4jMarkerFactory)'
at org.apache.logging.slf4j.SLF4JServiceProvider.initialize(SLF4JServiceProvider.java:54)
at org.slf4j.LoggerFactory.bind(LoggerFactory.java:183)
at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:170)
at org.slf4j.LoggerFactory.getProvider(LoggerFactory.java:455)
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:441)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:390)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:416)
at bmw.pcmasfd.zip.flink.udf.BuildGroupContextTest.<clinit>(BuildGroupContextTest.java:27)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:315)
Context:
IntelliJ IDEA 2022.3.3 RC (Ultimate Edition)
Java version: 11.0.18 (Azul)
Flink version: 1.16.1
pom.xml(v1):
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.example</groupId>
<artifactId>minimal-logging-example</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<flink.version>1.16.1</flink.version>
</properties>
<dependencies>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.18.26</version>
</dependency>
<!-- Flink -->
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java</artifactId>
<version>1.16.1</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-runtime</artifactId>
<version>${flink.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-core</artifactId>
<version>${flink.version}</version>
<scope>provided</scope>
</dependency>
<!-- TESTING -->
<!--
only including this dependency leads to:
java.lang.NoSuchMethodError: 'void org.apache.logging.slf4j.Log4jLoggerFactory.<init>(org.apache.logging.slf4j.Log4jMarkerFactory)'
-->
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-test-utils</artifactId>
<version>${flink.version}</version>
<scope>test</scope>
</dependency>
<!-- Logging -->
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.17.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>2.0.6</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j2-impl</artifactId>
<version>2.20.0</version>
<scope>test</scope>
</dependency>
<!-- Testing -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<details>
<summary>英文:</summary>
im currently on a apache flink job which provides log4j2 as default logging implementation at its runtime ([see here](https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/deployment/advanced/logging/)).
For flexibility SLF4J should be used.
Running the whole job in flink logs properly, but during Unit Test using JUnit5 there are no logs shown in IntelliJ.
For debugging purposes it would be necessary to have the log statements also during unit test execution.
I have made a minimal [example project](https://github.com/iLem0n/minmal-logging-example)
**The Problem:**
I have drilled it down to inclusion of `org.apache.flink:flink-test-utils:1.16.1` which results to the following:
Exception in thread "main" java.lang.NoSuchMethodError: 'void org.apache.logging.slf4j.Log4jLoggerFactory.<init>(org.apache.logging.slf4j.Log4jMarkerFactory)'
at org.apache.logging.slf4j.SLF4JServiceProvider.initialize(SLF4JServiceProvider.java:54)
at org.slf4j.LoggerFactory.bind(LoggerFactory.java:183)
at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:170)
at org.slf4j.LoggerFactory.getProvider(LoggerFactory.java:455)
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:441)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:390)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:416)
at bmw.pcmasfd.zip.flink.udf.BuildGroupContextTest.<clinit>(BuildGroupContextTest.java:27)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:315)
**Context:**
IntelliJ IDEA 2022.3.3 RC (Ultimate Edition)
Java version: 11.0.18 (Azul)
Flink version: 1.16.1
**pom.xml**(v1):
```xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.example</groupId>
<artifactId>minimal-logging-example</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<flink.version>1.16.1</flink.version>
</properties>
<dependencies>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.18.26</version>
</dependency>
<!-- Flink -->
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java</artifactId>
<version>1.16.1</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-runtime</artifactId>
<version>${flink.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-core</artifactId>
<version>${flink.version}</version>
<scope>provided</scope>
</dependency>
<!-- TESTING -->
<!--
only including this dependency leads to:
java.lang.NoSuchMethodError: 'void org.apache.logging.slf4j.Log4jLoggerFactory.<init>(org.apache.logging.slf4j.Log4jMarkerFactory)'
-->
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-test-utils</artifactId>
<version>${flink.version}</version>
<scope>test</scope>
</dependency>
<!-- Logging -->
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.17.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>2.0.6</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j2-impl</artifactId>
<version>2.20.0</version>
<scope>test</scope>
</dependency>
<!-- Testing -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>5.9.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.skyscreamer</groupId>
<artifactId>jsonassert</artifactId>
<version>1.5.1</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
EDIT:
pom.xml(v2):
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.example</groupId>
<artifactId>minimal-logging-example</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<flink.version>1.16.1</flink.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-bom</artifactId>
<version>2.20.0</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.18.26</version>
</dependency>
<!-- Flink -->
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java</artifactId>
<version>1.16.1</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-runtime</artifactId>
<version>${flink.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-core</artifactId>
<version>${flink.version}</version>
<scope>provided</scope>
</dependency>
<!-- TESTING -->
<!--
only including this dependency leads to:
java.lang.NoSuchMethodError: 'void org.apache.logging.slf4j.Log4jLoggerFactory.<init>(org.apache.logging.slf4j.Log4jMarkerFactory)'
-->
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-test-utils</artifactId>
<version>${flink.version}</version>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
</exclusion>
</exclusions>
</dependency>
<!--
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-test-utils-junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java</artifactId>
<scope>test</scope>
<classifier>tests</classifier>
</dependency>
-->
<!-- Logging -->
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>2.0.6</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j2-impl</artifactId>
</dependency>
<!-- Testing -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>5.9.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.skyscreamer</groupId>
<artifactId>jsonassert</artifactId>
<version>1.5.1</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
答案1
得分: 1
The flink-test-utils 依赖项引入了一个旧版本的 log4j-slf4j-impl 作为传递依赖项（通过 flink-test-utils-junit）。
这可能是一个 bug，您应该报告它：作者忘记将 Log4j2 依赖项放在 test 范围内。库不应该将日志后端包括为 compile/runtime 依赖项，而应该将该选择留给库用户。
同时，您可以简单地添加一些排除项：
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-test-utils</artifactId>
<version>1.16.1</version>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
</exclusion>
</exclusions>
</dependency>
备注： log4j-core 和 log4j-slf4j2-impl 的版本应该保持同步（当前 log4j-core 将 log4j-api 2.17.1 引入您的项目，而 log4j-slf4j2-impl 需要版本 2.20.0）。最简单的方法是在依赖管理中使用 log4j-bom。
备注 2：
为了灵活性，应该使用 SLF4J。
这是一个常见的误解/过时观念：当引入 SLF4J 时，它提供了 Logback 的所有功能，同时允许用户使用传统的日志后端（Log4j 1.x、java.util.logging）。
当约 10 年后引入了 Log4j 2.x API 时，它提供了 Log4j 2.x Core 的所有功能，同时允许用户使用其他日志后端（Logback、java.util.logging）。
通过在 Log4j 2.x Core 中使用 SLF4J，您限制了可用的日志 API 方法，而实际上并没有增加日志后端的选择。
编辑： flink-test-utils 依赖项的另一个问题是它依赖于 flink-core 和 flink-runtime 的测试 jar，其中包含一个 log4j2-test.properties 文件。这是在测试期间禁用日志记录的文件。
现在您有几种解决方法：
- 测试 jar 通常只包含对项目本身有用的代码。您可以尝试从
flink-test-utils中排除flink-core和flink-runtime， - 向测试资源中添加一个
log4j2-test.properties文件，并依赖于通常情况下测试启动器在其依赖项之前将测试类放在类路径上。不幸的是，属性配置格式的优先级高于所有其他格式，因此您不能只使用另一种格式。 - 向测试资源中添加一个
log4j2-test.*配置文件（例如log4j2-test.xml），以及一个具有以下内容的log4j2.component.properties文件：
log4j2.configurationFile=log4j2-test.xml
英文:
The flink-test-utils dependency pulls an old version of log4j-slf4j-impl as transitive dependency (through flink-test-utils-junit).
This is probably a bug and you should report it: the authors forgot to put the Log4j2 dependencies in the test scope. Libraries should not include a logging backend as compile/runtime dependency and leave that choice to the library user.
In the meantime you can simply add some exclusions:
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-test-utils</artifactId>
<version>1.16.1</version>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
</exclusion>
</exclusions>
</dependency>
Remark: The versions of log4j-core and log4j-slf4j2-impl should be in sync (right now log4j-core pulls log4j-api 2.17.1 into your project, whereas log4j-slf4j2-impl requires version 2.20.0). The easiest way is to use log4j-bom in your dependency management.
Remark 2:
> For flexibility SLF4J should be used.
That is a common misconception/anachronism: when SLF4J was introduced, it offered all the features of Logback, while allowing users to use a legacy logging backend (Log4j 1.x, java.util.logging).
When the Log4j 2.x API was introduced about 10 years later, it offered all the features of Log4j 2.x Core, while allowing users to use other logging backends (Logback, java.util.logging).
By using SLF4J with Log4j 2.x Core, you limit the available logging API methods, while you don't really increase the choice of logging backends.
Edit: Another problem with the flink-test-utils dependency is that it depends on the tests jar of flink-core and flink-runtime, which contains a log4j2-test.properties file. This is the file that disables logging during testing.
Now you have several workarounds:
-
The test jars usually contain code only useful for the project itself. You can try excluding
flink-coreandflink-runtimefromflink-test-utils, -
Add a
log4j2-test.propertiesfile to your test resources and rely on the fact that usually test launchers place the test classes on the classpath before their dependencies. Unfortunately the properties configuration format has a higher priority than all other formats, so you can't just use another format. -
Add a
log4j2-test.*configuration file (e.g.log4j2-test.xml) to your test resources and alog4j2.component.propertiesfile with content:log4j2.configurationFile=log4j2-test.xml
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论