英文:
How can I use Log4j2 logging on BEAM workers?
问题
I'm trying to use Log4j2 logging implementation together with Slf4j api in the BEAM pipeline. In maven it looks like this:
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.20.0</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j2-impl</artifactId>
<version>2.20.0</version>
</dependency>
It works fine locally. Also the logging works fine for the container where I start the pipeline. However, inside the workers this setup causes some issues:
Caused by: org.apache.logging.log4j.LoggingException: log4j-slf4j2-impl cannot be present with log4j-to-slf4j
This additional library log4j-to-slf4j
is not part of my project and it's not listed in dependencies (also not in transitive dependencies). Looks like it's added by BEAM sdk itself. The question is: how can I configure BEAM to use my version of Log4j?
英文:
I'm trying to use Log4j2 logging implementation together with Slf4j api in the BEAM pipeline. In maven it looks like this:
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.20.0</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j2-impl</artifactId>
<version>2.20.0</version>
</dependency>
It works fine locally. Also the logging works fine for the container where I start the pipeline. However, inside the workers this setup causes some issues:
Caused by: org.apache.logging.log4j.LoggingException: log4j-slf4j2-impl cannot be present with log4j-to-slf4j
This additional library log4j-to-slf4j
is not part of my project and it's not listed in dependencies (also not in transitive dependencies). Looks like it's added by BEAM sdk itself. The question is: how can I configure BEAM to use my version of Log4j?
答案1
得分: 0
当我们使用 Maven原型 生成 2.47.0
版本的 Beam
项目时:
mvn archetype:generate \
-DarchetypeGroupId=org.apache.beam \
-DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples \
-DarchetypeVersion=2.47.0 \
-DgroupId=org.example \
-DartifactId=word-count-beam \
-Dversion="0.1" \
-Dpackage=org.apache.beam.examples \
-DinteractiveMode=false
它将在 pom.xml
文件中生成以下依赖关系:
<properties>
<beam.version>2.47.0</beam.version>
<slf4j.version>1.7.30</slf4j.version>
</properties>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-jdk14</artifactId>
<version>${slf4j.version}</version>
<!-- When loaded at runtime this will wire up slf4j to the JUL backend -->
<scope>runtime</scope>
</dependency>
然后在代码中,你可以像在Java项目中一样使用 SL4J
:
public class TeamLeagueApp {
private static final Logger LOGGER = LoggerFactory.getLogger(TeamLeagueApp.class);
....
LOGGER.info("My log");
英文:
When we generate a Beam
project with 2.47.0
version using the Maven archetype :
mvn archetype:generate \
-DarchetypeGroupId=org.apache.beam \
-DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples \
-DarchetypeVersion=2.47.0 \
-DgroupId=org.example \
-DartifactId=word-count-beam \
-Dversion="0.1" \
-Dpackage=org.apache.beam.examples \
-DinteractiveMode=false
It generate the project with the following dependencies in the pom.xml
file :
<properties>
<beam.version>2.47.0</beam.version>
<slf4j.version>1.7.30</slf4j.version>
</properties>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-jdk14</artifactId>
<version>${slf4j.version}</version>
<!-- When loaded at runtime this will wire up slf4j to the JUL backend -->
<scope>runtime</scope>
</dependency>
Then in the code, you use SL4J
as usual in Java projects :
public class TeamLeagueApp {
private static final Logger LOGGER = LoggerFactory.getLogger(TeamLeagueApp.class);
....
LOGGER.info("My log");
I recommend you using the dependencies and version brought by Beam
natively to handle logs, in order to prevent some dependencies conflicts and issues.
答案2
得分: 0
所以解决方案很简单。看起来BEAM使用slf4j v1库,所以这些依赖项与之配合良好,并使用log4j2作为日志的后端:
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>${log4j.version}</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>${log4j.version}</version>
</dependency>
英文:
So the solution is easy. Looks like BEAM uses slf4j v1 libs so these dependencies work fine with it and use log4j2 as a backend for logging:
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>${log4j.version}</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>${log4j.version}</version>
</dependency>
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论