如何创建 ksql_processing_log 并将错误发送到 Kafka 主题。

huangapple go评论52阅读模式
英文:

How to make ksql_processing_log and send error to kafka topic

问题

I update log4j.properties in ksqldb and make auto create topic and stream in ksql.properties, but the error does not show in topic ksql_processing_log but shows in file ksql.log why?

and this my log4j.properties


log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=[%d] %p %m (%c)%n

# appenders
log4j.appender.ksqlAppender=org.apache.log4j.RollingFileAppender
log4j.appender.ksqlAppender.File={{ksql.appender_log_path}}{{ksql.appender_log_name}}
log4j.appender.ksqlAppender.layout=org.apache.log4j.PatternLayout
log4j.appender.ksqlAppender.layout.ConversionPattern=[%d] %p %m (%c:%L)%n
log4j.appender.ksqlAppender.MaxFileSize={{ksql.appender_log_file_size}}
log4j.appender.ksqlAppender.MaxBackupIndex={{ksql.appender_max_log_files}}
log4j.appender.ksqlAppender.append=true

log4j.appender.streamsAppender=org.apache.log4j.RollingFileAppender
log4j.appender.streamsAppender.File={{ksql.appender_log_path}}{{ksql.streams_appender_log_name}}
log4j.appender.streamsAppender.layout=org.apache.log4j.PatternLayout
log4j.appender.streamsAppender.layout.ConversionPattern=[%d] %p %m (%c:%L)%n
log4j.appender.streamsAppender.MaxFileSize={{ksql.streams_appender_log_file_size}}
log4j.appender.streamsAppender.MaxBackupIndex={{ksql.streams_appender_max_log_files}}
log4j.appender.streamsAppender.append=true

log4j.appender.kafkaAppender=org.apache.log4j.RollingFileAppender
log4j.appender.kafkaAppender.File={{ksql.appender_log_path}}{{ksql.kafka_appender_log_name}}
log4j.appender.kafkaAppender.layout=org.apache.log4j.PatternLayout
log4j.appender.kafkaAppender.layout.ConversionPattern=[%d] %p %m (%c:%L)%n
log4j.appender.kafkaAppender.MaxFileSize={{ksql.kafka_appender_log_file_size}}
log4j.appender.kafkaAppender.MaxBackupIndex={{ksql.kafka_appender_max_log_files}}
log4j.appender.kafkaAppender.append=true

log4j.appender.kafka_appender=org.apache.kafka.log4jappender.KafkaLog4jAppender
log4j.appender.kafka_appender.layout=io.confluent.common.logging.log4j.StructuredJsonLayout
log4j.appender.kafka_appender.BrokerList=localhost:9092
log4j.appender.kafka_appender.Topic=default_ksql_processing_log
log4j.logger.processing=ERROR, kafka_appender

# loggers
log4j.logger.org.apache.kafka.streams=INFO, streamsAppender
log4j.additivity.org.apache.kafka.streams=false

log4j.logger.kafka=ERROR, kafkaAppender
log4j.additivity.kafka=false

log4j.logger.org.apache.zookeeper=ERROR, kafkaAppender
log4j.additivity.org.apache.zookeeper=false

log4j.logger.org.apache.kafka=ERROR, kafkaAppender
log4j.additivity.org.apache.kafka=false

log4j.logger.org.I0Itec.zkclient=ERROR, kafkaAppender
log4j.additivity.org.I0Itec.zkclient=false

log4j.logger.processing=ERROR, kafka_appender
log4j.additivity.processing=false ```

i dont know why log error does not produce to kafka topics

<details>
<summary>英文:</summary>

I update log4j.properties in ksqldb and make auto create topic and stream in ksql.properties, but the error does not show in topic ksql_processing_log but shows in file ksql.log why?

and this my log4j.properties

log4j.rootLogger=INFO, stdout, ksqlAppender

log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=[%d] %p %m (%c)%n

appenders

log4j.appender.ksqlAppender=org.apache.log4j.RollingFileAppender
log4j.appender.ksqlAppender.File={{ksql.appender_log_path}}{{ksql.appender_log_name}}
log4j.appender.ksqlAppender.layout=org.apache.log4j.PatternLayout
log4j.appender.ksqlAppender.layout.ConversionPattern=[%d] %p %m (%c:%L)%n
log4j.appender.ksqlAppender.MaxFileSize={{ksql.appender_log_file_size}}
log4j.appender.ksqlAppender.MaxBackupIndex={{ksql.appender_max_log_files}}
log4j.appender.ksqlAppender.append=true

log4j.appender.streamsAppender=org.apache.log4j.RollingFileAppender
log4j.appender.streamsAppender.File={{ksql.appender_log_path}}{{ksql.streams_appender_log_name}}
log4j.appender.streamsAppender.layout=org.apache.log4j.PatternLayout
log4j.appender.streamsAppender.layout.ConversionPattern=[%d] %p %m (%c:%L)%n
log4j.appender.streamsAppender.MaxFileSize={{ksql.streams_appender_log_file_size}}
log4j.appender.streamsAppender.MaxBackupIndex={{ksql.streams_appender_max_log_files}}
log4j.appender.streamsAppender.append=true

log4j.appender.kafkaAppender=org.apache.log4j.RollingFileAppender
log4j.appender.kafkaAppender.File={{ksql.appender_log_path}}{{ksql.kafka_appender_log_name}}
log4j.appender.kafkaAppender.layout=org.apache.log4j.PatternLayout
log4j.appender.kafkaAppender.layout.ConversionPattern=[%d] %p %m (%c:%L)%n
log4j.appender.kafkaAppender.MaxFileSize={{ksql.kafka_appender_log_file_size}}
log4j.appender.kafkaAppender.MaxBackupIndex={{ksql.kafka_appender_max_log_files}}
log4j.appender.kafkaAppender.append=true

log4j.appender.kafka_appender=org.apache.kafka.log4jappender.KafkaLog4jAppender
log4j.appender.kafka_appender.layout=io.confluent.common.logging.log4j.StructuredJsonLayout
log4j.appender.kafka_appender.BrokerList=localhost:9092
log4j.appender.kafka_appender.Topic=default_ksql_processing_log
log4j.logger.processing=ERROR, kafka_appender

loggers

log4j.logger.org.apache.kafka.streams=INFO, streamsAppender
log4j.additivity.org.apache.kafka.streams=false

log4j.logger.kafka=ERROR, kafkaAppender
log4j.additivity.kafka=false

log4j.logger.org.apache.zookeeper=ERROR, kafkaAppender
log4j.additivity.org.apache.zookeeper=false

log4j.logger.org.apache.kafka=ERROR, kafkaAppender
log4j.additivity.org.apache.kafka=false

log4j.logger.org.I0Itec.zkclient=ERROR, kafkaAppender
log4j.additivity.org.I0Itec.zkclient=false

log4j.logger.processing=ERROR, kafka_appender
log4j.additivity.processing=false


i dont know why log error does not produce to kafka topics

</details>


# 答案1
**得分**: 0

你需要将 `kafka_appender` 添加到 `log4j.rootLogger` 行中,如果你想默认情况下将其用于所有事件。

否则,你需要使用正确的 KSQL 类包名称,而不是 `processing`。

例如,

log4j.logger.io.confluent.ksql=ERROR, kafka_appender
log4j.additivity.io.confluent.ksql=false


<details>
<summary>英文:</summary>

You need to add `kafka_appender` to the `log4j.rootLogger` line, if you want it to be used, by default, for all events. 

Otherwise, you need to use a correct KSQL class package name instead of `processing`

For example, 

log4j.logger.io.confluent.ksql=ERROR, kafka_appender
log4j.additivity.io.confluent.ksql=false


</details>



huangapple
  • 本文由 发表于 2023年3月31日 17:11:36
  • 转载请务必保留本文链接:https://go.coder-hub.com/75896726.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定