如何在Dropwizard中将HTTP请求记录到文件中？

huangapple
go评论68阅读模式
英文:

How to log HTTP requests in file in Dropwizard?

问题

I'm having issue in my Dropwizard (version 1.3.8) app logging HTTP Requests into file. I've followed Dropwizard's documentation for configuration of YAML file and my config.yml looks like this:

logging:
  level: INFO
  loggers:
    com.nikolas.master_thesis: DEBUG

  appenders:
    - type: file
      currentLogFilename: /home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG.log
      threshold: ALL
      queueSize: 512
      archive: true
      archivedLogFilenamePattern: /home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG-%d.log
      archivedFileCount: 5
      timeZone: UTC
      logFormat: "%-5p [%d{ISO8601,UTC}] %c: %m%n%rEx"
      bufferSize: 8KB
      immediateFlush: true
    
    - type: console
      threshold: ALL
      queueSize: 512
      discardingThreshold: 0
      timeZone: UTC
      target: stdout
      logFormat: "%-5p [%d{ISO8601,UTC}] %c: %m%n%rEx"

The thing is that HTTP Request is logged in console like this:

192.168.1.5 - - [09/Aug/2020:10:08:05 +0000] "GET /api/orders HTTP/1.1" 200 14094 "-" "Apache-HttpClient/4.5.10 (Java/1.8.0_161)" 710

...and I want to log it also in file, but it does NOT log it at all! I've tried to use commented logFormats, but I get error for each of those commented logFormat values.
E.g. for logFormat: "%h %l %u %t \"%r\" %>s %b" when I run it in terminal (I'm using RMI parameters to collect JMX samples from remote machine):

java -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.port=1616 -Dcom.sun.management.jmxremote.rmi.port=1616  -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -jar ./target/DW-Bookshop-1.0-SNAPSHOT.jar server config.yml

...I get NumberFormatException:

java.lang.NumberFormatException: For input string: ">"
    	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
    	at java.lang.Integer.parseInt(Integer.java:580)
    	at java.lang.Integer.parseInt(Integer.java:615)
    	at ch.qos.logback.core.pattern.FormatInfo.valueOf(FormatInfo.java:73)
    	at ch.qos.logback.core.pattern.parser.Parser.T(Parser.java:136)
    	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:96)
    	...

I've also looked into this answer, but no luck! Also, I'm running Fedora 31 OS and I'm using Java 8.

Any advice/help is greatly appreciated.

UPDATE1:
I've dug a bit deeper into Configuration Docs and found pattern under this link and edited value under logFormat key to be like this:

logFormat:  "%h %l %u [%t] '%r' %s %b '%i{Referer}' '%i{User-Agent}'"

Also found out that Logback project uses PatternLayout class and got this output (as starting lines and other lines are as usual) in terminal:

09:27:33,944 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
09:27:33,944 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
09:27:33,944 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.xml]
09:27:33,966 |-INFO in ch.qos.logback.classic.BasicConfigurator@316a598d - Setting up default configuration.
09:27:36,978 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@216914 - Propagating INFO level on Logger[ROOT] onto the JUL framework
09:27:37,008 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@216914 - Propagating DEBUG level on Logger[com.nikolas.master_thesis] onto the JUL framework
09:27:37,010 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[file-appender] - Setting bufferSize to [8 KB]
09:27:37,027 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1805845895 - No compression will be used
09:27:37,028 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1805845895 - Will use the pattern /home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG-%d.log for the active file
09:27:37,050 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - The date pattern is 'yyyy-MM-dd' from file name pattern '/home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG-%d.log'.
09:27:37,050 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Roll-over at midnight.
09:27:37,068 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Setting initial period to Thu Aug 13 09:27:06 CEST 2020
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [h]
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [h] is not a valid conversion word
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [l]
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@

<details>
<summary>英文:</summary>

I&#39;m having issue in my Dropwizard (**version 1.3.8**) app logging HTTP Requests into file. I&#39;ve followed Dropwizard&#39;s [documentation](https://www.dropwizard.io/en/release-1.3.x/manual/configuration.html#file) for configuration of YAML file and my `config.yml` looks like this:

```yaml
logging:
  level: INFO
  loggers:
    com.nikolas.master_thesis: DEBUG

  appenders:
    - type: file
      currentLogFilename: /home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG.log
      threshold: ALL
      queueSize: 512
      archive: true
      archivedLogFilenamePattern: /home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG-%d.log
      archivedFileCount: 5
      timeZone: UTC
      logFormat: &quot;%-5p [%d{ISO8601,UTC}] %c: %m%n%rEx&quot;
      # logFormat: &quot;%h %l %u %t \&quot;%r\&quot; %&gt;s %b \&quot;%{Referer}i\&quot; \&quot;%{User-agent}i\&quot;&quot; combined
#      logFormat: “%h %l %u %t “%r“ %s %b “%{Referer}i“ “%{User-Agent}i““ combined
#      logFormat: &#39;%h %l %u %t &#39;%r&#39; %\\&gt;s %b &#39;%{Referer}i&#39; &#39;%{User-Agent}i&#39;&#39;
#      logFormat: &quot;%d{HH:mm:ss.SSS} [%thread] %-5level %com.nikolas.master_thesis.examples.`config-kt`.logger{36} - %msg%n&quot;
#      logFormat: “%h %l %u %t “%r“ %s %b “%{Referer}i“ “%{User-Agent}i““
#      logFormat: &quot;%h %l %u %t \&quot;%r\&quot; %s %O \&quot;%{Referer}i\&quot;&quot;
#      logFormat: “%h %l %u %t “%r“ %\\&gt;s %b “%{Referer}i“ “%{User-Agent}i““
#      logFormat: &quot;%h %l %u %t \&quot;%r\&quot; \&quot;%&gt;s\&quot; %O \&quot;%{Referer}i\&quot;&quot;
#      logFormat: &quot;%h %l %u %t \&quot;%r\&quot; %&gt;s %b \&quot;%{Referer}i\&quot; \&quot;%{User-Agent}i\&quot;&quot; combined
#      logFormat: combined
#      logFormat: &quot;%h %l %u %t \&quot;%r\&quot; %&gt;s %b&quot; /etc/httpd
#      logFormat: “%h %l %u %t “%r” %s %b “%{User-Agent}i””
#      logFormat: &quot;%h %l %u %t \&quot;%r\&quot; %&gt;s %b&quot; /etc/httpd
#      logFormat: &quot;%h %l %u %t \&quot;%r\&quot; %&gt;s %b&quot;
#        logFormat: &quot;%h %l %u %t \&quot;%r\&quot; %&gt;s %b \&quot;%{Referer}i\&quot; \&quot;%{User-agent}i\&quot;&quot;
      bufferSize: 8KB
      immediateFlush: true
      # filterFactories:
      #   - type: URI
    
    - type: console
      threshold: ALL
      queueSize: 512
      discardingThreshold: 0
      timeZone: UTC
      target: stdout
      logFormat: &quot;%-5p [%d{ISO8601,UTC}] %c: %m%n%rEx&quot;

The thing is that HTTP Request is logged in console like this:

192.168.1.5 - - [09/Aug/2020:10:08:05 +0000] &quot;GET /api/orders HTTP/1.1&quot; 200 14094 &quot;-&quot; &quot;Apache-HttpClient/4.5.10 (Java/1.8.0_161)&quot; 710

...and I want to log it also in file, but it does NOT log it at all! I've tried to use commented logFormats, but I get error for each of those commented logFormat values.
E.g. for logFormat: &quot;%h %l %u %t \&quot;%r\&quot; %&gt;s %b&quot; when I run it in terminal (I'm using RMI parameters to collect JMX samples from remote machine):

java -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.port=1616 -Dcom.sun.management.jmxremote.rmi.port=1616 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -jar ./target/DW-Bookshop-1.0-SNAPSHOT.jar server config.yml

...I get NumberFormatException:

java.lang.NumberFormatException: For input string: &quot;&gt;&quot;
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
	at java.lang.Integer.parseInt(Integer.java:580)
	at java.lang.Integer.parseInt(Integer.java:615)
	at ch.qos.logback.core.pattern.FormatInfo.valueOf(FormatInfo.java:73)
	at ch.qos.logback.core.pattern.parser.Parser.T(Parser.java:136)
	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:96)
	at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
	at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
	at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
	at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
	at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
	at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
	at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
	at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
	at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
	at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
	at ch.qos.logback.core.pattern.parser.Parser.parse(Parser.java:91)
	at ch.qos.logback.core.pattern.PatternLayoutBase.start(PatternLayoutBase.java:83)
	at io.dropwizard.logging.AbstractAppenderFactory.buildLayout(AbstractAppenderFactory.java:243)
	at io.dropwizard.logging.AbstractOutputStreamAppenderFactory.build(AbstractOutputStreamAppenderFactory.java:25)
	at io.dropwizard.logging.DefaultLoggingFactory.configure(DefaultLoggingFactory.java:143)
	at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:83)
	at io.dropwizard.cli.Cli.run(Cli.java:78)
	at io.dropwizard.Application.run(Application.java:93)
	at com.nikolas.master_thesis.DropwizardMasterThesisApplication.main(DropwizardMasterThesisApplication.java:35)

I've also looked into this answer, but no luck! Also, I'm running Fedora 31 OS and I'm using Java 8.

Any advice/help is greatly appreciated.

UPDATE1:<br/>
I've digged bit deeper into Configuration Docs and found pattern under this link and edited value under logFormat key to be like this:

logFormat:  &quot;%h %l %u [%t] &#39;%r&#39; %s %b &#39;%i{Referer}&#39; &#39;%i{User-Agent}&#39;&quot;

Also found out that Logback project usess PatternLayout class and got this output (as starting lines and other lines are as usual) in terminal:

09:27:33,944 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
09:27:33,944 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
09:27:33,944 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.xml]
09:27:33,966 |-INFO in ch.qos.logback.classic.BasicConfigurator@316a598d - Setting up default configuration.
09:27:36,978 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@216914 - Propagating INFO level on Logger[ROOT] onto the JUL framework
09:27:37,008 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@216914 - Propagating DEBUG level on Logger[com.nikolas.master_thesis] onto the JUL framework
09:27:37,010 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[file-appender] - Setting bufferSize to [8 KB]
09:27:37,027 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1805845895 - No compression will be used
09:27:37,028 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1805845895 - Will use the pattern /home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG-%d.log for the active file
09:27:37,050 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - The date pattern is &#39;yyyy-MM-dd&#39; from file name pattern &#39;/home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG-%d.log&#39;.
09:27:37,050 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Roll-over at midnight.
09:27:37,068 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Setting initial period to Thu Aug 13 09:27:06 CEST 2020
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [h]
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [h] is not a valid conversion word
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [l]
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [l] is not a valid conversion word
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [u]
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [u] is not a valid conversion word
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word 
展开收缩

09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - 
展开收缩
 is not a valid conversion word
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [b]
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [b] is not a valid conversion word
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [i]
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [i] is not a valid conversion word
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [i]
09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [i] is not a valid conversion word
09:27:37,074 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[file-appender] - Active log file name: /home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG.log
09:27:37,074 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[file-appender] - File property is set to [/home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG.log]
09:27:37,075 |-INFO in ch.qos.logback.classic.AsyncAppender[async-file-appender] - Attaching appender named [file-appender] to AsyncAppender.
09:27:37,076 |-INFO in ch.qos.logback.classic.AsyncAppender[async-file-appender] - Setting discardingThreshold to 0
09:27:37,077 |-INFO in ch.qos.logback.classic.AsyncAppender[async-console-appender] - Attaching appender named [console-appender] to AsyncAppender.
09:27:37,077 |-INFO in ch.qos.logback.classic.AsyncAppender[async-console-appender] - Setting discardingThreshold to 0

Apperantly I need to create logback.xml file as this line states:

09:27:33,944 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.xml]

Any idea where does that logback.xml file needs to located in MAven project and which value for logFormat should I use then?

答案1

得分: 4

要将HTTP请求记录到文件中，您需要设置server: requestLog:配置。logging配置用于配置应用程序的日志记录。

查看有关该配置的文档以及有关logback-access和logback-access-pattern的引用：
https://www.dropwizard.io/en/latest/manual/configuration.html#request-log

这是我的示例：

...
logging:
  level: INFO
  appenders:
    - type: file
      currentLogFilename: /logs/api.log
      archivedLogFilenamePattern: /logs/api-%d{yyyy-MM-dd}.log.gz
      archivedFileCount: 7
      queueSize: 2048
  loggers:
    com.mstn: DEBUG
server:
    applicationConnectors:
      - type: http
        port: 8003
    adminConnectors:
      - type: http
        port: 9003
    requestLog:
      appenders:
        - type: file
          currentLogFilename: /logs/api-requests.log
          archivedLogFilenamePattern: /logs/api-requests-%d{yyyy-MM-dd}.log.gz
          archivedFileCount: 7
          queueSize: 2048
database:
...
英文:

To log http requests in a file you have to set server: requestLog: config. logging config is used for configurate logs from your application.

Look the documentation for that config and logback-access and logback-access-pattern references at:
https://www.dropwizard.io/en/latest/manual/configuration.html#request-log

This is my example:

...
logging:
  level: INFO
  appenders:
    - type: file
      currentLogFilename: /logs/api.log
      archivedLogFilenamePattern: /logs/api-%d{yyyy-MM-dd}.log.gz
      archivedFileCount: 7
      queueSize: 2048
  loggers:
    com.mstn: DEBUG
server:
    applicationConnectors:
      - type: http
        port: 8003
    adminConnectors:
      - type: http
        port: 9003
    requestLog:
      appenders:
        - type: file
          currentLogFilename: /logs/api-requests.log
          archivedLogFilenamePattern: /logs/api-requests-%d{yyyy-MM-dd}.log.gz
          archivedFileCount: 7
          queueSize: 2048
database:
...

huangapple
  • 本文由 发表于 2020年8月9日 19:28:32
  • 转载请务必保留本文链接：https://go.coder-hub.com/63325766.html
  • dropwizard
  • java
  • logging
  • yaml
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定