如何在Dropwizard中将HTTP请求记录到文件中？

huangapple
go评论102阅读模式
英文:

How to log HTTP requests in file in Dropwizard?

问题

I'm having issue in my Dropwizard (version 1.3.8) app logging HTTP Requests into file. I've followed Dropwizard's documentation for configuration of YAML file and my config.yml looks like this:

  1. logging:
  2.   level: INFO
  3.   loggers:
  4.     com.nikolas.master_thesis: DEBUG
  6.   appenders:
  7.     - type: file
  8.       currentLogFilename: /home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG.log
  9.       threshold: ALL
  10.       queueSize: 512
  11.       archive: true
  12.       archivedLogFilenamePattern: /home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG-%d.log
  13.       archivedFileCount: 5
  14.       timeZone: UTC
  15.       logFormat: "%-5p [%d{ISO8601,UTC}] %c: %m%n%rEx"
  16.       bufferSize: 8KB
  17.       immediateFlush: true
  18.     
  19.     - type: console
  20.       threshold: ALL
  21.       queueSize: 512
  22.       discardingThreshold: 0
  23.       timeZone: UTC
  24.       target: stdout
  25.       logFormat: "%-5p [%d{ISO8601,UTC}] %c: %m%n%rEx"

The thing is that HTTP Request is logged in console like this:

  1. 192.168.1.5 - - [09/Aug/2020:10:08:05 +0000] "GET /api/orders HTTP/1.1" 200 14094 "-" "Apache-HttpClient/4.5.10 (Java/1.8.0_161)" 710

...and I want to log it also in file, but it does NOT log it at all! I've tried to use commented logFormats, but I get error for each of those commented logFormat values.
E.g. for logFormat: "%h %l %u %t \"%r\" %>s %b" when I run it in terminal (I'm using RMI parameters to collect JMX samples from remote machine):

  1. java -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.port=1616 -Dcom.sun.management.jmxremote.rmi.port=1616  -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -jar ./target/DW-Bookshop-1.0-SNAPSHOT.jar server config.yml

...I get NumberFormatException:

  1. java.lang.NumberFormatException: For input string: ">"
  2.     	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
  3.     	at java.lang.Integer.parseInt(Integer.java:580)
  4.     	at java.lang.Integer.parseInt(Integer.java:615)
  5.     	at ch.qos.logback.core.pattern.FormatInfo.valueOf(FormatInfo.java:73)
  6.     	at ch.qos.logback.core.pattern.parser.Parser.T(Parser.java:136)
  7.     	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:96)
  8.     	...

I've also looked into this answer, but no luck! Also, I'm running Fedora 31 OS and I'm using Java 8.

Any advice/help is greatly appreciated.

UPDATE1:
I've dug a bit deeper into Configuration Docs and found pattern under this link and edited value under logFormat key to be like this:

  1. logFormat:  "%h %l %u [%t] '%r' %s %b '%i{Referer}' '%i{User-Agent}'"

Also found out that Logback project uses PatternLayout class and got this output (as starting lines and other lines are as usual) in terminal:

  1. 09:27:33,944 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
  2. 09:27:33,944 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
  3. 09:27:33,944 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.xml]
  4. 09:27:33,966 |-INFO in ch.qos.logback.classic.BasicConfigurator@316a598d - Setting up default configuration.
  5. 09:27:36,978 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@216914 - Propagating INFO level on Logger[ROOT] onto the JUL framework
  6. 09:27:37,008 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@216914 - Propagating DEBUG level on Logger[com.nikolas.master_thesis] onto the JUL framework
  7. 09:27:37,010 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[file-appender] - Setting bufferSize to [8 KB]
  8. 09:27:37,027 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1805845895 - No compression will be used
  9. 09:27:37,028 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1805845895 - Will use the pattern /home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG-%d.log for the active file
  10. 09:27:37,050 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - The date pattern is 'yyyy-MM-dd' from file name pattern '/home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG-%d.log'.
  11. 09:27:37,050 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Roll-over at midnight.
  12. 09:27:37,068 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Setting initial period to Thu Aug 13 09:27:06 CEST 2020
  13. 09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [h]
  14. 09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [h] is not a valid conversion word
  15. 09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [l]
  16. 09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@
  18. <details>
  19. <summary>英文:</summary>
  21. I&#39;m having issue in my Dropwizard (**version 1.3.8**) app logging HTTP Requests into file. I&#39;ve followed Dropwizard&#39;s [documentation](https://www.dropwizard.io/en/release-1.3.x/manual/configuration.html#file) for configuration of YAML file and my `config.yml` looks like this:
  23. ```yaml
  24. logging:
  25.   level: INFO
  26.   loggers:
  27.     com.nikolas.master_thesis: DEBUG
  29.   appenders:
  30.     - type: file
  31.       currentLogFilename: /home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG.log
  32.       threshold: ALL
  33.       queueSize: 512
  34.       archive: true
  35.       archivedLogFilenamePattern: /home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG-%d.log
  36.       archivedFileCount: 5
  37.       timeZone: UTC
  38.       logFormat: &quot;%-5p [%d{ISO8601,UTC}] %c: %m%n%rEx&quot;
  39.       # logFormat: &quot;%h %l %u %t \&quot;%r\&quot; %&gt;s %b \&quot;%{Referer}i\&quot; \&quot;%{User-agent}i\&quot;&quot; combined
  40. #      logFormat: “%h %l %u %t “%r“ %s %b “%{Referer}i“ “%{User-Agent}i““ combined
  41. #      logFormat: &#39;%h %l %u %t &#39;%r&#39; %\\&gt;s %b &#39;%{Referer}i&#39; &#39;%{User-Agent}i&#39;&#39;
  42. #      logFormat: &quot;%d{HH:mm:ss.SSS} [%thread] %-5level %com.nikolas.master_thesis.examples.`config-kt`.logger{36} - %msg%n&quot;
  43. #      logFormat: “%h %l %u %t “%r“ %s %b “%{Referer}i“ “%{User-Agent}i““
  44. #      logFormat: &quot;%h %l %u %t \&quot;%r\&quot; %s %O \&quot;%{Referer}i\&quot;&quot;
  45. #      logFormat: “%h %l %u %t “%r“ %\\&gt;s %b “%{Referer}i“ “%{User-Agent}i““
  46. #      logFormat: &quot;%h %l %u %t \&quot;%r\&quot; \&quot;%&gt;s\&quot; %O \&quot;%{Referer}i\&quot;&quot;
  47. #      logFormat: &quot;%h %l %u %t \&quot;%r\&quot; %&gt;s %b \&quot;%{Referer}i\&quot; \&quot;%{User-Agent}i\&quot;&quot; combined
  48. #      logFormat: combined
  49. #      logFormat: &quot;%h %l %u %t \&quot;%r\&quot; %&gt;s %b&quot; /etc/httpd
  50. #      logFormat: “%h %l %u %t “%r” %s %b “%{User-Agent}i””
  51. #      logFormat: &quot;%h %l %u %t \&quot;%r\&quot; %&gt;s %b&quot; /etc/httpd
  52. #      logFormat: &quot;%h %l %u %t \&quot;%r\&quot; %&gt;s %b&quot;
  53. #        logFormat: &quot;%h %l %u %t \&quot;%r\&quot; %&gt;s %b \&quot;%{Referer}i\&quot; \&quot;%{User-agent}i\&quot;&quot;
  54.       bufferSize: 8KB
  55.       immediateFlush: true
  56.       # filterFactories:
  57.       #   - type: URI
  58.     
  59.     - type: console
  60.       threshold: ALL
  61.       queueSize: 512
  62.       discardingThreshold: 0
  63.       timeZone: UTC
  64.       target: stdout
  65.       logFormat: &quot;%-5p [%d{ISO8601,UTC}] %c: %m%n%rEx&quot;

The thing is that HTTP Request is logged in console like this:

  1. 192.168.1.5 - - [09/Aug/2020:10:08:05 +0000] &quot;GET /api/orders HTTP/1.1&quot; 200 14094 &quot;-&quot; &quot;Apache-HttpClient/4.5.10 (Java/1.8.0_161)&quot; 710

...and I want to log it also in file, but it does NOT log it at all! I've tried to use commented logFormats, but I get error for each of those commented logFormat values.
E.g. for logFormat: &quot;%h %l %u %t \&quot;%r\&quot; %&gt;s %b&quot; when I run it in terminal (I'm using RMI parameters to collect JMX samples from remote machine):

java -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.port=1616 -Dcom.sun.management.jmxremote.rmi.port=1616 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -jar ./target/DW-Bookshop-1.0-SNAPSHOT.jar server config.yml

...I get NumberFormatException:

  1. java.lang.NumberFormatException: For input string: &quot;&gt;&quot;
  2. 	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
  3. 	at java.lang.Integer.parseInt(Integer.java:580)
  4. 	at java.lang.Integer.parseInt(Integer.java:615)
  5. 	at ch.qos.logback.core.pattern.FormatInfo.valueOf(FormatInfo.java:73)
  6. 	at ch.qos.logback.core.pattern.parser.Parser.T(Parser.java:136)
  7. 	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:96)
  8. 	at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
  9. 	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
  10. 	at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
  11. 	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
  12. 	at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
  13. 	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
  14. 	at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
  15. 	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
  16. 	at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
  17. 	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
  18. 	at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
  19. 	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
  20. 	at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
  21. 	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
  22. 	at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
  23. 	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
  24. 	at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
  25. 	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
  26. 	at ch.qos.logback.core.pattern.parser.Parser.Eopt(Parser.java:115)
  27. 	at ch.qos.logback.core.pattern.parser.Parser.E(Parser.java:100)
  28. 	at ch.qos.logback.core.pattern.parser.Parser.parse(Parser.java:91)
  29. 	at ch.qos.logback.core.pattern.PatternLayoutBase.start(PatternLayoutBase.java:83)
  30. 	at io.dropwizard.logging.AbstractAppenderFactory.buildLayout(AbstractAppenderFactory.java:243)
  31. 	at io.dropwizard.logging.AbstractOutputStreamAppenderFactory.build(AbstractOutputStreamAppenderFactory.java:25)
  32. 	at io.dropwizard.logging.DefaultLoggingFactory.configure(DefaultLoggingFactory.java:143)
  33. 	at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:83)
  34. 	at io.dropwizard.cli.Cli.run(Cli.java:78)
  35. 	at io.dropwizard.Application.run(Application.java:93)
  36. 	at com.nikolas.master_thesis.DropwizardMasterThesisApplication.main(DropwizardMasterThesisApplication.java:35)

I've also looked into this answer, but no luck! Also, I'm running Fedora 31 OS and I'm using Java 8.

Any advice/help is greatly appreciated.

UPDATE1:<br/>
I've digged bit deeper into Configuration Docs and found pattern under this link and edited value under logFormat key to be like this:

  1. logFormat:  &quot;%h %l %u [%t] &#39;%r&#39; %s %b &#39;%i{Referer}&#39; &#39;%i{User-Agent}&#39;&quot;

Also found out that Logback project usess PatternLayout class and got this output (as starting lines and other lines are as usual) in terminal:

  1. 09:27:33,944 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
  2. 09:27:33,944 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
  3. 09:27:33,944 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.xml]
  4. 09:27:33,966 |-INFO in ch.qos.logback.classic.BasicConfigurator@316a598d - Setting up default configuration.
  5. 09:27:36,978 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@216914 - Propagating INFO level on Logger[ROOT] onto the JUL framework
  6. 09:27:37,008 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@216914 - Propagating DEBUG level on Logger[com.nikolas.master_thesis] onto the JUL framework
  7. 09:27:37,010 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[file-appender] - Setting bufferSize to [8 KB]
  8. 09:27:37,027 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1805845895 - No compression will be used
  9. 09:27:37,028 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1805845895 - Will use the pattern /home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG-%d.log for the active file
  10. 09:27:37,050 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - The date pattern is &#39;yyyy-MM-dd&#39; from file name pattern &#39;/home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG-%d.log&#39;.
  11. 09:27:37,050 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Roll-over at midnight.
  12. 09:27:37,068 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Setting initial period to Thu Aug 13 09:27:06 CEST 2020
  13. 09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [h]
  14. 09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [h] is not a valid conversion word
  15. 09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [l]
  16. 09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [l] is not a valid conversion word
  17. 09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [u]
  18. 09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [u] is not a valid conversion word
  19. 09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word 
    展开收缩
  20. 09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - 
    展开收缩
     is not a valid conversion word
  21. 09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [b]
  22. 09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [b] is not a valid conversion word
  23. 09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [i]
  24. 09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [i] is not a valid conversion word
  25. 09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - There is no conversion class registered for conversion word [i]
  26. 09:27:37,071 |-ERROR in ch.qos.logback.core.pattern.parser.Compiler@35764bef - [i] is not a valid conversion word
  27. 09:27:37,074 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[file-appender] - Active log file name: /home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG.log
  28. 09:27:37,074 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[file-appender] - File property is set to [/home/nikola/Documents/DWApp-LOGs/DW-Bookshop_LOG.log]
  29. 09:27:37,075 |-INFO in ch.qos.logback.classic.AsyncAppender[async-file-appender] - Attaching appender named [file-appender] to AsyncAppender.
  30. 09:27:37,076 |-INFO in ch.qos.logback.classic.AsyncAppender[async-file-appender] - Setting discardingThreshold to 0
  31. 09:27:37,077 |-INFO in ch.qos.logback.classic.AsyncAppender[async-console-appender] - Attaching appender named [console-appender] to AsyncAppender.
  32. 09:27:37,077 |-INFO in ch.qos.logback.classic.AsyncAppender[async-console-appender] - Setting discardingThreshold to 0

Apperantly I need to create logback.xml file as this line states:

  1. 09:27:33,944 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.xml]

Any idea where does that logback.xml file needs to located in MAven project and which value for logFormat should I use then?

答案1

得分: 4

要将HTTP请求记录到文件中，您需要设置server: requestLog:配置。logging配置用于配置应用程序的日志记录。

查看有关该配置的文档以及有关logback-access和logback-access-pattern的引用：
https://www.dropwizard.io/en/latest/manual/configuration.html#request-log

这是我的示例：

  1. ...
  2. logging:
  3.   level: INFO
  4.   appenders:
  5.     - type: file
  6.       currentLogFilename: /logs/api.log
  7.       archivedLogFilenamePattern: /logs/api-%d{yyyy-MM-dd}.log.gz
  8.       archivedFileCount: 7
  9.       queueSize: 2048
  10.   loggers:
  11.     com.mstn: DEBUG
  12. server:
  13.     applicationConnectors:
  14.       - type: http
  15.         port: 8003
  16.     adminConnectors:
  17.       - type: http
  18.         port: 9003
  19.     requestLog:
  20.       appenders:
  21.         - type: file
  22.           currentLogFilename: /logs/api-requests.log
  23.           archivedLogFilenamePattern: /logs/api-requests-%d{yyyy-MM-dd}.log.gz
  24.           archivedFileCount: 7
  25.           queueSize: 2048
  26. database:
  27. ...
英文:

To log http requests in a file you have to set server: requestLog: config. logging config is used for configurate logs from your application.

Look the documentation for that config and logback-access and logback-access-pattern references at:
https://www.dropwizard.io/en/latest/manual/configuration.html#request-log

This is my example:

  1. ...
  2. logging:
  3.   level: INFO
  4.   appenders:
  5.     - type: file
  6.       currentLogFilename: /logs/api.log
  7.       archivedLogFilenamePattern: /logs/api-%d{yyyy-MM-dd}.log.gz
  8.       archivedFileCount: 7
  9.       queueSize: 2048
  10.   loggers:
  11.     com.mstn: DEBUG
  12. server:
  13.     applicationConnectors:
  14.       - type: http
  15.         port: 8003
  16.     adminConnectors:
  17.       - type: http
  18.         port: 9003
  19.     requestLog:
  20.       appenders:
  21.         - type: file
  22.           currentLogFilename: /logs/api-requests.log
  23.           archivedLogFilenamePattern: /logs/api-requests-%d{yyyy-MM-dd}.log.gz
  24.           archivedFileCount: 7
  25.           queueSize: 2048
  26. database:
  27. ...

huangapple
  • 本文由 发表于 2020年8月9日 19:28:32
  • 转载请务必保留本文链接：https://go.coder-hub.com/63325766.html
  • dropwizard
  • java
  • logging
  • yaml
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定