英文:
Promtail - timestamp parsing (not matching log timestamp and grafana ts)
问题
我一直在努力找到正确的格式来处理 promtail 配置中的时间戳。
我的日志格式如下：
2022-08-02 16:46:02.141 内容
我的 promtail 配置如下：
pipeline_stages:
- multiline:
firstline: '^\d{4}-\d{2}-\d{2}'
max_wait_time: 10s
- regex:
expression: '(?P<time>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3})(.*)$'
- labels:
time:
- timestamp:
source: time
format: 2006-01-02 15:04:05,999
使用这个配置，我在 Grafana 中仍然得到了一个错误的 "ts" 字段（该值与日志中的值不匹配）。
我不知道格式是否正确，我尝试了很多组合，但没有达到预期的效果。在 Go Playground 中，该值被解析为 "2022-08-02 16:46:02.141 +0000 UTC"，没有出现错误 - 这个值是正确的吗？如何使 Grafana 中的 "ts" 值与日志中的值相同？
昨天我尝试了调试模式，在 promtail 日志中出现了以下错误：
level=error ts=2022-08-03T12:37:14.977444831Z caller=client.go:380 component=client host=localhost:3100 msg="final error sending batch" status=400 error="server returned HTTP status 400 Bad Request (400): entry for stream '{filename=\"/opt/loki/my.log\", job=\"varlogs\", stream=\"stdout\", time=\"2022-08-03 14:38:49,150\"}' has timestamp too new: 2022-08-03T14:38:49Z"
我以为 loki 可以处理这个问题（unordered_writes 默认为 true），这是我的格式问题还是 loki 配置的问题？
非常感谢任何帮助。
英文:
I've been struggling to get correct format for handling timestamp in promtail config.
Format of my log:
2022-08-02 16:46:02.141 content
My promtail config:
pipeline_stages:
- multiline:
firstline: '^\d{4}-\d{2}-\d{2}'
max_wait_time: 10s
- regex:
expression: '(?P<time>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3})(.*)$'
- labels:
time:
- timestamp:
source: time
format: 2006-01-02 15:04:05,999
With this conf I'm still getting "ts" field in Grafana with incorrect value (the value doesn't match with the value from log).
I don't know if the format is correct, I tried many combinations but nothing worked as expected. In go playground the value was parsed as "2022-08-02 16:46:02.141 +0000 UTC" without errors - it it ok? How to make the "ts" value in grafana same as the value in log?
I tried debug mode yesterday and in promtail log was error like:
level=error ts=2022-08-03T12:37:14.977444831Z caller=client.go:380 component=client host=localhost:3100 msg="final error sending batch" status=400 error="server returned HTTP status 400 Bad Request (400): entry for stream '{filename=\"/opt/loki/my.log\", job=\"varlogs\", stream=\"stdout\", time=\"2022-08-03 14:38:49,150\"}' has timestamp too new: 2022-08-03T14:38:49Z"
I thought that loki can handle this (unordered_writes as true by default), is this problem of my format or loki conf?
Any help would be greatly appreciated.
答案1
得分: 2
我明白了，这个链接 https://github.com/grafana/loki/issues/2639 对我帮助很大。关键在于时间戳阶段的位置。
- timestamp:
source: time
format: 2006-01-02 15:04:05,999
location: "Europe/Berlin"
英文:
I figured it out, this https://github.com/grafana/loki/issues/2639 helped me a lot. It was all about location in timestamp stage.
- timestamp:
source: time
format: 2006-01-02 15:04:05,999
location: "Europe/Berlin"
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论