英文:
Decrypting Docker Container Names in Grafana with Loki and Promtail
问题
我的任务是配置Docker日志监控,涉及约70个容器。将所有日志合并到一个面板中在压力测试和监视Docker应用程序时并不理想。我希望能够在面板中选择一个Docker容器应用程序并查看相应的日志。我已经实现了这一点,但问题是容器名称是加密的,我需要解密它们。
我的问题是:如何使用Loki数据源和Promtail解密容器名称?
此外,我注意到Prometheus数据源显示了已解密的容器名称,但Loki数据源没有。以下是我为两者的配置:
另外,我正在像普通应用程序一样运行Loki和Promtail,并配置了systemd守护程序。
Loki:
auth_enabled: false
server:
http_listen_port: 3100
grpc_listen_port: 9096
ingester:
lifecycler:
address: 127.0.0.1
ring:
kvstore:
store: inmemory
replication_factor: 1
final_sleep: 0s
chunk_idle_period: 5m
chunk_retain_period: 30s
schema_config:
configs:
- from: 2021-03-08
store: boltdb
object_store: filesystem
schema: v11
index:
prefix: index_
period: 24h
storage_config:
boltdb:
directory: /tmp/loki/index
Promtail:
server:
http_listen_port: 9080
grpc_listen_port: 0
positions:
filename: /tmp/positions.yaml
clients:
- url: http://localhost:3100/loki/api/v1/push
scrape_configs:
- job_name: cadvisor
static_configs:
- targets: ['localhost:8080/containers/']
labels:
job: cadvisor
__path__: /containers
- job_name: system
static_configs:
- targets:
- localhost
labels:
job: varlogs
__path__: /var/log/*log
- job_name: containers
static_configs:
- targets:
- localhost
labels:
job: containerlogs
__path__: /var/lib/docker/containers/*/*log
pipeline_stages:
- json:
expressions:
output: log
stream: stream
attrs:
- json:
expressions:
tag:
source: attrs
- regex:
expression: (?P<container_name>(?:[^|]*[^|]))
source: tag
- timestamp:
format: RFC3339Nano
source: time
- labels:
# tag:
stream:
container_name:
- output:
source: output
此外,我了解到loki-driver-docker插件,但我尝试运行它后没有任何变化,而且文档很糟糕,只是说运行它,几乎没有关于配置或如何检查它是否正常工作的信息。
所以,我很乐意听取任何想法和建议。论坛是我解决这个问题的最后机会。
英文:
My task is to configure Docker log monitoring for around 70 containers. Consolidating all logs into a single panel is not ideal, especially during stress tests and monitoring Docker applications. I want to be able to select a Docker container application in a panel and see the corresponding logs. I have achieved this, but the problem is that the container names are encrypted, and I need to decrypt them.
My question is: How can I decrypt the container names using the Loki data source and Promtail?
Additionally, I noticed that the Prometheus data source displays decrypted container names, but the Loki data source does not. Here are my configurations for both:
Also, I am running Loki and Promtail like normal application with configured systemd daemon.
Loki:
auth_enabled: false
server:
http_listen_port: 3100
grpc_listen_port: 9096
ingester:
lifecycler:
address: 127.0.0.1
ring:
kvstore:
store: inmemory
replication_factor: 1
final_sleep: 0s
chunk_idle_period: 5m
chunk_retain_period: 30s
schema_config:
configs:
- from: 2021-03-08
store: boltdb
object_store: filesystem
schema: v11
index:
prefix: index_
period: 24h
storage_config:
boltdb:
directory: /tmp/loki/index
Promtail:
server:
http_listen_port: 9080
grpc_listen_port: 0
positions:
filename: /tmp/positions.yaml
clients:
- url: http://localhost:3100/loki/api/v1/push
scrape_configs:
- job_name: cadvisor
static_configs:
- targets: ['localhost:8080/containers/']
labels:
job: cadvisonr
__path__: /containers
- job_name: system
static_configs:
- targets:
- localhost
labels:
job: varlogs
__path__: /var/log/*log
- job_name: containers
static_configs:
- targets:
- localhost
labels:
job: containerlogs
__path__: /var/lib/docker/containers/*/*log
pipeline_stages:
- json:
expressions:
output: log
stream: stream
attrs:
- json:
expressions:
tag:
source: attrs
- regex:
expression: (?P<container_name>(?:[^|]*[^|]))
source: tag
- timestamp:
format: RFC3339Nano
source: time
- labels:
# tag:
stream:
container_name:
- output:
source: output
Also, I know about loki-driver-docker plugin, but I tried to run it and nothing changes for me, and documentation is terrible, it says just run it and almost nothing about configuration or how to check if it is working correct.
So, I would be glad to hear any ideas and suggestions. Forum is the last chance for me to solve this problem.
答案1
得分: 5
它的工作原理如下:
- promtail具有后端
<something>_configs
- 每个后端略有不同
- loki收集日志行
- loki中的每行日志都有“标签”
- promtail后端可以将一些内部信息转换为“标签”并发送到loki
- promtail文件后端只能看到文件,所以
static_config
只能看到文件 - promtail Docker后端连接到Docker守护程序,并且实际上了解这些Docker容器的某些信息
Promtail配置可能如下所示:
- job_name: "docker"
docker_sd_configs:
- host: "unix:///var/run/docker.sock"
refresh_interval: "1s"
relabel_configs:
- source_labels: ['__meta_docker_container_name']
target_label: "container_name"
- source_labels: ['__meta_docker_container_id']
target_label: "container_id"
# 等等。
然后在grafana-loki中,您可以使用LogQL语言过滤日志,例如:
logcli '{container_name="something"}'
英文:
It works like this:
- promtail has backends
<something>_configs
- every backend is slightly different
- loki collects lines of logs
- every line of logs in loki has "labels"
- promtail backends can convert some internal info to "labels" and send that loki
- promtail file backend only sees files, so
static_config
only sees files - promtail docker backend connects to docker deamon and actually knows something about those dockers
Promtail config may look like this:
- job_name: "docker"
docker_sd_configs:
- host: "unix:///var/run/docker.sock"
refresh_interval: "1s"
relabel_configs:
- source_labels: ['__meta_docker_container_name']
target_label: "container_name"
- source_labels: ['__meta_docker_container_id']
target_label: "container_id"
# etc.
Then in grafana-loki you can filter the logs using LogQL language, like:
logcli '{container_name="something"}'
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论