英文:
How to configure Grafana Agent?
问题
我想使用Grafana Agent将指标发送到Prometheus。我在静态模式下安装了Grafana Agent(版本0.34),启动了它,并进行了配置,但是Prometheus(版本2.24.0)没有接收到来自Grafana Agent的任何指标。
配置
我使用了示例配置,请参见集成。我只将localhost
替换为运行Prometheus的机器的IP地址。
metrics:
wal_directory: /tmp/wal
global:
remote_write:
- url: http://172.31.11.242:9009/api/v1/prom/push
integrations:
agent:
enabled: true
日志
...
Jul 13 11:51:37 prometheus grafana-agent[28095]: ts=2023-07-13T11:51:37.559384828Z caller=dedupe.go:112 agent=prometheus instance=b9394761ac53f9217ed8581af212bcd2 component=remote level=inf
o remote_name=b93947-21c28b url=http://172.31.11.242:9009/api/v1/prom/push msg="Done replaying WAL" duration=17.612896148s
Jul 13 11:51:39 prometheus grafana-agent[28095]: ts=2023-07-13T11:51:39.981665238Z caller=dedupe.go:112 agent=prometheus instance=b9394761ac53f9217ed8581af212bcd2 component=remote level=war
n remote_name=b93947-21c28b url=http://172.31.11.242:9009/api/v1/prom/push msg="Failed to send batch, retrying" err="Post \"http://172.31.11.242:9009/api/v1/prom/push\": dial tcp 172.31.11.242:90
09: connect: connection refused"
...
研究
- 我检查了运行Prometheus的机器上的端口,没有打开的
9009
端口:
$ sudo netstat -tulpn | grep 9009
-
我阅读了启动Prometheus:
Prometheus应该启动起来。您还应该能够在http://localhost:9090上浏览到有关自身的状态页面。
因此,Prometheus的默认端口是
9090
。我将9009
替换为9090
,但仍然没有收到任何指标。 -
我阅读了HTTP API:
当前稳定的HTTP API 可以在Prometheus服务器上的/api/v1下访问。
因此,API位于
/api/v1
,而不是/api/
。我将/api/prom/push
替换为/api/v1/prom/push
,但仍然没有收到任何指标。
问题
如何配置Grafana Agent将指标发送到Prometheus?
英文:
I want to use Grafana Agent to send metrics to Prometheus. I installed Grafana Agent (version 0.34) in static mode, started it and configured it, but Prometheus (version 2.24.0) doesn't receive any metrics from Grafana Agent.
Configuration
I use the example configuration, see Integrations. I only replaced localhost
with the IP address of the machine running Prometheus.
metrics:
wal_directory: /tmp/wal
global:
remote_write:
- url: http://172.31.11.242:9009/api/prom/push
integrations:
agent:
enabled: true
Logs
Jul 13 11:51:19 prometheus systemd[1]: Started Monitoring system and forwarder.
Jul 13 11:51:19 prometheus grafana-agent[28095]: ts=2023-07-13T11:51:19.934501653Z caller=server.go:191 level=info msg="server listening on addresses" http=127.0.0.1:9090 grpc=127.0.0.1:909
1 http_tls_enabled=false grpc_tls_enabled=false
Jul 13 11:51:19 prometheus grafana-agent[28095]: ts=2023-07-13T11:51:19.935000668Z caller=node.go:85 level=info agent=prometheus component=cluster msg="applying config"
Jul 13 11:51:19 prometheus grafana-agent[28095]: ts=2023-07-13T11:51:19.935158879Z caller=remote.go:180 level=info agent=prometheus component=cluster msg="not watching the KV, none set"
Jul 13 11:51:19 prometheus grafana-agent[28095]: ts=2023-07-13T11:51:19.936021874Z caller=zapadapter.go:78 level=info component=traces msg="Traces Logger Initialized"
Jul 13 11:51:19 prometheus grafana-agent[28095]: ts=2023-07-13T11:51:19.939769201Z caller=reporter.go:103 level=info msg="running usage stats reporter"
Jul 13 11:51:19 prometheus grafana-agent[28095]: ts=2023-07-13T11:51:19.942906407Z caller=wal.go:211 level=info agent=prometheus instance=b9394761ac53f9217ed8581af212bcd2 msg="replaying WAL
, this may take a while" dir=/tmp/wal/b9394761ac53f9217ed8581af212bcd2/wal
Jul 13 11:51:19 prometheus grafana-agent[28095]: ts=2023-07-13T11:51:19.943101908Z caller=wal.go:260 level=info agent=prometheus instance=b9394761ac53f9217ed8581af212bcd2 msg="WAL segment l
oaded" segment=0 maxSegment=2
Jul 13 11:51:19 prometheus grafana-agent[28095]: ts=2023-07-13T11:51:19.945060553Z caller=wal.go:260 level=info agent=prometheus instance=b9394761ac53f9217ed8581af212bcd2 msg="WAL segment l
oaded" segment=1 maxSegment=2
Jul 13 11:51:19 prometheus grafana-agent[28095]: ts=2023-07-13T11:51:19.945229786Z caller=wal.go:260 level=info agent=prometheus instance=b9394761ac53f9217ed8581af212bcd2 msg="WAL segment l
oaded" segment=2 maxSegment=2
Jul 13 11:51:19 prometheus grafana-agent[28095]: ts=2023-07-13T11:51:19.946371012Z caller=dedupe.go:112 agent=prometheus instance=b9394761ac53f9217ed8581af212bcd2 component=remote level=inf
o remote_name=b93947-21c28b url=http://172.31.11.242:9009/api/prom/push msg="Starting WAL watcher" queue=b93947-21c28b
Jul 13 11:51:19 prometheus grafana-agent[28095]: ts=2023-07-13T11:51:19.946418916Z caller=dedupe.go:112 agent=prometheus instance=b9394761ac53f9217ed8581af212bcd2 component=remote level=inf
o remote_name=b93947-21c28b url=http://172.31.11.242:9009/api/prom/push msg="Starting scraped metadata watcher"
Jul 13 11:51:19 prometheus grafana-agent[28095]: ts=2023-07-13T11:51:19.946532386Z caller=dedupe.go:112 agent=prometheus instance=b9394761ac53f9217ed8581af212bcd2 component=remote level=inf
o remote_name=b93947-21c28b url=http://172.31.11.242:9009/api/prom/push msg="Replaying WAL" queue=b93947-21c28b
Jul 13 11:51:37 prometheus grafana-agent[28095]: ts=2023-07-13T11:51:37.559384828Z caller=dedupe.go:112 agent=prometheus instance=b9394761ac53f9217ed8581af212bcd2 component=remote level=inf
o remote_name=b93947-21c28b url=http://172.31.11.242:9009/api/prom/push msg="Done replaying WAL" duration=17.612896148s
Jul 13 11:51:39 prometheus grafana-agent[28095]: ts=2023-07-13T11:51:39.981665238Z caller=dedupe.go:112 agent=prometheus instance=b9394761ac53f9217ed8581af212bcd2 component=remote level=war
n remote_name=b93947-21c28b url=http://172.31.11.242:9009/api/prom/push msg="Failed to send batch, retrying" err="Post \"http://172.31.11.242:9009/api/prom/push\": dial tcp 172.31.11.242:90
09: connect: connection refused"
Research
-
I checked the port on the machine running Prometheus and there is no open port
9009
:$ sudo netstat -tulpn | grep 9009
-
I read Starting Prometheus:
> Prometheus should start up. You should also be able to browse to a status page about itself at http://localhost:9090.So the default port of Prometheus is
9090
. I replaced9009
with9090
, but I still get no metrics. -
I read HTTP API:
> The current stable HTTP API is reachable under /api/v1 on a Prometheus server.So the API is at
/api/v1
, not at/api/
. I replaced/api/prom/push
with/api/v1/prom/push
, but I still get no metrics.
Question
How to configure Grafana Agent to send metrics to Prometheus?
答案1
得分: 1
> Prometheus可以配置为Prometheus远程写入协议的接收器。这不被视为摄取样本的有效方式。请小心在特定的低容量用例中使用它。它不适用于替代通过抓取进行摄取并将Prometheus变成基于推送的指标收集系统。
> 通过设置--web.enable-remote-write-receiver
来启用远程写入接收器。启用后,远程写入接收器的端点是/api/v1/write
。在此处找到更多详细信息。
> 在v2.33中新增
因此,我不得不升级Prometheus并更改其服务配置。还必须更改Grafana Agent的配置中的URL。
Prometheus的服务配置
[Unit]
Description=Prometheus
Documentation=https://prometheus.io/docs/introduction/overview/
Wants=network-online.target
After=network-online.target
[Service]
Type=simple
User=prometheus
Group=prometheus
ExecReload=/bin/kill -HUP $MAINPID
ExecStart=/usr/local/bin/prometheus \
--config.file=/etc/prometheus/prometheus.yml \
--storage.tsdb.path=/var/lib/prometheus \
--web.console.templates=/opt/prometheus/consoles \
--web.console.libraries=/opt/prometheus/console_libraries \
--web.listen-address=0.0.0.0:9090 \
--web.route-prefix=/ \
--web.enable-remote-write-receiver
SyslogIdentifier=prometheus
Restart=always
[Install]
WantedBy=multi-user.target
Grafana Agent的配置
metrics:
wal_directory: /tmp/wal
global:
remote_write:
- url: http://172.31.11.242:9090/api/v1/write
integrations:
agent:
enabled: true
英文:
I read Remote Write Receiver:
> Prometheus can be configured as a receiver for the Prometheus remote write protocol. This is not considered an efficient way of ingesting samples. Use it with caution for specific low-volume use cases. It is not suitable for replacing the ingestion via scraping and turning Prometheus into a push-based metrics collection system.
>
> Enable the remote write receiver by setting --web.enable-remote-write-receiver
. When enabled, the remote write receiver endpoint is /api/v1/write
. Find more details here.
>
> New in v2.33
So I had to upgrade Prometheus and change its service configuration. Also I had to change the URL in Grafana Agent's configuration.
Prometheus' service configuration
[Unit]
Description=Prometheus
Documentation=https://prometheus.io/docs/introduction/overview/
Wants=network-online.target
After=network-online.target
[Service]
Type=simple
User=prometheus
Group=prometheus
ExecReload=/bin/kill -HUP $MAINPID
ExecStart=/usr/local/bin/prometheus \
--config.file=/etc/prometheus/prometheus.yml \
--storage.tsdb.path=/var/lib/prometheus \
--web.console.templates=/opt/prometheus/consoles \
--web.console.libraries=/opt/prometheus/console_libraries \
--web.listen-address=0.0.0.0:9090 \
--web.route-prefix=/ \
--web.enable-remote-write-receiver
SyslogIdentifier=prometheus
Restart=always
[Install]
WantedBy=multi-user.target
Grafana Agent's configuration
metrics:
wal_directory: /tmp/wal
global:
remote_write:
- url: http://172.31.11.242:9090/api/v1/write
integrations:
agent:
enabled: true
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论