英文:
Why is filebeat creating a new datastream everyday?
问题
一个非常简单的Filebeat配置。
该配置创建以下内容:
- 一个名为:voucher-app-logs-8.7.0-2023.04.19 的数据流
- 一个名为:.ds-voucher-app-logs-8.7.0-2023.04.19-2023.04.19-000001 的索引
- 每天创建一个新的数据流
我的问题:
- 为什么每天都要创建一个新的数据流?
- 如何更改数据流的名称?
- 如何仅为这些索引模式创建一个数据流,而不是每天都创建一个新的数据流?
英文:
A very simple filebeat
filebeat:
inputs:
- type: filestream
id: vouchers-logs-stream
paths:
- /path/to/logs/*.log
json:
keys_under_root: true
add_error_key: true
overwrite_keys: true
message_key: message
parsers:
- ndjson:
target: ""
add_error_key: true
output:
elasticsearch:
hosts: [ "..." ]
username: "..."
password: "..."
index: voucher-app-logs-%{[agent.version]}-%{+yyyy.MM.dd}
setup:
template:
name: "voucher-app-logs"
pattern: "voucher-app-logs*"
overwrite: false
ilm:
enabled: true
policy_name: "voucher-app-logs-lifecycle-policy"
This configuration creates the following:
- A data stream called: voucher-app-logs-8.7.0-2023.04.19
- A index called
.ds-voucher-app-logs-8.7.0-2023.04.19-2023.04.19-000001 - And at each day, a new datastream is created
My questions:
- Why create a new datastream everyday?
- How to change the name of the datastream?
- How to create only one datastream for these index patterns instead of
creating one new ds every day?
I have read the docs and found nothing about that. Perhaps I missed something?
答案1
得分: 1
Tldr;
Beats 正在按照指示执行 ^^
在输出部分:
output:
elasticsearch:
hosts: [ "..." ]
username: "..."
password: "..."
index: voucher-app-logs-%{[agent.version]}-%{+yyyy.MM.dd}
^^^^^^^^^^^
这将每天更改
所以每天 Beat 会寻找尚不存在的数据流。并将创建一个。
Solution
删除数据是您的选择吗?
output:
elasticsearch:
hosts: [ "..." ]
username: "..."
password: "..."
index: voucher-app-logs-%{[agent.version]}
英文:
Tldr;
Beats is doing what he is told to ^^
In the output section:
output:
elasticsearch:
hosts: [ "..." ]
username: "..."
password: "..."
index: voucher-app-logs-%{[agent.version]}-%{+yyyy.MM.dd}
^^^^^^^^^^^
This will change every day
So everyday the beat is going to look for a datastream that does not exist yet. And will create one.
Solution
Is removing the data an option for you ?
output:
elasticsearch:
hosts: [ "..." ]
username: "..."
password: "..."
index: voucher-app-logs-%{[agent.version]}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论