英文:
Elasticsearch/Filebeat _meta/fields.yml not loading into kibana
问题
我被分配到我们产品的一个Filebeat插件。我编写了一个IngestPipeline,并且默认映射在各种方式上不幸地,所以我想通过_meta/fields.yml来改变这个。我根据以下配置了yml文件:
https://www.elastic.co/guide/en/beats/devguide/7.8/filebeat-modules-devguide.html#_metafields_yml_2
https://www.elastic.co/guide/en/beats/devguide/current/event-fields-yml.html
https://github.com/elastic/kibana/issues/82273
我的yml文件看起来像这样:
- 键:pac
标题:"pac"
描述:缺少描述
pac模块
字段:- 名称:pac.log
类型:组
描述:缺少描述
字段:- 名称:deo
类型:组
描述:缺少描述
字段:- 名称:duration
类型:长整型
描述:deo的持续时间 - 名称:category
类型:关键词
描述:deo的类别 - 名称:owner
类型:关键词
描述:deo的用户 - 名称:version
类型:浮点数
描述:deo的版本 - 名称:name
类型:关键词
描述:deo的名称 - 名称:reference
类型:双精度
描述:deo的参考编号 - 名称:state
类型:关键词
描述:deo的状态 - 名称:status
类型:关键词
描述:deo的状态 - 名称:trigger
类型:组
描述:缺少描述
字段:- 名称:category
类型:关键词
描述:deo-trigger的类别 - 名称:name
类型:文本
描述:deo-trigger的名称 - 名称:path
类型:文本
描述:触发器属性的全文本 - 名称:provider
类型:关键词
描述:deo-trigger的供应商
- 名称:category
- 名称:wiring
类型:组
描述:缺少描述
字段:- 名称:async
类型:布尔值
描述:deo是否异步连接 - 名称:deoId
类型:关键词
描述:deo的识别号 - 名称:execute
类型:布尔值
描述:deo是否执行 - 名称:owner
类型:关键词
描述:deo的用户 - 名称:shared
类型:布尔值
描述:deo是否共享 - 名称:stopOnError
类型:布尔值
描述:deo是否在错误时停止
- 名称:async
- 名称:duration
- 名称:deo
- 名称:do
类型:组
描述:缺少描述
字段:- 名称:name
类型:关键词
描述:do任务的名称 - 名称:state
类型:组
描述:缺少描述
字段:- 名称:from
类型:关键词
描述:do任务使用的状态 - 名称:to
类型:关键词
描述:do任务切换到的状态
- 名称:from
- 名称:name
- 名称:esa
类型:组
描述:缺少描述
字段:- 名称:connection
类型:关键词
描述:ESA的连接状态 - 名称:name
类型:关键词
描述:ESA的名称 - 名称:state
类型:组
描述:缺少描述
字段:- 名称:from
类型:关键词
描述:建立ESA连接的状态 - 名称:to
类型:关键词
描述:ESA连接建立的状态
- 名称:from
- 名称:connection
- 名称:monitor
类型:组
描述:缺少描述
字段:- 名称:heap
类型:组
描述:缺少描述
字段:- 名称:bytes
类型:长整型
描述:堆的已使用字节数 - 名称:pct
类型:浮点数
描述:最大可用字节数的百分比
- 名称:bytes
- 名称:heapgc
类型:组
- 名称:heap
- 名称:pac.log
英文:
I'm assigned to a filebeat plugin of our product. I wrote a IngestPipeline and de default mapping is in various ways unfortunately so i wanted to change this with the _meta/fields.yml.
I configured the yml file accordingly to:
https://www.elastic.co/guide/en/beats/devguide/7.8/filebeat-modules-devguide.html#_metafields_yml_2
https://www.elastic.co/guide/en/beats/devguide/current/event-fields-yml.html
https://github.com/elastic/kibana/issues/82273
My yml file looks like this:
- key: pac
title: "pac"
description: Description missing
pac Module
fields:
- name: pac.log
type: group
description: Description missing
fields:
- name: deo
type: group
description: Description missing
fields:
- name: duration
type: long
description: Duration of the deo
- name: category
type: keyword
description: Category of the deo
- name: owner
type: keyword
description: User of the deo
- name: version
type: float
description: Version of the deo
- name: name
type: keyword
description: Name of the deo
- name: reference
type: double
description: Referencenumber of the deo
- name: state
type: keyword
description: State of the deo
- name: status
type: keyword
description: Status of the deo
- name: trigger
type: group
description: Description missing
fields:
- name: category
type: keyword
description: Category of the deo-trigger
- name: name
type: text
description: Name of the deo-trigger
- name: path
type: text
description: Full-Text of the trigger properties
- name: provider
type: keyword
description: Supplier of the deo-trigger
- name: wiring
type: group
description: Description missing
fields:
- name: async
type: boolean
description: If deos wired asynchronously
- name: deoId
type: keyword
description: Identification number of the deo
- name: execute
type: boolean
description: If deo is executed or not
- name: owner
type: keyword
description: User of the deo
- name: shared
type: boolean
description: Deo was shared
- name: stopOnError
type: boolean
description: If deo stopped on Error
- name: do
type: group
description: Description missing
fields:
- name: name
type: keyword
description: Name of the do task
- name: state
type: group
description: Description missing
fields:
- name: from
type: keyword
description: State from which the do task was used
- name: to
type: keyword
description: State to which the do task was switched
- name: esa
type: group
description: Description missing
fields:
- name: connection
type: keyword
description: Connection status of the ESA
- name: name
type: keyword
description: Name of the ESA
- name: state
type: group
description: Description missing
fields:
- name: from
type: keyword
description: State from which the ESA Connection was established
- name: to
type: keyword
description: State to which the ESA Connection was established
- name: monitor
type: group
description: Description missing
fields:
- name: heap
type: group
description: Description missing
fields:
- name: bytes
type: long
description: Used bytes of the heap
- name: pct
type: float
description: Percentage of the maximum available bytes
- name: heapgc
type: group
description: Description missing
fields:
- name: bytes
type: long
description: Used bytes of the heapgc
- name: pct
type: float
description: Percentage of the maximum available bytes
- name: service
type: group
description: Description missing
fields:
- name: class
type: keyword
description: Class of the service
- name: duration
type: long
description: How long the service call took
- name: name
type: keyword
description: Name of the service call
- name: operation
type: keyword
description: Operation type of the service call
- name: success
type: boolean
description: Was the service call successfull or not
- name: system
type: group
description: Description missing
fields:
- name: category
type: keyword
description: Category of the System
- name: priority
type: keyword
description: Priority of the System
- name: monitor
type: group
description: Description missing
fields:
- name: cpu
type: group
description: Description missing
fields:
- name: pct
type: float
description: Percentage of the cpu usage
- name: wiring
type: text
description: Description missing
- name: meta
type: text
description: Description missing
- name: tags
type: keyword
description: Description missing
- name: timestamp
type: date
description: Description missing
- name: level
type: keyword
description: Description missing
- name: logger
type: keyword
description: Description missing
I would like to have the field types as configured in the fields.yml file. But at first no mapping is available and if i start to send logs the mapping for my fields is created automatically with unfortunate types.
答案1
得分: 0
yaml文件按照上面的描述正常工作。我的问题是,我继承项目的同事没有使用filebeat构建脚本。因此,上传的fields.yml文件没有包含我的字段。我在filebeat.yml配置文件中重新配置了fields.yaml路径,现在它可以工作了。
英文:
The yaml file works as it is in the description above. My Problem was that my collegue from whom i inherited the project didn't use the filebeat build script. So the uploaded fields.yml file wasn't extended by my fields. I reconfigured the fields.yaml path inside the filebeat.yml configuration file and now it works.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论