英文:
Elasticsearch/Filebeat _meta/fields.yml not loading into kibana
问题
我被分配到我们产品的一个Filebeat插件。我编写了一个IngestPipeline,并且默认映射在各种方式上不幸地,所以我想通过_meta/fields.yml来改变这个。我根据以下配置了yml文件:
https://www.elastic.co/guide/en/beats/devguide/7.8/filebeat-modules-devguide.html#_metafields_yml_2
https://www.elastic.co/guide/en/beats/devguide/current/event-fields-yml.html
https://github.com/elastic/kibana/issues/82273
我的yml文件看起来像这样:
- 键:pac
标题:"pac"
描述:缺少描述
pac模块
字段:- 名称:pac.log
类型:组
描述:缺少描述
字段:- 名称:deo
类型:组
描述:缺少描述
字段:- 名称:duration
类型:长整型
描述:deo的持续时间 - 名称:category
类型:关键词
描述:deo的类别 - 名称:owner
类型:关键词
描述:deo的用户 - 名称:version
类型:浮点数
描述:deo的版本 - 名称:name
类型:关键词
描述:deo的名称 - 名称:reference
类型:双精度
描述:deo的参考编号 - 名称:state
类型:关键词
描述:deo的状态 - 名称:status
类型:关键词
描述:deo的状态 - 名称:trigger
类型:组
描述:缺少描述
字段:- 名称:category
类型:关键词
描述:deo-trigger的类别 - 名称:name
类型:文本
描述:deo-trigger的名称 - 名称:path
类型:文本
描述:触发器属性的全文本 - 名称:provider
类型:关键词
描述:deo-trigger的供应商 
 - 名称:category
 - 名称:wiring
类型:组
描述:缺少描述
字段:- 名称:async
类型:布尔值
描述:deo是否异步连接 - 名称:deoId
类型:关键词
描述:deo的识别号 - 名称:execute
类型:布尔值
描述:deo是否执行 - 名称:owner
类型:关键词
描述:deo的用户 - 名称:shared
类型:布尔值
描述:deo是否共享 - 名称:stopOnError
类型:布尔值
描述:deo是否在错误时停止 
 - 名称:async
 
 - 名称:duration
 
 - 名称:deo
 - 名称:do
类型:组
描述:缺少描述
字段:- 名称:name
类型:关键词
描述:do任务的名称 - 名称:state
类型:组
描述:缺少描述
字段:- 名称:from
类型:关键词
描述:do任务使用的状态 - 名称:to
类型:关键词
描述:do任务切换到的状态 
 - 名称:from
 
 - 名称:name
 - 名称:esa
类型:组
描述:缺少描述
字段:- 名称:connection
类型:关键词
描述:ESA的连接状态 - 名称:name
类型:关键词
描述:ESA的名称 - 名称:state
类型:组
描述:缺少描述
字段:- 名称:from
类型:关键词
描述:建立ESA连接的状态 - 名称:to
类型:关键词
描述:ESA连接建立的状态 
 - 名称:from
 
 - 名称:connection
 - 名称:monitor
类型:组
描述:缺少描述
字段:- 名称:heap
类型:组
描述:缺少描述
字段:- 名称:bytes
类型:长整型
描述:堆的已使用字节数 - 名称:pct
类型:浮点数
描述:最大可用字节数的百分比 
 - 名称:bytes
 - 名称:heapgc
类型:组 
 - 名称:heap
 
 - 名称:pac.log
 
英文:
I'm assigned to a filebeat plugin of our product. I wrote a IngestPipeline and de default mapping is in various ways unfortunately so i wanted to change this with the _meta/fields.yml.
I configured the yml file accordingly to:
https://www.elastic.co/guide/en/beats/devguide/7.8/filebeat-modules-devguide.html#_metafields_yml_2
https://www.elastic.co/guide/en/beats/devguide/current/event-fields-yml.html
https://github.com/elastic/kibana/issues/82273
My yml file looks like this:
- key: pac
  title: "pac"
  description: Description missing
    pac Module
  fields:
    - name: pac.log
      type: group
      description: Description missing
      fields:
        - name: deo
          type: group
          description: Description missing
          fields:
            - name: duration
              type: long
              description: Duration of the deo
            - name: category
              type: keyword
              description: Category of the deo
            - name: owner
              type: keyword
              description: User of the deo
            - name: version
              type: float
              description: Version of the deo
            - name: name
              type: keyword
              description: Name of the deo
            - name: reference
              type: double
              description: Referencenumber of the deo
            - name: state
              type: keyword
              description: State of the deo
            - name: status
              type: keyword
              description: Status of the deo
            - name: trigger
              type: group
              description: Description missing
              fields:
                - name: category
                  type: keyword
                  description: Category of the deo-trigger
                - name: name
                  type: text
                  description: Name of the deo-trigger
                - name: path
                  type: text
                  description: Full-Text of the trigger properties
                - name: provider
                  type: keyword
                  description: Supplier of the deo-trigger
            - name: wiring
              type: group
              description: Description missing
              fields:
                - name: async
                  type: boolean
                  description: If deos wired asynchronously
                - name: deoId
                  type: keyword
                  description: Identification number of the deo
                - name: execute
                  type: boolean
                  description: If deo is executed or not
                - name: owner
                  type: keyword
                  description: User of the deo
                - name: shared
                  type: boolean
                  description: Deo was shared
                - name: stopOnError
                  type: boolean
                  description: If deo stopped on Error
        - name: do
          type: group
          description: Description missing
          fields:
            - name: name
              type: keyword
              description: Name of the do task
            - name: state
              type: group
              description: Description missing
              fields:
                - name: from
                  type: keyword
                  description: State from which the do task was used
                - name: to
                  type: keyword
                  description: State to which the do task was switched
        - name: esa
          type: group
          description: Description missing
          fields:
            - name: connection
              type: keyword
              description: Connection status of the ESA
            - name: name
              type: keyword
              description: Name of the ESA
            - name: state
              type: group
              description: Description missing
              fields:
                - name: from
                  type: keyword
                  description: State from which the ESA Connection was established
                - name: to
                  type: keyword
                  description: State to which the ESA Connection was established
        - name: monitor
          type: group
          description: Description missing
          fields:
            - name: heap
              type: group
              description: Description missing
              fields:
                - name: bytes
                  type: long
                  description: Used bytes of the heap
                - name: pct
                  type: float
                  description: Percentage of the maximum available bytes
            - name: heapgc
              type: group
              description: Description missing
              fields:
                - name: bytes
                  type: long
                  description: Used bytes of the heapgc
                - name: pct
                  type: float
                  description: Percentage of the maximum available bytes
        - name: service
          type: group
          description: Description missing
          fields:
            - name: class
              type: keyword
              description: Class of the service
            - name: duration
              type: long
              description: How long the service call took
            - name: name
              type: keyword
              description: Name of the service call
            - name: operation
              type: keyword
              description: Operation type of the service call
            - name: success
              type: boolean
              description: Was the service call successfull or not
        - name: system
          type: group
          description: Description missing
          fields:
            - name: category
              type: keyword
              description: Category of the System
            - name: priority
              type: keyword
              description: Priority of the System
            - name: monitor
              type: group
              description: Description missing
              fields:
                - name: cpu
                  type: group
                  description: Description missing
                  fields:
                    - name: pct
                      type: float
                      description: Percentage of the cpu usage
        - name: wiring
          type: text
          description: Description missing
        - name: meta
          type: text
          description: Description missing
        - name: tags
          type: keyword
          description: Description missing
        - name: timestamp
          type: date
          description: Description missing
        - name: level
          type: keyword
          description: Description missing
        - name: logger
          type: keyword
          description:  Description missing
I would like to have the field types as configured in the fields.yml file. But at first no mapping is available and if i start to send logs the mapping for my fields is created automatically with unfortunate types.
答案1
得分: 0
yaml文件按照上面的描述正常工作。我的问题是,我继承项目的同事没有使用filebeat构建脚本。因此,上传的fields.yml文件没有包含我的字段。我在filebeat.yml配置文件中重新配置了fields.yaml路径,现在它可以工作了。
英文:
The yaml file works as it is in the description above. My Problem was that my collegue from whom i inherited the project didn't use the filebeat build script. So the uploaded fields.yml file wasn't extended by my fields. I reconfigured the fields.yaml path inside the filebeat.yml configuration file and now it works.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。


评论