How to pass or parse a resource event source body generated from an argo workflow?

huangapple go评论77阅读模式
英文:

How to pass or parse a resource event source body generated from an argo workflow?

问题

我有一个资源传感器,用于触发一个工作流,该工作流报告了导致事件发生的原始工作流的情况,类似于一个记录器。
在传感器的参数中,我想获取原始工作流的主体,以便从中提取特定的值 - 目前我计划将整个JSON作为输入字符串传递。

问题出现在工作流是非平凡的并且包含可变文件时,例如:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: test-es-workflow-
  labels:
    class: dummy-in
spec:
  entrypoint: whalesay
  arguments:
    parameters:
      - name: message
        value: hello world
  templates:
    - name: whalesay
      inputs:
        parameters:
          - name: message
      container:
        image: docker/whalesay:latest
        command: [cowsay]
        args: ["{{inputs.parameters.message}}"]

当发生这种情况时,触发的工作流如下:

apiVersion: argoproj.io/v1alpha1
kind: Sensor
metadata:
  name: demo-sensor
spec:
  template:
    serviceAccountName: operate-workflow-sa
  dependencies:
    - name: test-dep-sensor
      eventSourceName: demo-wf-submit
      eventName: demo-log
  triggers:
    - template:
        name: argo-workflow
        k8s:
          operation: create
          source:
            resource:
              apiVersion: argoproj.io/v1alpha1
              kind: Workflow
              metadata:
                generateName: demo-log-
              spec:
                entrypoint: demolog
                arguments:
                  parameters:
                    - name: body
                      value: hello world
                templates:
                - name: demolog
                  inputs:
                    parameters:
                      - name: body
                  container:
                    image: demolog:latest
                    imagePullPolicy: "Always"
                    command: [/app/demoapp.py]
                    args: ["-j", "{{inputs.parameters.body}}"]
          parameters:
            - src:
                dependencyName: test-dep-sensor
                dataKey: body
                value: wow! a default value
              dest: spec.arguments.parameters.0.value
      retryStrategy:
        steps: 3

失败并显示消息 Message: invalid spec: templates.demolog: failed to resolve {{inputs.parameters.message}}

如果工作流不包含任何变量(没有用{{}}括起来的内容),则触发的工作流会按预期执行。

我希望找到以下一个或多个解决方案:

  1. 警告变量无法解析并替换为null
  2. 不要解析事件主体中的变量,即将其呈现为纯字符串
  3. 在创建事件时解析变量,以便可以解析主体

环境:
Kubernetes: v1.20.15;
Argo: v3.2.9;
Argo Events: v1.6.3

英文:

I have a resource sensor to trigger a workflow that reports on the original workflow that led to the event - a kind of logger.
In the parameters of the sensor, I want to grab the body of the original workflow in order to extract certain values from it - currently I was planning to simply pass the whole JSON as an input string.

The problem arises when the workflow is non-trivial and contains variable files, e.g.:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: test-es-workflow-
  labels:
    class: dummy-in
spec:
  entrypoint: whalesay
  arguments:
    parameters:
      - name: message
        value: hello world
  templates:
    - name: whalesay
      inputs:
        parameters:
          - name: message
      container:
        image: docker/whalesay:latest
        command: [cowsay]
        args: ["{{inputs.parameters.message}}"]

When this happens the triggered workflow:

apiVersion: argoproj.io/v1alpha1
kind: Sensor
metadata:
  name: demo-sensor
spec:
  template:
    serviceAccountName: operate-workflow-sa
  dependencies:
    - name: test-dep-sensor
      eventSourceName: demo-wf-submit
      eventName: demo-log
  triggers:
    - template:
        name: argo-workflow
        k8s:
          operation: create
          source:
            resource:
              apiVersion: argoproj.io/v1alpha1
              kind: Workflow
              metadata:
                generateName: demo-log-
              spec:
                entrypoint: demolog
                arguments:
                  parameters:
                    - name: body
                      value: hello world
                templates:
                - name: demolog
                  inputs:
                    parameters:
                      - name: body
                  container:
                    image: demolog:latest
                    imagePullPolicy: "Always"
                    command: [/app/demoapp.py]
                    args: ["-j", "{{inputs.parameters.body}}"]
          parameters:
            - src:
                dependencyName: test-dep-sensor
                dataKey: body
                value: wow! a default value
              dest: spec.arguments.parameters.0.value
      retryStrategy:
        steps: 3

fails with the message Message: invalid spec: templates.demolog: failed to resolve {{inputs.parameters.message}}

If the workflow does not contain any variables (nothing enclosed in {{}}), then the triggered workflow executes as anticipated.

I would be happy with figuring out how to get to any or all of the following solutions:

  1. Warn that a variable failed to resolve and substitute null
  2. Do not resolve variables within the body of the event, i.e. render it as a plain string
  3. Resolve the variables at the time of creating the event such that the body may be parsed

Environment:
Kubernetes: v1.20.15;
Argo: v3.2.9;
Argo Events: v1.6.3

答案1

得分: 0

我开发了一种类型为2的答案,即渲染双花括号中的变量。
这种方法可能不够优雅,但似乎可以工作:

将传感器的依赖部分进行修补,用其他字符(我选择了“%%”)替换双花括号,使用jq命令:

  dependencies:
    - name: test-dep-sensor
      eventSourceName: demo-wf-submit
      eventName: demo-log
      transform:
        jq: 'walk( if type=="string" then gsub("{{";"%%") else . end ) | walk( if type=="string" then gsub("}}";"%%") else . end )'

附加信息:

Argo Events传感器中的转换

jq手册

英文:

I developed an answer of type 2, that is to render the variables enclosed in double curly braces.
It's not elegant, but it appears to work:

Patch the dependencies section of the sensor to replace the double curly braces with something else (I chose "%%") using jq

  dependencies:
    - name: test-dep-sensor
      eventSourceName: demo-wf-submit
      eventName: demo-log
      transform:
        jq: 'walk( if type=="string" then gsub("{{";"%%") else . end ) | walk( if type=="string" then gsub("}}";"%%") else . end )'

Additional information:

Argo Events Transformations in sensors

jq manual

huangapple
  • 本文由 发表于 2022年6月9日 15:37:39
  • 转载请务必保留本文链接:https://go.coder-hub.com/72556393.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定