如何将离散事件格式化为与状态时间线兼容的表格?

huangapple go评论71阅读模式
英文:

How to format discrete events into a state-timeline compatible table?

问题

我试图使用Grafana的状态时间线图,为了使其工作,我需要转换我的源数据:

TimeGenerated playbook_name status runtime
11:02 config_backup success 3600
12:33 update_inventory success 52
13:04 config_backup partial_success 3600
14:31 update_inventory success 47

变成这样:(见下文)

我希望任务列能够在“任务开始时间” + “运行时间”在TimeGenerated列内时显示状态。明白我的意思吗?

比如,如果config_backup任务在11:00开始,持续3600秒,那么在11:00到12:00之间的config_backup列中的每一行都应该被标记,最终状态应该是“status”列的状态,但我愿意妥协。

TimeGenerated config_backup update_inventory
11:00 success
11:15 success
11:30 success
11:45 success
12:00
12:15
12:30 success
12:45
13:00 partial_success
13:15 partial_success
13:30 partial_success
以此类推...

这里涉及几个步骤,我已经做了相当多的工作,但现在卡住了,所以我需要帮助。

首先,我们需要将TimeGenerated重新写成15分钟的块。这是我实现的方式:

let _start=ago(1d);
let _end=now();
let interval = 15m;
let timetable = range TimeGenerated from ago(1d) to now() step 15m;
log_ansible_playbook_stats_CL
| union timetable

我也知道如何使用make_series来转置行和列,所以现在我更关注标记剩余运行时间。

这是我想出的:

| extend runtime_s = toint(prev(runtime_s)) - 900

我认为,之后我可以使用条件来扩展,例如“如果运行时间不为空,则状态=某事”。

问题是,这仅适用于一行:

TimeGenerated playbook_name status runtime
11:00 config_backup success 3600
11:15 2700
11:30
11:45

这时我意识到,整个表格一次性被评估,而不是像我希望的一行一行评估。现在,我真的不知道该如何继续了。我查看了fillforward()函数,但它没有条件,所以我无法在任务运行时间结束后停止它。

然而,现在我这么说,我在想我可以在任务开始时发出日志,同时在任务结束时发出日志,然后使用fillforward()来填补两个时间戳之间的间隙。

但也许,我希望我只是错过了明显的解决方案,你们中的一个可以指引我走上正确的道路。

谢谢。

英文:

I'm trying to use Grafana's state timeline graf and for this to work I need to transform my source data:

TimeGenerated playbook_name status runtime
11:02 config_backup success 3600
12:33 update_inventory success 52
13:04 config_backup partial_success 3600
14:31 update_inventory success 47

Into this: (see below)

I want the job column to illustrate the status for as long as "job start time " + "runtime" is within the TimeGenerated column. Does that make sense?

Like, if the config_backup job starts at 11:00 and lasts for 3600 seconds, every row in the config_backup column between 11:00 and 12:00 should be marked, ultimately with the status of the "status" column, but I'm willing to compromise.

TimeGenerated config_backup update_inventory
11:00 success
11:15 success
11:30 success
11:45 success
12:00
12:15
12:30 success
12:45
13:00 partial_success
13:15 partial_success
13:30 partial_success
so on...

There are a few steps involved here and I got fairly far but now I'm getting stuck so I need help.

First, we need to re-write the TimeGenerated into 15 minute blocks. This is how I've achieved it:

let _start=ago(1d);
let _end=now();
let interval = 15m;
let timetable = range TimeGenerated from ago(1d) to now() step 15m;
log_ansible_playbook_stats_CL
| union timetable

I also know how to use make_series to pivot the rows and columns, so for now I was focusing more on marking the status for as long as there is runtime left.

This is what I came up with

| extend runtime_s = toint(prev(runtime_s)) - 900

Thinking that, I can later do an extend case "if runtime is not empty, status = something"

The problem is, this only works for one row:

TimeGenerated playbook_name status runtime
11:00 config_backup success 3600
11:15 2700
11:30
11:45

And that's when I realized, the whole table is evaluated at once, not row-by-row as I was hoping. Now, I really don't know how to proceed. I've looked at the fillforward() function but it doesn't have a conditional so I can't stop it once we're outside the runtime of the job.

However, now that I say this I'm thinking I could emit a log at the START of the job, as well as at the END, then use fillforward() to fill the gap between the two timestamps.

But perhaps, I'm hoping that I simply missed the obvious solution and one of you can set me on the right path.

Thank you,

答案1

得分: 1

你可以使用mv-expand运算符来生成额外的时间条目:

datatable(TimeGenerated:datetime,playbook_name:string,status:string,runtime:int 
)[
datetime(11:02), "config_backup", "success", 3600,
datetime(12:33), "update_inventory", "success", 52,
datetime(13:04), "config_backup", "partial_success", 3600,
datetime(14:31), "update_inventory", "success", 47,
]
| mv-expand TimeGenerated=
    range(bin(TimeGenerated, 15m), TimeGenerated + runtime * 1s, 15m)
| project TimeGenerated, playbook_name, status

使用mv-apply,你还可以生成带有空状态的时间事件:

datatable (
TimeGenerated: datetime,
playbook_name: string,
status: string,
runtime: int
)[
datetime(11:02), "config_backup", "success", 3600,
datetime(12:33), "update_inventory", "success", 52,
datetime(13:04), "config_backup", "partial_success", 3600,
datetime(14:31), "update_inventory", "success", 47,
]
| mv-apply TimeGenerated2= range(datetime(11:00), datetime(15:00), 15m) to typeof(datetime) on (
  extend status= iff(TimeGenerated2 between (bin(TimeGenerated, 15m) .. (TimeGenerated + runtime * 1s)), status, "")
)
| project TimeGenerated= TimeGenerated2, playbook_name, status
| evaluate pivot(playbook_name, take_any(status))
英文:

You can use the mv-expand operator to generate the additional time entries:

datatable(TimeGenerated:datetime,playbook_name:string,status:string,runtime:int 
)[
datetime(11:02), "config_backup", "success", 3600,
datetime(12:33), "update_inventory", "success", 52,
datetime(13:04), "config_backup", "partial_success", 3600,
datetime(14:31), "update_inventory", "success", 47,
]
| mv-expand TimeGenerated=
    range(bin(TimeGenerated, 15m), TimeGenerated + runtime * 1s, 15m)
| project TimeGenerated, playbook_name, status

With mv-apply you can also generate time events with an empty status:

datatable (
    TimeGenerated: datetime,
    playbook_name: string,
    status: string,
    runtime: int
)[
    datetime(11:02), "config_backup", "success", 3600,
    datetime(12:33), "update_inventory", "success", 52,
    datetime(13:04), "config_backup", "partial_success", 3600,
    datetime(14:31), "update_inventory", "success", 47,
]
| mv-apply TimeGenerated2= range(datetime(11:00), datetime(15:00), 15m) to typeof(datetime) on (
  extend status= iff(TimeGenerated2 between (bin(TimeGenerated, 15m) .. (TimeGenerated + runtime * 1s)), status, "")
)
| project TimeGenerated= TimeGenerated2, playbook_name, status
| evaluate pivot(playbook_name, take_any(status))

huangapple
  • 本文由 发表于 2023年8月5日 04:24:33
  • 转载请务必保留本文链接:https://go.coder-hub.com/76838934.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定