英文:
How to format discrete events into a state-timeline compatible table?
问题
我试图使用Grafana的状态时间线图,为了使其工作,我需要转换我的源数据:
TimeGenerated | playbook_name | status | runtime |
---|---|---|---|
11:02 | config_backup | success | 3600 |
12:33 | update_inventory | success | 52 |
13:04 | config_backup | partial_success | 3600 |
14:31 | update_inventory | success | 47 |
变成这样:(见下文)
我希望任务列能够在“任务开始时间” + “运行时间”在TimeGenerated列内时显示状态。明白我的意思吗?
比如,如果config_backup任务在11:00开始,持续3600秒,那么在11:00到12:00之间的config_backup列中的每一行都应该被标记,最终状态应该是“status”列的状态,但我愿意妥协。
TimeGenerated | config_backup | update_inventory |
---|---|---|
11:00 | success | |
11:15 | success | |
11:30 | success | |
11:45 | success | |
12:00 | ||
12:15 | ||
12:30 | success | |
12:45 | ||
13:00 | partial_success | |
13:15 | partial_success | |
13:30 | partial_success | |
以此类推... |
这里涉及几个步骤,我已经做了相当多的工作,但现在卡住了,所以我需要帮助。
首先,我们需要将TimeGenerated重新写成15分钟的块。这是我实现的方式:
let _start=ago(1d);
let _end=now();
let interval = 15m;
let timetable = range TimeGenerated from ago(1d) to now() step 15m;
log_ansible_playbook_stats_CL
| union timetable
我也知道如何使用make_series来转置行和列,所以现在我更关注标记剩余运行时间。
这是我想出的:
| extend runtime_s = toint(prev(runtime_s)) - 900
我认为,之后我可以使用条件来扩展,例如“如果运行时间不为空,则状态=某事”。
问题是,这仅适用于一行:
TimeGenerated | playbook_name | status | runtime |
---|---|---|---|
11:00 | config_backup | success | 3600 |
11:15 | 2700 | ||
11:30 | |||
11:45 |
这时我意识到,整个表格一次性被评估,而不是像我希望的一行一行评估。现在,我真的不知道该如何继续了。我查看了fillforward()函数,但它没有条件,所以我无法在任务运行时间结束后停止它。
然而,现在我这么说,我在想我可以在任务开始时发出日志,同时在任务结束时发出日志,然后使用fillforward()来填补两个时间戳之间的间隙。
但也许,我希望我只是错过了明显的解决方案,你们中的一个可以指引我走上正确的道路。
谢谢。
英文:
I'm trying to use Grafana's state timeline graf and for this to work I need to transform my source data:
TimeGenerated | playbook_name | status | runtime |
---|---|---|---|
11:02 | config_backup | success | 3600 |
12:33 | update_inventory | success | 52 |
13:04 | config_backup | partial_success | 3600 |
14:31 | update_inventory | success | 47 |
Into this: (see below)
I want the job column to illustrate the status for as long as "job start time " + "runtime" is within the TimeGenerated column. Does that make sense?
Like, if the config_backup job starts at 11:00 and lasts for 3600 seconds, every row in the config_backup column between 11:00 and 12:00 should be marked, ultimately with the status of the "status" column, but I'm willing to compromise.
TimeGenerated | config_backup | update_inventory |
---|---|---|
11:00 | success | |
11:15 | success | |
11:30 | success | |
11:45 | success | |
12:00 | ||
12:15 | ||
12:30 | success | |
12:45 | ||
13:00 | partial_success | |
13:15 | partial_success | |
13:30 | partial_success | |
so on... |
There are a few steps involved here and I got fairly far but now I'm getting stuck so I need help.
First, we need to re-write the TimeGenerated into 15 minute blocks. This is how I've achieved it:
let _start=ago(1d);
let _end=now();
let interval = 15m;
let timetable = range TimeGenerated from ago(1d) to now() step 15m;
log_ansible_playbook_stats_CL
| union timetable
I also know how to use make_series to pivot the rows and columns, so for now I was focusing more on marking the status for as long as there is runtime left.
This is what I came up with
| extend runtime_s = toint(prev(runtime_s)) - 900
Thinking that, I can later do an extend case "if runtime is not empty, status = something"
The problem is, this only works for one row:
TimeGenerated | playbook_name | status | runtime |
---|---|---|---|
11:00 | config_backup | success | 3600 |
11:15 | 2700 | ||
11:30 | |||
11:45 |
And that's when I realized, the whole table is evaluated at once, not row-by-row as I was hoping. Now, I really don't know how to proceed. I've looked at the fillforward() function but it doesn't have a conditional so I can't stop it once we're outside the runtime of the job.
However, now that I say this I'm thinking I could emit a log at the START of the job, as well as at the END, then use fillforward() to fill the gap between the two timestamps.
But perhaps, I'm hoping that I simply missed the obvious solution and one of you can set me on the right path.
Thank you,
答案1
得分: 1
你可以使用mv-expand运算符来生成额外的时间条目:
datatable(TimeGenerated:datetime,playbook_name:string,status:string,runtime:int
)[
datetime(11:02), "config_backup", "success", 3600,
datetime(12:33), "update_inventory", "success", 52,
datetime(13:04), "config_backup", "partial_success", 3600,
datetime(14:31), "update_inventory", "success", 47,
]
| mv-expand TimeGenerated=
range(bin(TimeGenerated, 15m), TimeGenerated + runtime * 1s, 15m)
| project TimeGenerated, playbook_name, status
使用mv-apply,你还可以生成带有空状态的时间事件:
datatable (
TimeGenerated: datetime,
playbook_name: string,
status: string,
runtime: int
)[
datetime(11:02), "config_backup", "success", 3600,
datetime(12:33), "update_inventory", "success", 52,
datetime(13:04), "config_backup", "partial_success", 3600,
datetime(14:31), "update_inventory", "success", 47,
]
| mv-apply TimeGenerated2= range(datetime(11:00), datetime(15:00), 15m) to typeof(datetime) on (
extend status= iff(TimeGenerated2 between (bin(TimeGenerated, 15m) .. (TimeGenerated + runtime * 1s)), status, "")
)
| project TimeGenerated= TimeGenerated2, playbook_name, status
| evaluate pivot(playbook_name, take_any(status))
英文:
You can use the mv-expand operator to generate the additional time entries:
datatable(TimeGenerated:datetime,playbook_name:string,status:string,runtime:int
)[
datetime(11:02), "config_backup", "success", 3600,
datetime(12:33), "update_inventory", "success", 52,
datetime(13:04), "config_backup", "partial_success", 3600,
datetime(14:31), "update_inventory", "success", 47,
]
| mv-expand TimeGenerated=
range(bin(TimeGenerated, 15m), TimeGenerated + runtime * 1s, 15m)
| project TimeGenerated, playbook_name, status
With mv-apply you can also generate time events with an empty status:
datatable (
TimeGenerated: datetime,
playbook_name: string,
status: string,
runtime: int
)[
datetime(11:02), "config_backup", "success", 3600,
datetime(12:33), "update_inventory", "success", 52,
datetime(13:04), "config_backup", "partial_success", 3600,
datetime(14:31), "update_inventory", "success", 47,
]
| mv-apply TimeGenerated2= range(datetime(11:00), datetime(15:00), 15m) to typeof(datetime) on (
extend status= iff(TimeGenerated2 between (bin(TimeGenerated, 15m) .. (TimeGenerated + runtime * 1s)), status, "")
)
| project TimeGenerated= TimeGenerated2, playbook_name, status
| evaluate pivot(playbook_name, take_any(status))
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论