问题

我仍然在尝试提取Ansible中的数据方面遇到困难。

我有以下清单：

---
all:
  children:
    vms:
      hosts:
        vm1:
          services:
          - exim
          - apache
        vm2:
          services:
          - exim
          - nginx
        vm2:
          services:
          - exim
          - nginx

我定义了以下变量：

services_parameters:
  nginx:
    log_files:
      - /var/log/nginx/*.log
    fail2ban_rules:
      nginx-limit-req:
      nginx-http-auth:
      nginx-botsearch:
      nginx-bad-request:
  exim:
    log_files:
      - /var/log/exim4/mainlog
    fail2ban_rules:
      exim:
      exim-spam:
  apache:
    log_files:
      - /var/log/apache2/*.log
    fail2ban_rules:
      apache-botsearch:
      apache-auth:
        parameters:
          logpath: /var/log/apache2/*.log
          findtime: 3600
          bantime: 86400
          maxretry: 3
      apache-badbots:

使用以下代码，我自豪地设定了一个事实：

ok: [vm1] => {
    "fail2ban_rules": {
        "apache-auth": {
            "parameters": {
                "logpath": "/var/log/apache2/*.log"
            }
        },
        "apache-badbots": null,
        "apache-botsearch": null,
        "exim": null,
        "exim-spam": null
    }
}
ok: [vm2] => {
    "fail2ban_rules": {
        "exim": null,
        "exim-spam": null,
        "nginx-bad-request": null,
        "nginx-botsearch": null,
        "nginx-http-auth": null,
        "nginx-limit-req": null
    }
}
ok: [vm3] => {
    "fail2ban_rules": {
        "exim": null,
        "exim-spam": null,
        "nginx-bad-request": null,
        "nginx-botsearch": null,
        "nginx-http-auth": null,
        "nginx-limit-req": null
    }
}

现在我努力获取每个主机上与附加服务关联的所有日志文件，类似以下（手动编写，可能包含一些拼写错误）：

ok: [vm1] => [
      {"exim": ["/var/log/exim4/mainlog"]},
      {"apache": ["/var/log/apache2/*.log"]}
]
ok: [vm2] => [
      {"exim": ["/var/log/exim4/mainlog"]},
      {"nginx": ["/var/log/nginx/*.log"]}
]
ok: [vm3] => [
      {"exim": ["/var/log/exim4/mainlog"]},
      {"nginx": ["/var/log/nginx/*.log"]}
]

有了这个事实，我将循环遍历服务以生成类似以下的块：

nginx:
  paths:
    - /var/log/nginx/*.log
    - <如果在services_parameters变量中定义，则还有其他路径>

这可能有一个变通方法，但这是我自从使用Ansible以来一直面临的痛苦数据处理的一个很好的示例。

---
all:
  children:
vms:
  hosts:
    vm1:
      services:
      - exim
      - apache
    vm2:
      services:
      - exim
      - nginx
    vm2:
      services:
      - exim
      - nginx

services_parameters:
  nginx:
    log_files:
      - /var/log/nginx/*.log
    fail2ban_rules:
      nginx-limit-req:
      nginx-http-auth:
      nginx-botsearch:
      nginx-bad-request:
  exim:
    log_files:
      - /var/log/exim4/mainlog
    fail2ban_rules:
      exim:
      exim-spam:
  apache:
    log_files:
      - /var/log/apache2/*.log
    fail2ban_rules:
      apache-botsearch:
      apache-auth:
        parameters:
          logpath: /var/log/apache2/*.log
          findtime: 3600
          bantime: 86400
          maxretry: 3
      apache-badbots:

"{{ hostvars[inventory_hostname].services | map('extract', services_parameters, 'fail2ban_rules') | combine }}"

ok: [vm1] => {
    "fail2ban_rules": {
        "apache-auth": {
            "parameters": {
                "logpath": "/var/log/apache2/*.log"
            }
        },
        "apache-badbots": null,
        "apache-botsearch": null,
        "exim": null,
        "exim-spam": null
    }
}
ok: [vm2] => {
    "fail2ban_rules": {
        "exim": null,
        "exim-spam": null,
        "nginx-bad-request": null,
        "nginx-botsearch": null,
        "nginx-http-auth": null,
        "nginx-limit-req": null
    }
}
ok: [vm3] => {
    "fail2ban_rules": {
        "exim": null,
        "exim-spam": null,
        "nginx-bad-request": null,
        "nginx-botsearch": null,
        "nginx-http-auth": null,
        "nginx-limit-req": null
    }
}

ok: [vm1] => [
      {"exim": [
	    "/var/log/exim4/mainlog"
	    ]
      },
      {"apache": [
	    "/var/log/apache2/*.log"
	    ]
      }
]
ok: [vm2] => [
      {"exim": [
	    "/var/log/exim4/mainlog"
	    ]
      },
      {"nginx": [
	    "/var/log/nginx/*.log"
	    ]
      }
]
ok: [vm3] => [
      {"exim": [
	    "/var/log/exim4/mainlog"
	    ]
      },
      {"nginx": [
	    "/var/log/nginx/*.log"
	    ]
      }
]

nginx:
  paths:
    - /var/log/nginx/*.log
    - <other paths if defined in services_parameters variable>

答案1

得分: 1

以下是翻译好的部分：

---
- 名称: 调试每个主机的服务参数
  主机: vms
  收集事实: false

  变量:
    服务列表: >-
      {{
        services_parameters
        | dict2items(key_name='service_name', value_name='service_parameters')
        | selectattr('service_name', 'in', services)
      }}      

  任务:
    - 名称: 显示每个目标服务器的服务列表
      ansible.builtin.debug:
        var: services_list

    - 名称: 循环遍历每个目标的合并的fail2ban规则列表
      ansible.builtin.debug:
        var: item
      loop: "{{ services_list | map(attribute='service_parameters.fail2ban_rules') | combine | dict2items }}"
      loop_control:
        label: "{{ item.key }}"

    - 名称: 循环遍历每个日志文件，保留相关的服务名称
      ansible.builtin.debug:
        msg: "{{ item.1 }} 是服务 {{ item.0.service_name }} 的日志文件"
      loop: "{{ services_list | subelements('service_parameters.log_files') }}"
      loop_control:
        label: "{{ item.1 }}"

演示运行结果:

$ ansible-playbook -i inventory.yml playbook.yml 

PLAY [Debug service params for each host] *****************************************************************************************************************************************************************

TASK [Show service list for each target server] ***********************************************************************************************************************************************************
ok: [vm1] => {
    "services_list": [
        {
            "service_name": "exim",
            "service_parameters": {
                "fail2ban_rules": {
                    "exim": null,
                    "exim-spam": null
                },
                "log_files": [
                    "/var/log/exim4/mainlog"
                ]
            }
        },
        {
            "service_name": "apache",
            "service_parameters": {
                "fail2ban_rules": {
                    "apache-auth": {
                        "parameters": {
                            "bantime": 86400,
                            "findtime": 3600,
                            "logpath": "/var/log/apache2/*.log",
                            "maxretry": 3
                        }
                    },
                    "apache-badbots": null,
                    "apache-botsearch": null
                },
                "log_files": [
                    "/var/log/apache2/*.log"
                ]
            }
        }
    ]
}
ok: [vm2] => {
    "services_list": [
        {
            "service_name": "nginx",
            "service_parameters": {
                "fail2ban_rules": {
                    "nginx-bad-request": null,
                    "nginx-botsearch": null,
                    "nginx-http-auth": null,
                    "nginx-limit-req": null
                },
                "log_files": [
                    "/var/log/nginx/*.log"
                ]
            }
        },
        {
            "service_name": "exim",
            "service_parameters": {
                "fail2ban_rules": {
                    "exim": null,
                    "exim-spam": null
                },
                "log_files": [
                    "/var/log/exim4/mainlog"
                ]
            }
        }
    ]
}
ok: [vm3] => {
    "services_list": [
        {
            "service_name": "nginx",
            "service_parameters": {
                "fail2ban_rules": {
                    "nginx-bad-request": null,
                    "nginx-botsearch": null,
                    "nginx-http-auth": null,
                    "nginx-limit-req": null
                },
                "log_files": [
                    "/var/log/nginx/*.log"
                ]
            }
        },
        {
            "service_name": "apache",
            "service_parameters": {
                "fail2ban_rules": {
                    "apache-auth": {
                        "parameters": {
                            "bantime": 86400,
                            "findtime": 3600,
                            "logpath": "/var/log/apache2/*.log",
                            "maxretry": 3
                        }
                    },
                    "apache-badbots": null,
                    "apache-botsearch": null
                },
                "log_files": [
                    "/var/log/apache2/*.log"
                ]
            }
        }
    ]
}

TASK [loop over a consolidated list of fail2ban rules for each target] ************************************************************************************************************************************
ok: [vm1] => (item=exim) => {
    "ansible_loop_var": "item",
    "item": {
        "key": "exim",
        "value": null
    }
}
ok: [vm2] => (item=nginx-limit-req) => {
    "ansible_loop_var": "item",
    "item": {
        "key": "nginx-limit-req",
        "value": null
    }
}
ok: [vm1] => (item=exim-spam) => {
    "ansible_loop_var": "item",
    "item": {
        "key": "exim-spam",
        "value": null
    }
}
ok: [vm3] => (item=nginx-limit-req) => {
    "ansible_loop_var": "item",
    "item": {
        "key": "nginx-limit-req",
        "value": null
    }
}
ok: [vm2] => (item=nginx-http-auth) => {
    "ansible_loop_var": "item",
    "item": {
        "key": "nginx-http-auth",
        "value": null
    }
}
ok: [vm1] => (item=apache-botsearch) => {
    "ansible_loop_var": "item",
    "item": {
        "key": "apache-botsearch",
        "value": null
    }
}
ok: [vm3] => (item=nginx-http-auth) => {
    "ansible_loop_var": "item",
    "item": {
        "key": "nginx-http-auth",
        "value": null
    }
}
ok: [vm2] => (item=nginx-botsearch) => {
    "ansible_loop_var": "item",
    "item": {
        "key": "nginx-botsearch",
        "value": null
    }
}
ok: [vm1] => (item=apache-auth) => {
    "ansible_loop_var": "item",
    "item": {
        "key": "apache-auth",
        "value": {
            "parameters": {
                "bantime": 86400,
                "findtime": 3600,
                "logpath": "/var/log/apache2/*.log",
                "maxretry": 3
            }
        }
    }
}
ok: [vm3] => (item=nginx-botsearch) => {
    "ansible_loop_var": "item",
    "item": {
        "key": "nginx-botsearch",
        "value": null
    }
}
ok: [vm1] => (item=apache-badbots) => {
    "ansible_loop_var": "item",
    "item": {
        "key": "apache-badbots",
        "value": null
    }
}
ok: [vm2] => (item=nginx-bad-request) => {
    "ansible_loop_var": "

<details>
<summary>英文:</summary>

Given the following `inventory.yml`
```yaml
---
all:
  vars:
    services_parameters:
      nginx:
        log_files:
          - /var/log/nginx/*.log
        fail2ban_rules:
          nginx-limit-req:
          nginx-http-auth:
          nginx-botsearch:
          nginx-bad-request:
      exim:
        log_files:
          - /var/log/exim4/mainlog
        fail2ban_rules:
          exim:
          exim-spam:
      apache:
        log_files:
          - /var/log/apache2/*.log
        fail2ban_rules:
          apache-botsearch:
          apache-auth:
            parameters:
              logpath: /var/log/apache2/*.log
              findtime: 3600
              bantime: 86400
              maxretry: 3
          apache-badbots:
vms:
  hosts:
    vm1:
      services:
        - exim
        - apache
    vm2:
      services:
        - exim
        - nginx
    vm3:
      services:
        - apache
        - nginx

the following self explanatory playbook.yml contains some data manipulation and possible loops to acheive what you need (from what I understood)

---
- name: Debug service params for each host
  hosts: vms
  gather_facts: false

  vars:
    services_list: >- 
      {{ 
        services_parameters 
        | dict2items(key_name='service_name', value_name='service_parameters')
        | selectattr('service_name', 'in', services)
      }}

  tasks:
    - name: Show service list for each target server
      ansible.builtin.debug:
        var: services_list

    - name: loop over a consolidated list of fail2ban rules for each target
      ansible.builtin.debug:
        var: item
      loop: "{{ services_list | map(attribute='service_parameters.fail2ban_rules') | combine | dict2items }}"
      loop_control:
        label: "{{ item.key }}"

    - name: loop over each log file keeping the relevant service name
      ansible.builtin.debug:
        msg: "{{ item.1 }} is a log file of service {{ item.0.service_name }}"
      loop: "{{ services_list | subelements('service_parameters.log_files') }}"
      loop_control:
        label: "{{ item.1 }}"

Demo run

$ ansible-playbook -i inventory.yml playbook.yml 
PLAY [Debug service params for each host] *****************************************************************************************************************************************************************
TASK [Show service list for each target server] ***********************************************************************************************************************************************************
ok: [vm1] => {
"services_list": [
{
"service_name": "exim",
"service_parameters": {
"fail2ban_rules": {
"exim": null,
"exim-spam": null
},
"log_files": [
"/var/log/exim4/mainlog"
]
}
},
{
"service_name": "apache",
"service_parameters": {
"fail2ban_rules": {
"apache-auth": {
"parameters": {
"bantime": 86400,
"findtime": 3600,
"logpath": "/var/log/apache2/*.log",
"maxretry": 3
}
},
"apache-badbots": null,
"apache-botsearch": null
},
"log_files": [
"/var/log/apache2/*.log"
]
}
}
]
}
ok: [vm2] => {
"services_list": [
{
"service_name": "nginx",
"service_parameters": {
"fail2ban_rules": {
"nginx-bad-request": null,
"nginx-botsearch": null,
"nginx-http-auth": null,
"nginx-limit-req": null
},
"log_files": [
"/var/log/nginx/*.log"
]
}
},
{
"service_name": "exim",
"service_parameters": {
"fail2ban_rules": {
"exim": null,
"exim-spam": null
},
"log_files": [
"/var/log/exim4/mainlog"
]
}
}
]
}
ok: [vm3] => {
"services_list": [
{
"service_name": "nginx",
"service_parameters": {
"fail2ban_rules": {
"nginx-bad-request": null,
"nginx-botsearch": null,
"nginx-http-auth": null,
"nginx-limit-req": null
},
"log_files": [
"/var/log/nginx/*.log"
]
}
},
{
"service_name": "apache",
"service_parameters": {
"fail2ban_rules": {
"apache-auth": {
"parameters": {
"bantime": 86400,
"findtime": 3600,
"logpath": "/var/log/apache2/*.log",
"maxretry": 3
}
},
"apache-badbots": null,
"apache-botsearch": null
},
"log_files": [
"/var/log/apache2/*.log"
]
}
}
]
}
TASK [loop over a consolidated list of fail2ban rules for each target] ************************************************************************************************************************************
ok: [vm1] => (item=exim) => {
"ansible_loop_var": "item",
"item": {
"key": "exim",
"value": null
}
}
ok: [vm2] => (item=nginx-limit-req) => {
"ansible_loop_var": "item",
"item": {
"key": "nginx-limit-req",
"value": null
}
}
ok: [vm1] => (item=exim-spam) => {
"ansible_loop_var": "item",
"item": {
"key": "exim-spam",
"value": null
}
}
ok: [vm3] => (item=nginx-limit-req) => {
"ansible_loop_var": "item",
"item": {
"key": "nginx-limit-req",
"value": null
}
}
ok: [vm2] => (item=nginx-http-auth) => {
"ansible_loop_var": "item",
"item": {
"key": "nginx-http-auth",
"value": null
}
}
ok: [vm1] => (item=apache-botsearch) => {
"ansible_loop_var": "item",
"item": {
"key": "apache-botsearch",
"value": null
}
}
ok: [vm3] => (item=nginx-http-auth) => {
"ansible_loop_var": "item",
"item": {
"key": "nginx-http-auth",
"value": null
}
}
ok: [vm2] => (item=nginx-botsearch) => {
"ansible_loop_var": "item",
"item": {
"key": "nginx-botsearch",
"value": null
}
}
ok: [vm1] => (item=apache-auth) => {
"ansible_loop_var": "item",
"item": {
"key": "apache-auth",
"value": {
"parameters": {
"bantime": 86400,
"findtime": 3600,
"logpath": "/var/log/apache2/*.log",
"maxretry": 3
}
}
}
}
ok: [vm3] => (item=nginx-botsearch) => {
"ansible_loop_var": "item",
"item": {
"key": "nginx-botsearch",
"value": null
}
}
ok: [vm1] => (item=apache-badbots) => {
"ansible_loop_var": "item",
"item": {
"key": "apache-badbots",
"value": null
}
}
ok: [vm2] => (item=nginx-bad-request) => {
"ansible_loop_var": "item",
"item": {
"key": "nginx-bad-request",
"value": null
}
}
ok: [vm3] => (item=nginx-bad-request) => {
"ansible_loop_var": "item",
"item": {
"key": "nginx-bad-request",
"value": null
}
}
ok: [vm2] => (item=exim) => {
"ansible_loop_var": "item",
"item": {
"key": "exim",
"value": null
}
}
ok: [vm3] => (item=apache-botsearch) => {
"ansible_loop_var": "item",
"item": {
"key": "apache-botsearch",
"value": null
}
}
ok: [vm2] => (item=exim-spam) => {
"ansible_loop_var": "item",
"item": {
"key": "exim-spam",
"value": null
}
}
ok: [vm3] => (item=apache-auth) => {
"ansible_loop_var": "item",
"item": {
"key": "apache-auth",
"value": {
"parameters": {
"bantime": 86400,
"findtime": 3600,
"logpath": "/var/log/apache2/*.log",
"maxretry": 3
}
}
}
}
ok: [vm3] => (item=apache-badbots) => {
"ansible_loop_var": "item",
"item": {
"key": "apache-badbots",
"value": null
}
}
TASK [loop over each log file keeping the relevant service name] ******************************************************************************************************************************************
ok: [vm1] => (item=/var/log/exim4/mainlog) => {
"msg": "/var/log/exim4/mainlog is a log file of service exim"
}
ok: [vm2] => (item=/var/log/nginx/*.log) => {
"msg": "/var/log/nginx/*.log is a log file of service nginx"
}
ok: [vm1] => (item=/var/log/apache2/*.log) => {
"msg": "/var/log/apache2/*.log is a log file of service apache"
}
ok: [vm2] => (item=/var/log/exim4/mainlog) => {
"msg": "/var/log/exim4/mainlog is a log file of service exim"
}
ok: [vm3] => (item=/var/log/nginx/*.log) => {
"msg": "/var/log/nginx/*.log is a log file of service nginx"
}
ok: [vm3] => (item=/var/log/apache2/*.log) => {
"msg": "/var/log/apache2/*.log is a log file of service apache"
}
PLAY RECAP ************************************************************************************************************************************************************************************************
vm1                        : ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
vm2                        : ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
vm3                        : ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0