使用宏设置列元数据的DBT

huangapple go评论62阅读模式
英文:

DBT set column metadata using macros

问题

我正在尝试在SQL中使用宏来设置我的列元数据,而不是在YML中设置,但我无法弄清楚是否可能。

我目前可以在YML中成功设置policy_tags,例如test.yml:

version: 2

models:
- name: test
  columns:
    - name: a
      policy_tags: ["MY_TAG"]

但是,我想在SQL中做类似的事情,像这样:

{# 伪代码,不起作用 %#}
{{ config.set('columns', [{ name: "a", policy_tags: ["MY_TAG"] }]) }}

我正在使用宏在SQL中动态生成可变数量的列,因此这将允许我动态标记所有这些列。感谢任何建议!

英文:

I'm trying to set my column metadata in sql using macros instead of in yml, but can't figure out if this is possible.

I'm currently successfully setting policy_tags in the yml, eg test.yml:

version: 2

models:
- name: test
  columns:
    - name: a
      policy_tags: ["MY_TAG"]

However, I'd like to do the same thing in SQL, with something like:

{# FAKE CODE, DOESN'T WORK %#}
{{ config.set('columns', [{ name: "a", policy_tags: ["MY_TAG"] }]) }}

I'm dynamically generating a variable number of columns in the sql using macros, so this would let me also dynamically tag all those columns. Any suggestions appreciated!

答案1

得分: 1

以下是翻译好的部分:

"Turned out I was able to do this in a post_hook by calling persist_docs directly:"
"事实证明,我能够在 post_hook 中直接调用 persist_docs 来完成这个任务:"

"# my_model.sql"
"# 我的模型.sql"

"{{ config({'post-hook': "{{ write_policy_tags() }}"}) }}"
"{{ 配置({'post-hook': "{{ write_policy_tags() }}"}) }}"

"## write_policy_tags.sql"
"## write_policy_tags.sql"

"{% macro write_policy_tags() %}"
"{% 宏 write_policy_tags() %}"

"{% set columns = {"
"{% 设置列 = {"

""col1": { "policy_tags": ["{{ col1tag }}"] },"
""col1": { "policy_tags": ["{{ col1tag }}"] },"

""col2": { "policy_tags": ["{{ col2tag }}"] }"
""col2": { "policy_tags": ["{{ col2tag }}"] }"

"} %}"
"} %}"

"{% do persist_docs(this, { "columns": columns }) %}"
"{% 执行 persist_docs(this, { "columns": columns }) %}"

"Within the macro, I was then able to call run_query and use the results to dynamically generate the columns variable to pass to persist_docs, which achieved my original goal of dynamic policy_tags."
"在宏内部,我随后能够调用 run_query 并使用结果来动态生成传递给 persist_docscolumns 变量,从而实现了我最初的动态 policy_tags 目标。"

英文:

Turned out I was able to do this in a post_hook by calling persist_docs directly:

# my_model.sql
{{ config({'post-hook': "{{ write_policy_tags() }}"}) }}

## write_policy_tags.sql
{% macro write_policy_tags() %}
  {% set columns = {
    "col1": { "policy_tags": ["{{ col1tag }}"] },
    "col2": { "policy_tags": ["{{ col2tag }}"] }
  } %}
  {% do persist_docs(this, { "columns": columns }) %}
{% endmacro %}

Within the macro, I was then able to call run_query and use the results to dynamically generate the columns variable to pass to persist_docs, which achieved my original goal of dynamic policy_tags.

huangapple
  • 本文由 发表于 2023年6月13日 06:12:10
  • 转载请务必保留本文链接:https://go.coder-hub.com/76460631.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定