ruamel.yaml中的锚点与Roundtriploader/Roundtripdumper。

huangapple go评论69阅读模式
英文:

ruamel.yaml anchors with Roundtriploader/Roundtripdumper

问题

# 用 ruamel.yaml Python 包加载以下示例 yaml 文件。
- database: dev_db
  <<: &defaults
    adapter: postgres
    host: localhost
    username: postgres
    password: password
  
- database: test_db
  <<: *defaults
- database: prod_db
  <<: *defaults

# 以下是代码部分:
from pydantic import BaseModel
from ruamel.yaml import YAML

yaml = YAML(typ='rt')
with open('config.yaml', 'r') as file:
    envs = yaml.load(file)

for env in envs:
    print(c)

# 使用 typ='rt' 生成以下输出,其中完全缺少别名标签。但是当我将 typ 更改为 'safe' 时,甚至别名标签也会正确输出。

{&#39;database&#39;: &#39;dev_db&#39;}
{&#39;database&#39;: &#39;test_db&#39;}
{&#39;database&#39;: &#39;prod_db&#39;}

我正在尝试为 YAML 中的每个条目创建 Pydantic 数据模型如何使用默认加载器(rt)获取所有属性
英文:

I am trying to load below example yaml file using the ruamel.yaml python package.

- database: dev_db
  &lt;&lt;: &amp;defaults
    adapter: postgres
    host: localhost
    username: postgres
    password: password
  
- database: test_db
  &lt;&lt;: *defaults
- database: prod_db
  &lt;&lt;: *defaults
from pydantic import BaseModel
from ruamel.yaml import YAML

yaml = YAML(typ=&#39;rt&#39;)
with open(&#39;config.yaml&#39;, &#39;r&#39;) as file:
    envs = yaml.load(file)

for env in envs:
  print(c)

This generates below output which misses the aliased tags completely. But when I change the typ to 'safe', even the aliased tags are output correctly.

{&#39;database&#39;: &#39;dev_db&#39;}
{&#39;database&#39;: &#39;test_db&#39;}
{&#39;database&#39;: &#39;prod_db&#39;}

I am trying to create Pydantic data models with each entry in the YAML. How to get all the attributes with default loader(rt)?

答案1

得分: 1

以下是翻译好的内容:

尽管您应该能够将映射指定为合并键的值(而不是先前已锚定映射的别名或此类别名的列表),但在 ruamel.yaml 的往返模式中,对于 ruamel.yaml<0.17.27,这不起作用:

import sys
import ruamel.yaml

yaml_str = &quot;&quot;&quot;\
a: 42
&lt;&lt;: {b: 96}
&quot;&quot;&quot;
yaml = ruamel.yaml.YAML()
data = yaml.load(yaml_str)
print(data)

结果为:

{&#39;a&#39;: 42}

这是由于该节点的创建不正确引起的,尽管您的示例中创建了别名的锚点 default(否则您无法使用别名),但该别名的值似乎为空字典。

这个错误的修复相当小,但它在 RoundTripConstructorflatten_mapping 方法的本地函数中,因此您将需要提供完整的替代方法:

from ruamel.yaml.nodes import MappingNode
from ruamel.yaml.constructor import ConstructorError, DuplicateKeyError, DuplicateKeyFutureWarning

def my_flatten_mapping(self, node):
    def constructed(value_node):
        if value_node in self.constructed_objects:
            value = self.constructed_objects[value_node]
        else:
            value = self.construct_object(value_node, deep=True) # &lt;&lt; used to be deep=False
        return value

    merge_map_list = []
    index = 0
    while index &lt; len(node.value):
        key_node, value_node = node.value[index]
        if key_node.tag == &#39;tag:yaml.org,2002:merge&#39;:
            if merge_map_list:  # double &lt;&lt; key
                if self.allow_duplicate_keys:
                    del node.value[index]
                    index += 1
                    continue
                args = [
                    &#39;while constructing a mapping&#39;,
                    node.start_mark,
                    f&#39;found duplicate key &quot;{key_node.value}&quot;&#39;,
                    key_node.start_mark,
                    &quot;&quot;&quot;
                    To suppress this check see:
                       http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys
                    &quot;&quot;&quot;,
                    &quot;&quot;&quot;\
                    Duplicate keys will become an error in future releases, and are errors
                    by default when using the new API.
                    &quot;&quot;&quot;,
                ]
                if self.allow_duplicate_keys is None:
                    warnings.warn(DuplicateKeyFutureWarning(*args), stacklevel=1)
                else:
                    raise DuplicateKeyError(*args)
            del node.value[index]
            if isinstance(value_node, MappingNode):
                merge_map_list.append((index, constructed(value_node)))
            elif isinstance(value_node, SequenceNode):
                for subnode in value_node.value:
                    if not isinstance(subnode, MappingNode):
                        raise ConstructorError(
                            &#39;while constructing a mapping&#39;,
                            node.start_mark,
                            f&#39;expected a mapping for merging, but found {subnode.id!s}&#39;,
                            subnode.start_mark,
                        )
                    merge_map_list.append((index, constructed(subnode)))
            else:
                raise ConstructorError(
                    &#39;while constructing a mapping&#39;,
                    node.start_mark,
                    &#39;expected a mapping or list of mappings for merging, &#39;
                    f&#39;but found {value_node.id!s}&#39;,
                    value_node.start_mark,
                )
        elif key_node.tag == &#39;tag:yaml.org,2002:value&#39;:
            key_node.tag = &#39;tag:yaml.org,2002:str&#39;
            index += 1
        else:
            index += 1
    return merge_map_list

yaml = ruamel.yaml.YAML()
yaml.Constructor.flatten_mapping = my_flatten_mapping

data = yaml.load(&quot;&quot;&quot;\
a: 42
&lt;&lt;: {b: 96}
&quot;&quot;&quot;)
print(data)
print(&#39;=&#39; * 20)

data = yaml.load(&quot;&quot;&quot;\
- database: dev_db
  &lt;&lt;: &amp;defaults
    adapter: postgres
    host: localhost
    username: postgres
    password: password
  
- database: test_db
  &lt;&lt;: *defaults
- database: prod_db
  &lt;&lt;: *defaults
&quot;&quot;&quot;)

for d in data:
    print(d)

结果为:

{&#39;a&#39;: 42, &#39;b&#39;: 96}
====================
{&#39;database&#39;: &#39;dev_db&#39;, &#39;adapter&#39;: &#39;postgres&#39;, &#39;host&#39;: &#39;localhost&#39;, &#39;username&#39;: &#39;postgres&#39;, &#39;password&#39;: &#39;password&#39;}
{&#39;database&#39;: &#39;test_db&#39;, &#39;adapter&#39;: &#39;postgres&#39;, &#39;host&#39;: &#39;localhost&#39;, &#39;username&#39;: &#39;postgres&#39;, &#39;password&#39;: &#39;password&#39;}
{&#39;database&#39;: &#39;prod_db&#39;, &#39;adapter&#39;: &#39;postgres&#39;, &#39;host&#39;: &#39;localhost&#39;, &#39;username&#39;: &#39;postgres&#39;, &#39;password&#39;: &#39;password&#39;}

此错误的修复在 ruamel.yaml>=0.17.27 中可用。

英文:

Although you should be able to specify a mapping as value for a merge key (instead of an alias to
some previously anchored mapping, or a list of such aliases), this doesn't work properly
in ruamel.yaml's round-trip mode for ruamel.yaml&lt;0.17.27:

import sys
import ruamel.yaml

yaml_str = &quot;&quot;&quot;\
a: 42
&lt;&lt;: {b: 96}
&quot;&quot;&quot;
yaml = ruamel.yaml.YAML()
data = yaml.load(yaml_str)
print(data)

gives:

{&#39;a&#39;: 42}

This is caused by an incorrect creation of that node, and although the anchor default in your example
is created (otherwise you could not use the alias), the value for that appears to be an empty dict.

The fix for this bug is rather small, but it is in a local function of the flatten_mapping method
of the RoundTripConstructor, so you'll have to provide the full replacement
for that:

from ruamel.yaml.nodes import MappingNode
from ruamel.yaml.constructor import ConstructorError, DuplicateKeyError, DuplicateKeyFutureWarning

def my_flatten_mapping(self, node):
    def constructed(value_node):
        if value_node in self.constructed_objects:
            value = self.constructed_objects[value_node]
        else:
            value = self.construct_object(value_node, deep=True) # &lt;&lt; used to be deep=False
        return value

    merge_map_list = []
    index = 0
    while index &lt; len(node.value):
        key_node, value_node = node.value[index]
        if key_node.tag == &#39;tag:yaml.org,2002:merge&#39;:
            if merge_map_list:  # double &lt;&lt; key
                if self.allow_duplicate_keys:
                    del node.value[index]
                    index += 1
                    continue
                args = [
                    &#39;while constructing a mapping&#39;,
                    node.start_mark,
                    f&#39;found duplicate key &quot;{key_node.value}&quot;&#39;,
                    key_node.start_mark,
                    &quot;&quot;&quot;
                    To suppress this check see:
                       http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys
                    &quot;&quot;&quot;,
                    &quot;&quot;&quot;\
                    Duplicate keys will become an error in future releases, and are errors
                    by default when using the new API.
                    &quot;&quot;&quot;,
                ]
                if self.allow_duplicate_keys is None:
                    warnings.warn(DuplicateKeyFutureWarning(*args), stacklevel=1)
                else:
                    raise DuplicateKeyError(*args)
            del node.value[index]
            if isinstance(value_node, MappingNode):
                merge_map_list.append((index, constructed(value_node)))
            elif isinstance(value_node, SequenceNode):
                for subnode in value_node.value:
                    if not isinstance(subnode, MappingNode):
                        raise ConstructorError(
                            &#39;while constructing a mapping&#39;,
                            node.start_mark,
                            f&#39;expected a mapping for merging, but found {subnode.id!s}&#39;,
                            subnode.start_mark,
                        )
                    merge_map_list.append((index, constructed(subnode)))
            else:
                raise ConstructorError(
                    &#39;while constructing a mapping&#39;,
                    node.start_mark,
                    &#39;expected a mapping or list of mappings for merging, &#39;
                    f&#39;but found {value_node.id!s}&#39;,
                    value_node.start_mark,
                )
        elif key_node.tag == &#39;tag:yaml.org,2002:value&#39;:
            key_node.tag = &#39;tag:yaml.org,2002:str&#39;
            index += 1
        else:
            index += 1
    return merge_map_list


yaml = ruamel.yaml.YAML()
yaml.Constructor.flatten_mapping = my_flatten_mapping

data = yaml.load(&quot;&quot;&quot;\
a: 42
&lt;&lt;: {b: 96}
&quot;&quot;&quot;)
print(data)
print(&#39;=&#39; * 20)

data = yaml.load(&quot;&quot;&quot;\
- database: dev_db
  &lt;&lt;: &amp;defaults
    adapter: postgres
    host: localhost
    username: postgres
    password: password
  
- database: test_db
  &lt;&lt;: *defaults
- database: prod_db
  &lt;&lt;: *defaults
&quot;&quot;&quot;)

for d in data:
    print(d)

gives:

{&#39;a&#39;: 42, &#39;b&#39;: 96}
====================
{&#39;database&#39;: &#39;dev_db&#39;, &#39;adapter&#39;: &#39;postgres&#39;, &#39;host&#39;: &#39;localhost&#39;, &#39;username&#39;: &#39;postgres&#39;, &#39;password&#39;: &#39;password&#39;}
{&#39;database&#39;: &#39;test_db&#39;, &#39;adapter&#39;: &#39;postgres&#39;, &#39;host&#39;: &#39;localhost&#39;, &#39;username&#39;: &#39;postgres&#39;, &#39;password&#39;: &#39;password&#39;}
{&#39;database&#39;: &#39;prod_db&#39;, &#39;adapter&#39;: &#39;postgres&#39;, &#39;host&#39;: &#39;localhost&#39;, &#39;username&#39;: &#39;postgres&#39;, &#39;password&#39;: &#39;password&#39;}

The fix for this is in ruamel.yaml&gt;=0.17.27

huangapple
  • 本文由 发表于 2023年5月25日 17:59:24
  • 转载请务必保留本文链接:https://go.coder-hub.com/76331049.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定