英文:
ruamel.yaml anchors with Roundtriploader/Roundtripdumper
问题
# 用 ruamel.yaml Python 包加载以下示例 yaml 文件。
- database: dev_db
<<: &defaults
adapter: postgres
host: localhost
username: postgres
password: password
- database: test_db
<<: *defaults
- database: prod_db
<<: *defaults
# 以下是代码部分:
from pydantic import BaseModel
from ruamel.yaml import YAML
yaml = YAML(typ='rt')
with open('config.yaml', 'r') as file:
envs = yaml.load(file)
for env in envs:
print(c)
# 使用 typ='rt' 生成以下输出,其中完全缺少别名标签。但是当我将 typ 更改为 'safe' 时,甚至别名标签也会正确输出。
{'database': 'dev_db'}
{'database': 'test_db'}
{'database': 'prod_db'}
我正在尝试为 YAML 中的每个条目创建 Pydantic 数据模型。如何使用默认加载器(rt)获取所有属性?
英文:
I am trying to load below example yaml file using the ruamel.yaml python package.
- database: dev_db
<<: &defaults
adapter: postgres
host: localhost
username: postgres
password: password
- database: test_db
<<: *defaults
- database: prod_db
<<: *defaults
from pydantic import BaseModel
from ruamel.yaml import YAML
yaml = YAML(typ='rt')
with open('config.yaml', 'r') as file:
envs = yaml.load(file)
for env in envs:
print(c)
This generates below output which misses the aliased tags completely. But when I change the typ to 'safe', even the aliased tags are output correctly.
{'database': 'dev_db'}
{'database': 'test_db'}
{'database': 'prod_db'}
I am trying to create Pydantic data models with each entry in the YAML. How to get all the attributes with default loader(rt)?
答案1
得分: 1
以下是翻译好的内容:
尽管您应该能够将映射指定为合并键的值(而不是先前已锚定映射的别名或此类别名的列表),但在 ruamel.yaml
的往返模式中,对于 ruamel.yaml<0.17.27
,这不起作用:
import sys
import ruamel.yaml
yaml_str = """\
a: 42
<<: {b: 96}
"""
yaml = ruamel.yaml.YAML()
data = yaml.load(yaml_str)
print(data)
结果为:
{'a': 42}
这是由于该节点的创建不正确引起的,尽管您的示例中创建了别名的锚点 default
(否则您无法使用别名),但该别名的值似乎为空字典。
这个错误的修复相当小,但它在 RoundTripConstructor
的 flatten_mapping
方法的本地函数中,因此您将需要提供完整的替代方法:
from ruamel.yaml.nodes import MappingNode
from ruamel.yaml.constructor import ConstructorError, DuplicateKeyError, DuplicateKeyFutureWarning
def my_flatten_mapping(self, node):
def constructed(value_node):
if value_node in self.constructed_objects:
value = self.constructed_objects[value_node]
else:
value = self.construct_object(value_node, deep=True) # << used to be deep=False
return value
merge_map_list = []
index = 0
while index < len(node.value):
key_node, value_node = node.value[index]
if key_node.tag == 'tag:yaml.org,2002:merge':
if merge_map_list: # double << key
if self.allow_duplicate_keys:
del node.value[index]
index += 1
continue
args = [
'while constructing a mapping',
node.start_mark,
f'found duplicate key "{key_node.value}"',
key_node.start_mark,
"""
To suppress this check see:
http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys
""",
"""\
Duplicate keys will become an error in future releases, and are errors
by default when using the new API.
""",
]
if self.allow_duplicate_keys is None:
warnings.warn(DuplicateKeyFutureWarning(*args), stacklevel=1)
else:
raise DuplicateKeyError(*args)
del node.value[index]
if isinstance(value_node, MappingNode):
merge_map_list.append((index, constructed(value_node)))
elif isinstance(value_node, SequenceNode):
for subnode in value_node.value:
if not isinstance(subnode, MappingNode):
raise ConstructorError(
'while constructing a mapping',
node.start_mark,
f'expected a mapping for merging, but found {subnode.id!s}',
subnode.start_mark,
)
merge_map_list.append((index, constructed(subnode)))
else:
raise ConstructorError(
'while constructing a mapping',
node.start_mark,
'expected a mapping or list of mappings for merging, '
f'but found {value_node.id!s}',
value_node.start_mark,
)
elif key_node.tag == 'tag:yaml.org,2002:value':
key_node.tag = 'tag:yaml.org,2002:str'
index += 1
else:
index += 1
return merge_map_list
yaml = ruamel.yaml.YAML()
yaml.Constructor.flatten_mapping = my_flatten_mapping
data = yaml.load("""\
a: 42
<<: {b: 96}
""")
print(data)
print('=' * 20)
data = yaml.load("""\
- database: dev_db
<<: &defaults
adapter: postgres
host: localhost
username: postgres
password: password
- database: test_db
<<: *defaults
- database: prod_db
<<: *defaults
""")
for d in data:
print(d)
结果为:
{'a': 42, 'b': 96}
====================
{'database': 'dev_db', 'adapter': 'postgres', 'host': 'localhost', 'username': 'postgres', 'password': 'password'}
{'database': 'test_db', 'adapter': 'postgres', 'host': 'localhost', 'username': 'postgres', 'password': 'password'}
{'database': 'prod_db', 'adapter': 'postgres', 'host': 'localhost', 'username': 'postgres', 'password': 'password'}
此错误的修复在 ruamel.yaml>=0.17.27
中可用。
英文:
Although you should be able to specify a mapping as value for a merge key (instead of an alias to
some previously anchored mapping, or a list of such aliases), this doesn't work properly
in ruamel.yaml
's round-trip mode for ruamel.yaml<0.17.27
:
import sys
import ruamel.yaml
yaml_str = """\
a: 42
<<: {b: 96}
"""
yaml = ruamel.yaml.YAML()
data = yaml.load(yaml_str)
print(data)
gives:
{'a': 42}
This is caused by an incorrect creation of that node, and although the anchor default
in your example
is created (otherwise you could not use the alias), the value for that appears to be an empty dict.
The fix for this bug is rather small, but it is in a local function of the flatten_mapping method
of the RoundTripConstructor
, so you'll have to provide the full replacement
for that:
from ruamel.yaml.nodes import MappingNode
from ruamel.yaml.constructor import ConstructorError, DuplicateKeyError, DuplicateKeyFutureWarning
def my_flatten_mapping(self, node):
def constructed(value_node):
if value_node in self.constructed_objects:
value = self.constructed_objects[value_node]
else:
value = self.construct_object(value_node, deep=True) # << used to be deep=False
return value
merge_map_list = []
index = 0
while index < len(node.value):
key_node, value_node = node.value[index]
if key_node.tag == 'tag:yaml.org,2002:merge':
if merge_map_list: # double << key
if self.allow_duplicate_keys:
del node.value[index]
index += 1
continue
args = [
'while constructing a mapping',
node.start_mark,
f'found duplicate key "{key_node.value}"',
key_node.start_mark,
"""
To suppress this check see:
http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys
""",
"""\
Duplicate keys will become an error in future releases, and are errors
by default when using the new API.
""",
]
if self.allow_duplicate_keys is None:
warnings.warn(DuplicateKeyFutureWarning(*args), stacklevel=1)
else:
raise DuplicateKeyError(*args)
del node.value[index]
if isinstance(value_node, MappingNode):
merge_map_list.append((index, constructed(value_node)))
elif isinstance(value_node, SequenceNode):
for subnode in value_node.value:
if not isinstance(subnode, MappingNode):
raise ConstructorError(
'while constructing a mapping',
node.start_mark,
f'expected a mapping for merging, but found {subnode.id!s}',
subnode.start_mark,
)
merge_map_list.append((index, constructed(subnode)))
else:
raise ConstructorError(
'while constructing a mapping',
node.start_mark,
'expected a mapping or list of mappings for merging, '
f'but found {value_node.id!s}',
value_node.start_mark,
)
elif key_node.tag == 'tag:yaml.org,2002:value':
key_node.tag = 'tag:yaml.org,2002:str'
index += 1
else:
index += 1
return merge_map_list
yaml = ruamel.yaml.YAML()
yaml.Constructor.flatten_mapping = my_flatten_mapping
data = yaml.load("""\
a: 42
<<: {b: 96}
""")
print(data)
print('=' * 20)
data = yaml.load("""\
- database: dev_db
<<: &defaults
adapter: postgres
host: localhost
username: postgres
password: password
- database: test_db
<<: *defaults
- database: prod_db
<<: *defaults
""")
for d in data:
print(d)
gives:
{'a': 42, 'b': 96}
====================
{'database': 'dev_db', 'adapter': 'postgres', 'host': 'localhost', 'username': 'postgres', 'password': 'password'}
{'database': 'test_db', 'adapter': 'postgres', 'host': 'localhost', 'username': 'postgres', 'password': 'password'}
{'database': 'prod_db', 'adapter': 'postgres', 'host': 'localhost', 'username': 'postgres', 'password': 'password'}
The fix for this is in ruamel.yaml>=0.17.27
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论