获取嵌套TypedDicts的键

huangapple
go评论129阅读模式
英文:

Get keys of nested TypedDicts

问题

以下是已翻译的内容：

拥有以下的 TypedDict：

  1. class MySubClass(TypedDict):
  2.     name: str
  3.     color: str
  5. class MyClass(TypedDict):
  6.     x: MySubClass
  7.     y: str

有一个能够递归提取键的函数是什么：

  1. [x_name, x_color, y]

该函数应该是动态的，以便可以提取所有种类的嵌套结构，但一层嵌套已经足够了。

非常感谢！

英文:

Having the following TypedDict:

  1. class MySubClass(TypedDict):
  2.     name: str
  3.     color: str
  5. class MyClass(TypedDict):
  6.     x: MySubClass
  7.     y: str

What is the function, that can extract the keys recursively like this:

  1. [x_name, x_color, y]

The function should be dynamic so that it can extract all kinds of nested structures, but one level of nesting is already enough.

Many Thanks!

答案1

得分: 3

以下是您要翻译的内容：

Python >=3.10

对于 Python >=3.10，我们有typing.is_typeddict函数，可以用来检查是否实际上是TypedDict的子类型。

我们可以在TypedDict类上使用typing.get_type_hints（Python >=3.9）来获取其上定义的键和相应的类型。 （这可能比直接使用其__annotations__字典更好，正如@chepner在评论中指出的。）

一个简单的递归函数以您想要的方式获取键可能如下所示：

  1. from typing import is_typeddict
  4. def get_typed_dict_keys(cls: type) -> list[str]:
  5.     keys: list[str] = []
  6.     for key, type_ in get_type_hints(cls).items():
  7.         if is_typeddict(type_):
  8.             keys.extend(
  9.                 f"{key}_{sub_key}"
  10.                 for sub_key in get_typed_dict_keys(type_)
  11.             )
  12.         else:
  13.             keys.append(key)
  14.     return keys

演示：

  1. from typing import TypedDict
  4. class Foo(TypedDict):
  5.     name: str
  6.     color: str
  9. class Bar(TypedDict):
  10.     x: Foo
  11.     y: str
  14. class Baz(TypedDict):
  15.     a: int
  16.     b: Bar
  19. print(get_typed_dict_keys(Baz))

输出：['a', 'b_x_name', 'b_x_color', 'b_y']

显然，您可能会遇到嵌套键的名称冲突。但我相信您已经意识到了这一点。

附注：如果您使用的是Python >=3.11，并且有一些用NotRequired注释的键，这个解决方案仍然可以工作，因为get_type_hints会将这些注释解析为基础类型。

例如，以下Baz类：

  1. from typing import TypedDict, NotRequired
  3. ...
  5. class Baz(TypedDict):
  6.     a: int
  7.     b: NotRequired[Bar]

该函数仍然可以工作并返回相同的输出。

Python 3.9

在这里，我们需要创造性地解决问题，因为我们无法使用is_typeddict。为了毫不羞耻地从Pydantic中窃取一些内容，我们可以简单地检查某个对象是否1) 是dict的子类型并且2) 具有典型的TypedDict类属性。这显然不够可靠，但在大多数情况下应该足够好：

  1. from typing import get_type_hints
  4. def is_typeddict(cls: type) -> bool:
  5.     if not (isinstance(cls, type) and issubclass(cls, dict)):
  6.         return False
  7.     return hasattr(cls, "__total__") and hasattr(cls, "__required_keys__")
  10. def get_typed_dict_keys(cls: type) -> list[str]:
  11.     keys: list[str] = []
  12.     for key, type_ in get_type_hints(cls).items():
  13.         if is_typeddict(type_):
  14.             keys.extend(
  15.                 f"{key}_{sub_key}"
  16.                 for sub_key in get_typed_dict_keys(type_)
  17.             )
  18.         else:
  19.             keys.append(key)
  20.     return keys

同样的演示，同样的输出。

Python 3.8

没有typing.get_type_hints，我们可以在get_typed_dict_keys函数中替换那个调用为cls.__annotations__。

此外，TypedDict.__required_keys__类属性仅在Python 3.9中添加，因此要查看某个对象是否是TypedDict，我们只能检查__total__。这显然更不稳健，但在Python 3.8中是我们能做的最好的了。

适当调整类型注释等，3.8代码如下所示：

  1. from typing import Any, List, Type
  4. def is_typeddict(cls: Type[Any]) -> bool:
  5.     if not (isinstance(cls, type) and issubclass(cls, dict)):
  6.         return False
  7.     return hasattr(cls, "__total__")
  10. def get_typed_dict_keys(cls: Type[Any]) -> List[str]:
  11.     keys: List[str] = []
  12.     for key, type_ in cls.__annotations__.items():
  13.         if is_typeddict(type_):
  14.             keys.extend(
  15.                 f"{key}_{sub_key}"
  16.                 for sub_key in get_typed_dict_keys(type_)
  17.             )
  18.         else:
  19.             keys.append(key)
  20.     return keys

同样的演示，同样的输出。

附注，仅供娱乐

以下是一个函数，可以返回嵌套TypedDict的注释的嵌套字典，并可选择将其展平，只是因为有趣：

  1. from typing import Any, get_type_hints, is_typeddict
  4. def get_typed_dict_annotations(
  5.     cls: type,
  6.     recursive: bool = False,
  7.     flatten: bool = False,
  8. ) -> dict[str, Any]:
  9.     if not recursive:
  10.         return get_type_hints(cls)
  11.     output: dict[str, Any] = {}
  12.     for key, type_ in get_type_hints(cls).items():
  13.         if not is_typeddict(type_):
  14.             output[key] = type_
  15.             continue
  16.         sub_dict = get_typed_dict_annotations(
  17.             type_,
  18.             recursive=recursive,
  19.             flatten=flatten,
  20.         )
  21.         if flatten:
  22.             for sub_key, sub_type in sub_dict.items():
  23.                 output[f"{key}_{sub_key}"] = sub_type
  24.         else:
  25.             output[key] = sub_dict
  26.     return output

演示：

  1. from pprint import pprint
  2. from typing import TypedDict
  5. class Foo(TypedDict):
  6.     name: str
  7.     color: str
  10. class Bar(TypedDict):
  11.     x: Foo
  12.     y: str
  15. class Baz(TypedDict):
  16.     a: int
  17.     b: Bar
  20. baz_annotations = get_typed_dict_annotations(Baz, recursive=True, flatten=True)
  21. pprint(baz_annotations, sort_dicts=False)
  23. print(list(baz_annotations.keys()))
  25. baz_annotations = get_typed_dict_annotations(Baz, recursive=True)
  26. pprint(baz_annotations, sort_dicts=False)

输出：

  1. {'a': <class 'int'>,
  2.  'b_x_name': <class 'str'>,
  3.  'b_x_color': <class 'str'>,
  4.  '
  6. <details>
  7. <summary>英文:</summary>
  9. ## Python `&gt;=3.10`
  11. For Python `&gt;=3.10` we have the [`typing.is_typeddict`][1] function that we can use to check, if something is in fact a [`TypedDict`][2] subtype.
  13. We can use [`typing.get_type_hints`][3] (Python `&gt;=3.9`) on a `TypedDict` class to get the keys and corresponding types defined on it. (This is likely better that using its [`__annotations__`][4] dictionary directly as pointed out by @chepner in a comment.)
  15. A simple recursive function to get the keys the way you wanted might look like this:
  17. ```python
  18. from typing import is_typeddict
  21. def get_typed_dict_keys(cls: type) -&gt; list[str]:
  22.     keys: list[str] = []
  23.     for key, type_ in get_type_hints(cls).items():
  24.         if is_typeddict(type_):
  25.             keys.extend(
  26.                 f&quot;{key}_{sub_key}&quot;
  27.                 for sub_key in get_typed_dict_keys(type_)
  28.             )
  29.         else:
  30.             keys.append(key)
  31.     return keys

Demo:

  1. from typing import TypedDict
  4. class Foo(TypedDict):
  5.     name: str
  6.     color: str
  9. class Bar(TypedDict):
  10.     x: Foo
  11.     y: str
  14. class Baz(TypedDict):
  15.     a: int
  16.     b: Bar
  19. print(get_typed_dict_keys(Baz))

Output: [&#39;a&#39;, &#39;b_x_name&#39;, &#39;b_x_color&#39;, &#39;b_y&#39;]

Obviously, you might run into name collisions with the way the nested keys are formed. But I am sure you are aware of that.

Side note: If you are on Python &gt;=3.11 and have some keys that are annotated with NotRequired, this solution should still work because get_type_hints resolves those annotations to the underlying type.

E.g. the following Baz class:

  1. from typing import TypedDict, NotRequired
  3. ...
  5. class Baz(TypedDict):
  6.     a: int
  7.     b: NotRequired[Bar]

The function would still work and return the same output.

Python 3.9

Here we need to get creative because is_typeddict is not available to us. To shamelessly steal bits from Pydantic, we could simply check if something is 1) a dict subtype and 2) has the typical TypedDict class attributes. This is obviously less reliable, but should work well enough in most cases:

  1. from typing import get_type_hints
  4. def is_typeddict(cls: type) -&gt; bool:
  5.     if not (isinstance(cls, type) and issubclass(cls, dict)):
  6.         return False
  7.     return hasattr(cls, &quot;__total__&quot;) and hasattr(cls, &quot;__required_keys__&quot;)
  10. def get_typed_dict_keys(cls: type) -&gt; list[str]:
  11.     keys: list[str] = []
  12.     for key, type_ in get_type_hints(cls).items():
  13.         if is_typeddict(type_):
  14.             keys.extend(
  15.                 f&quot;{key}_{sub_key}&quot;
  16.                 for sub_key in get_typed_dict_keys(type_)
  17.             )
  18.         else:
  19.             keys.append(key)
  20.     return keys

Same Demo, same output.

Python 3.8

Without typing.get_type_hints, we can just use replace that call in the get_typed_dict_keys function with cls.__annotations__.

Also, the TypedDict.__required_keys__ class attribute was only added in Python 3.9, so to see if something is a TypedDict, we can only check for __total__. This is obviously even less robust, but the best we can do with Python 3.8.

With type annotations etc. adjusted properly, the 3.8 code would look like this:

  1. from typing import Any, List, Type
  4. def is_typeddict(cls: Type[Any]) -&gt; bool:
  5.     if not (isinstance(cls, type) and issubclass(cls, dict)):
  6.         return False
  7.     return hasattr(cls, &quot;__total__&quot;)
  10. def get_typed_dict_keys(cls: Type[Any]) -&gt; List[str]:
  11.     keys: List[str] = []
  12.     for key, type_ in cls.__annotations__.items():
  13.         if is_typeddict(type_):
  14.             keys.extend(
  15.                 f&quot;{key}_{sub_key}&quot;
  16.                 for sub_key in get_typed_dict_keys(type_)
  17.             )
  18.         else:
  19.             keys.append(key)
  20.     return keys

Same Demo, same output.

PS, just for fun

Here is a function that can return a nested dictionary of annotations of nested TypedDicts and optionally flatten it, just because:

  1. from typing import Any, get_type_hints, is_typeddict
  4. def get_typed_dict_annotations(
  5.     cls: type,
  6.     recursive: bool = False,
  7.     flatten: bool = False,
  8. ) -&gt; dict[str, Any]:
  9.     if not recursive:
  10.         return get_type_hints(cls)
  11.     output: dict[str, Any] = {}
  12.     for key, type_ in get_type_hints(cls).items():
  13.         if not is_typeddict(type_):
  14.             output[key] = type_
  15.             continue
  16.         sub_dict = get_typed_dict_annotations(
  17.             type_,
  18.             recursive=recursive,
  19.             flatten=flatten,
  20.         )
  21.         if flatten:
  22.             for sub_key, sub_type in sub_dict.items():
  23.                 output[f&quot;{key}_{sub_key}&quot;] = sub_type
  24.         else:
  25.             output[key] = sub_dict
  26.     return output

Demo:

  1. from pprint import pprint
  2. from typing import TypedDict
  5. class Foo(TypedDict):
  6.     name: str
  7.     color: str
  10. class Bar(TypedDict):
  11.     x: Foo
  12.     y: str
  15. class Baz(TypedDict):
  16.     a: int
  17.     b: Bar
  20. baz_annotations = get_typed_dict_annotations(Baz, recursive=True, flatten=True)
  21. pprint(baz_annotations, sort_dicts=False)
  23. print(list(baz_annotations.keys()))
  25. baz_annotations = get_typed_dict_annotations(Baz, recursive=True)
  26. pprint(baz_annotations, sort_dicts=False)

Output:

  1. {&#39;a&#39;: &lt;class &#39;int&#39;&gt;,
  2.  &#39;b_x_name&#39;: &lt;class &#39;str&#39;&gt;,
  3.  &#39;b_x_color&#39;: &lt;class &#39;str&#39;&gt;,
  4.  &#39;b_y&#39;: &lt;class &#39;str&#39;&gt;}
  1. [&#39;a&#39;, &#39;b_x_name&#39;, &#39;b_x_color&#39;, &#39;b_y&#39;]
  1. {&#39;a&#39;: &lt;class &#39;int&#39;&gt;,
  2.  &#39;b&#39;: {&#39;x&#39;: {&#39;name&#39;: &lt;class &#39;str&#39;&gt;, &#39;color&#39;: &lt;class &#39;str&#39;&gt;},
  3.        &#39;y&#39;: &lt;class &#39;str&#39;&gt;}}

huangapple
  • 本文由 发表于 2023年6月16日 02:18:50
  • 转载请务必保留本文链接：https://go.coder-hub.com/76484484.html
  • python
  • python-typing
  • typeddict
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定