用不同的根名称为Python软件包设置别名,当旧的软件包使用延迟导入时。

huangapple go评论69阅读模式
英文:

Alias Python package with a different root name when the old one uses lazy imports

问题

以下是您要翻译的内容:

"I'm currently in the middle of renaming a project. In the meantime I need to not break existing users, so I'd like to provide an alias that lets users use it exactly the same as they previously would. This is turning out to be tricker than I initially thought.

Say I have a repo with a package:

├── setup.py
├── pyproject.toml
└── oldpkg
    ├── __init__.py
    ├── __main__.py
    ├── submod.py
    └── subpkg
        ├── __init__.py
        └── nested.py

My first thought was to introduce:

├── newpkg.py

and fill its contents with from oldpkg import *

Then in setup.py add:

        packages=find_packages(include=[
            'oldpkg', 'oldpkg.*',
            # Alias the module while we transition to a new name.
            'newpkg', 'newpkg.*',
        ]),

so both packages are installed.

Works at a the top level for import newpkg but fails if you from newpkg import subpkg.

My next thought was that I could write a script that for every python file in the old package I autogenerate a dumy python file in the new package with from <oldpkg>.<subname> import *.

Now this functionally works, but it introduces two namespaces and it has the problem that it actually accesses the attributes in the old package. The old package uses lazy imports, which means it uses the module level __getattr__ to expose submodules dynamically without doing the time consuming work of importing them all at startup.

I'm not sure if there is any way around this problem. Really all I want is the user to be able to use oldpkg and newpkg interchangably. In fact it would be really great if import oldpkg, newpkg; oldpkg is newpkg was True.

Is there any way that I can write a newpkg that is a strict alias of oldpkg? If possible I'd like the following statements to be functionally equivalent:

# In bash I'd like the CLI to be the same. The __main__.py should be mirrored
python -m oldpkg
python -m newpkg
# In Python
import oldpkg
import newpkg

assert oldpkg is newpkg

from oldpkg.subpkg import nested as n1
from newpkg.subpkg import nested as n2

assert n1 is n2

Perhaps the above is not possible, but I'm wondering what the best way to go about this would be. I want to go through the following transition phases:

  1. newpkg is just a pointer to oldpkg.
  2. contents move from oldpkg to newpkg and now oldpkg is a pointer to newpkg.
  3. oldpkg now includes a deprecation warning.
  4. oldpkg now errors when it is imported and tells the user to use newpkg.
  5. oldpkg is removed.

Is there any prior art on acomplishing this transition plan?

请注意,我只提供翻译,不包括代码部分。如果需要任何进一步的帮助,请随时提问。

英文:

I'm currently in the middle of renaming a project. In the meantime I need to not break existing users, so I'd like to provide an alias that lets users use it exactly the same as they previously would. This is turning out to be tricker than I initially thought.

Say I have a repo with a package:

├── setup.py
├── pyproject.toml
└── oldpkg
    ├── __init__.py
    ├── __main__.py
    ├── submod.py
    └── subpkg
        ├── __init__.py
        └── nested.py

My first thought was to introduce:

├── newpkg.py

and fill its contents with from oldpkg import *

Then in setup.py add:

        packages=find_packages(include=[
            &#39;oldpkg&#39;, &#39;oldpkg.*&#39;,
            # Alias the module while we transition to a new name.
            &#39;newpkg&#39;, &#39;newpkg.*&#39;,
        ]),

so both packages are installed.

Works at a the top level for import newpkg but fails if you from newpkg import subpkg.

My next thought was that I could write a script that for every python file in the old package I autogenerate a dumy python file in the new package with from &lt;oldpkg&gt;.&lt;subname&gt; import *.

Now this functionally works, but it introduces two namespaces and it has the problem that it actually accesses the attributes in the old package. The old package uses lazy imports, which means it uses the module level __getattr__ to expose submodules dynamically without doing the time consuming work of importing them all at startup.

I'm not sure if there is any way around this problem. Really all I want is the user to be able to use oldpkg and newpkg interchangably. In fact it would be really great if import oldpkg, newpkg; oldpkg is newpkg was True.

Is there any way that I can write a newpkg that is a strict alias of oldpkg? If possible I'd like the following statements to be functionally equivalent:

# In bash I&#39;d like the CLI to be the same. The `__main__.py` should be mirrored
python -m oldpkg
python -m newpkg
# In Python
import oldpkg
import newpkg

assert oldpkg is newpkg

from oldpkg.subpkg import nested as n1
from newpkg.subpkg import nested as n2

assert n1 is n2

Perhaps the above is not possible, but I'm wondering what the best way to go about this would be. I want to go through the following transition phases:

  1. newpkg is just a pointer to oldpkg.
  2. contents move from oldpkg to newpkg and now oldpkg is a pointer to newpkg.
  3. oldpkg now includes a deprecation warning.
  4. oldpkg now errors when it is imported and tells the user to use newpkg.
  5. oldpkg is removed.

Is there any prior art on acomplishing this transition plan?

答案1

得分: 0

以下是翻译好的内容:

我已经创建了一个自动生成处理大多数情况所需的文件的脚本。它在python -m方面仍然存在问题,但我认为我可以找到一种方法使其工作。当我完成后,我会更新答案。

以下脚本创建了另一个模块,其中包含与您的模块完全相同的文件,并添加了一个__getattr__,以便所有这些模块属性都引用回原始模块。

import networkx as nx
import ubelt as ub

new_name = 'NEW_MODULE'
module = ub.import_module_from_name('OLD_MODULE')
module_dpath = ub.Path(module.__file__).parent
repo_dpath = module_dpath.parent

g = nx.DiGraph()
g.add_node(module_dpath, label=module_dpath.name, type='dir')

for root, dnames, fnames in module_dpath.walk():
    if '__init__.py' not in fnames:
        dnames.clear()
        continue

    g.add_node(root, label=root.name, type='dir')
    if root != module_dpath:
        g.add_edge(root.parent, root)

    for f in fnames:
        if f.endswith('.py'):
            fpath = root / f
            g.add_node(fpath, label=fpath.name, type='file')
            g.add_edge(root, fpath)

for p in list(g.nodes):
    node_data = g.nodes

ntype = node_data.get('type', None) if ntype == 'dir': node_data['label'] = ub.color_text(node_data['label'], 'blue') elif ntype == 'file': node_data['label'] = ub.color_text(node_data['label'], 'green') nx.write_network_text(g) for node, node_data in g.nodes(data=True): if node_data['type'] == 'file': relpath = node.relative_to(module_dpath) new_fpath = repo_dpath / new_name / relpath new_fpath.parent.ensuredir() modname = ub.modpath_to_modname(node) print(f'new_fpath={new_fpath}') if new_fpath.name == '__main__.py': new_fpath.write_text(ub.codeblock( f''' from {modname} import * # NOQA ''')) else: new_fpath.write_text(ub.codeblock( f''' # Autogenerated via: # python dev/maintain/mirror_package.py def __getattr__(key): import {modname} as mirror return getattr(mirror, key) '''))

这种方式虽然需要很多样板代码,但它是可行的,并且可以通过脚本进行维护。

英文:

I've come up with a script that autogenerates the files necessary to handle most cases. It still has issues with python -m, but I think I can figure out a way to make it work. I will update the answer when I finish that.

The following script makes a another module with the same exact files as yours and adds a __getattr__ so all of those module attributes refer back ot the original module.

import networkx as nx
import ubelt as ub
new_name = &#39;NEW_MODULE&#39;
module = ub.import_module_from_name(&#39;OLD_MODULE&#39;)
module_dpath = ub.Path(module.__file__).parent
repo_dpath = module_dpath.parent
g = nx.DiGraph()
g.add_node(module_dpath, label=module_dpath.name, type=&#39;dir&#39;)
for root, dnames, fnames in module_dpath.walk():
# dnames[:] = [d for d in dnames if not dname_block_pattern.match(d)]
if &#39;__init__.py&#39; not in fnames:
dnames.clear()
continue
g.add_node(root, label=root.name, type=&#39;dir&#39;)
if root != module_dpath:
g.add_edge(root.parent, root)
# for d in dnames:
#     dpath = root / d
#     g.add_node(dpath, label=dpath.name)
#     g.add_edge(root, dpath)
for f in fnames:
if f.endswith(&#39;.py&#39;):
fpath = root / f
g.add_node(fpath, label=fpath.name, type=&#39;file&#39;)
g.add_edge(root, fpath)
for p in list(g.nodes):
node_data = g.nodes

ntype = node_data.get(&#39;type&#39;, None) if ntype == &#39;dir&#39;: node_data[&#39;label&#39;] = ub.color_text(node_data[&#39;label&#39;], &#39;blue&#39;) elif ntype == &#39;file&#39;: node_data[&#39;label&#39;] = ub.color_text(node_data[&#39;label&#39;], &#39;green&#39;) nx.write_network_text(g) for node, node_data in g.nodes(data=True): if node_data[&#39;type&#39;] == &#39;file&#39;: relpath = node.relative_to(module_dpath) new_fpath = repo_dpath / new_name / relpath new_fpath.parent.ensuredir() modname = ub.modpath_to_modname(node) print(f&#39;new_fpath={new_fpath}&#39;) if new_fpath.name == &#39;__main__.py&#39;: new_fpath.write_text(ub.codeblock( f&#39;&#39;&#39; from {modname} import * # NOQA &#39;&#39;&#39;)) else: new_fpath.write_text(ub.codeblock( f&#39;&#39;&#39; # Autogenerated via: # python dev/maintain/mirror_package.py def __getattr__(key): import {modname} as mirror return getattr(mirror, key) &#39;&#39;&#39;))

It's annoying that this takes so much boilerplate, but it works, and it is maintainable via a script.

huangapple
  • 本文由 发表于 2023年4月6日 22:26:57
  • 转载请务必保留本文链接:https://go.coder-hub.com/75950681.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定