英文:
Exporting a conda environment to a yaml file won't let me rebuild it on the same machine
问题
我有一个可用的conda环境,我想与正在同一计算机集群上工作的同事共享。我将我的环境导出为YAML文件,但然后我无法使用该YAML文件再次创建环境,因为存在不兼容的包。完整的错误消息如下:
无法解决环境规范。
以下软件包不兼容
- alsa-lib 1.2.6.1** 被请求并可安装;
- click 8.1.3** 可以安装,并具有以下潜在选项
- click 8.1.3,可安装;
- click 8.1.3将需要 __win ,而该系统中缺少此项;
- dask 2022.2.0** 无法安装,因为它需要
- distributed >=2022.2.0,<2022.2.1.0a0 ,而这又需要
- click >=6.6,<8.1.0 ,与先前报告的任何可安装版本相冲突;
- openssl !=1.1.1e ,可以安装;
distributed 2022.2.0** ,无法安装 (如前所述);
- distributed >=2022.2.0,<2022.2.1.0a0 ,而这又需要
- gst-plugins-base 1.20.3** 可以安装,并具有以下潜在选项
- gst-plugins-base 1.20.3,可以安装;
- gst-plugins-base 1.20.3将需要
- alsa-lib >=1.2.7.2,<1.2.8.0a0 ,与先前报告的任何可安装版本相冲突;
- icu 69.1** 被请求并可安装;
- openssl 3.0.5** 被请求并可安装;
- qt 5.12.9** 无法安装,因为没有可行的选项
- qt 5.12.9将需要
- openssl <3 ,与先前报告的任何可安装版本相冲突;
- qt 5.12.9将需要
- icu >=67.1,<68.0a0 ,与先前报告的任何可安装版本相冲突;
- qt 5.12.9将需要
- icu >=68.1,<69.0a0 ,与先前报告的任何可安装版本相冲突;
- qt 5.12.9将需要
- gst-plugins-base >=1.18.5,<1.19.0a0 ,但没有可行的选项
- gst-plugins-base 1.18.5与先前报告的任何可安装版本相冲突;
- gst-plugins-base 1.18.5将需要
- alsa-lib >=1.2.3,<1.2.4.0a0 ,与先前报告的任何可安装版本相冲突。
- gst-plugins-base >=1.18.5,<1.19.0a0 ,但没有可行的选项
- qt 5.12.9将需要
为什么会发生这种情况?一个可用的conda环境为什么会有不兼容的包?
- 我从一个可用的conda环境开始。
- 我试图在同一系统上创建环境。
- 我的YAML文件同时包含了默认和conda-forge渠道。
- 我尝试从一个
--no-build
YAML文件和一个完整的YAML文件安装conda环境,但两者都有不兼容的包。
英文:
I have a working conda environment that I want to share with a colleague who is working on the same computer cluster. I exported my environment to a YAML file but then I cannot recreate the environment using that YAML file again due to incompatible packages. The full error message is:
Could not solve for environment specs.
The following packages are incompatible
- alsa-lib 1.2.6.1** is requested and can be installed;
- click 8.1.3** is installable with the potential options
- click 8.1.3, which can be installed;
- click 8.1.3 would require __win , which is missing on the system;
- dask 2022.2.0** is uninstallable because it requires
- distributed >=2022.2.0,<2022.2.1.0a0 , which requires
- click >=6.6,<8.1.0 , which conflicts with any installable versions previously reported;
- openssl !=1.1.1e , which can be installed;
distributed 2022.2.0** , which cannot be installed (as previously explained);
- distributed >=2022.2.0,<2022.2.1.0a0 , which requires
- gst-plugins-base 1.20.3** is installable with the potential options
- gst-plugins-base 1.20.3, which can be installed;
- gst-plugins-base 1.20.3 would require
- alsa-lib >=1.2.7.2,<1.2.8.0a0 , which conflicts with any installable versions previously reported;
- icu 69.1** is requested and can be installed;
- openssl 3.0.5** is requested and can be installed;
- qt 5.12.9** is uninstallable because there are no viable options
- qt 5.12.9 would require
- openssl <3 , which conflicts with any installable versions previously reported;
- qt 5.12.9 would require
- icu >=67.1,<68.0a0 , which conflicts with any installable versions previously reported;
- qt 5.12.9 would require
- icu >=68.1,<69.0a0 , which conflicts with any installable versions previously reported;
- qt 5.12.9 would require
- gst-plugins-base >=1.18.5,<1.19.0a0 but there are no viable options
- gst-plugins-base 1.18.5 conflicts with any installable versions previously reported;
- gst-plugins-base 1.18.5 would require
- alsa-lib >=1.2.3,<1.2.4.0a0 , which conflicts with any installable versions previously reported.
- gst-plugins-base >=1.18.5,<1.19.0a0 but there are no viable options
- qt 5.12.9 would require
Why is this happening? How can a working conda environment have incompatible packages?
- I start with a working conda environment.
- I try to create the environment on the same system.
- My YAML file has both the default and the conda-forge channel.
- I tried installing the conda environment both from a
--no-build
YAML file and a full YAML file but both had incompatible packages.
答案1
得分: 0
conda env export
函数不正确地捕获了可能在改变环境时使用了-c
标志的临时命令的渠道来源。相反,它只是捕获了当前用户配置的渠道(包括活动环境中的任何环境特定的渠道,这不一定是导出的环境!)。此外,解决方案对channel_priority
设置是上下文敏感的。例如,strict
模式可能会导致渠道中的软件包被屏蔽。换句话说,通过一系列临时安装命令,可以以这样的方式切换渠道优先级,以至于最终的软件包集不能从只有一个严格渠道优先级的YAML中安装。
由于您在同一系统上,肯定有一条路径可以重建完全相同的环境。
首先,确保channel_priority
设置为“flexible”。其次,查看环境的conda list
输出,并确保其中列出的所有渠道都包含在YAML中。
在同一系统上工作时可以使用的更严格的重建策略是conda list --explicit
。
长期来看,可以通过始终从YAML定义开始工作来采用更可重复的软件包管理实践。也就是说,避免对环境进行临时变更,而是始终使用conda env {create,update}
命令。
在这个领域的一个较新的解决方案是conda-lock
,它可以预先解决环境并只包含下载所需的URL。
英文:
The conda env export
function doesn’t capture channels of origin properly for any ad hoc commands that may have used a -c
flag when mutating the environment. Rather, it simply captures the current user configuration of channels (including any environment-specific ones in the active environment, which is not necessarily the exported one!). Moreover, solving is context-sensitive to the channel_priority
setting. E.g., strict
mode can cause packages from channels to be masked. That is, it is entirely feasible to go through a series of ad hoc install commands that switch channel priority in such a way that the final set of packages could not be installed from a single YAML with just one strict channel priority.
Since you are on the same system, there should definitely be a path forward to recreating the exact environment.
First, make sure that channel_priority is “flexible”. Second, have a look through the conda list
output for the environment and make sure that all the channels listed there are included in the YAML.
A more strict recreation strategy that can be used when working on the same system is conda list --explicit
.
Longer term, one can adopt more reproducible package management practices by always working from a YAML definition from the start. That is, avoid ad hoc mutations of the environment and instead always use conda env {create,update}
commands.
A newer solution in this space is conda-lock
, which literally presolves environments and includes just the URLs needed for download.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论