将conda环境导出为yaml文件无法在同一台机器上重新构建它。

huangapple go评论74阅读模式
英文:

Exporting a conda environment to a yaml file won't let me rebuild it on the same machine

问题

我有一个可用的conda环境,我想与正在同一计算机集群上工作的同事共享。我将我的环境导出为YAML文件,但然后我无法使用该YAML文件再次创建环境,因为存在不兼容的包。完整的错误消息如下:


无法解决环境规范。

以下软件包不兼容

  • alsa-lib 1.2.6.1** 被请求并可安装;
  • click 8.1.3** 可以安装,并具有以下潜在选项
    • click 8.1.3,可安装;
    • click 8.1.3将需要 __win ,而该系统中缺少此项;
  • dask 2022.2.0** 无法安装,因为它需要
    • distributed >=2022.2.0,<2022.2.1.0a0 ,而这又需要
      • click >=6.6,<8.1.0 ,与先前报告的任何可安装版本相冲突;
      • openssl !=1.1.1e ,可以安装;
        distributed 2022.2.0** ,无法安装 (如前所述);
  • gst-plugins-base 1.20.3** 可以安装,并具有以下潜在选项
    • gst-plugins-base 1.20.3,可以安装;
    • gst-plugins-base 1.20.3将需要
      • alsa-lib >=1.2.7.2,<1.2.8.0a0 ,与先前报告的任何可安装版本相冲突;
  • icu 69.1** 被请求并可安装;
  • openssl 3.0.5** 被请求并可安装;
  • qt 5.12.9** 无法安装,因为没有可行的选项
    • qt 5.12.9将需要
      • openssl <3 ,与先前报告的任何可安装版本相冲突;
    • qt 5.12.9将需要
      • icu >=67.1,<68.0a0 ,与先前报告的任何可安装版本相冲突;
    • qt 5.12.9将需要
      • icu >=68.1,<69.0a0 ,与先前报告的任何可安装版本相冲突;
    • qt 5.12.9将需要
      • gst-plugins-base >=1.18.5,<1.19.0a0 ,但没有可行的选项
        • gst-plugins-base 1.18.5与先前报告的任何可安装版本相冲突;
        • gst-plugins-base 1.18.5将需要
          • alsa-lib >=1.2.3,<1.2.4.0a0 ,与先前报告的任何可安装版本相冲突。

为什么会发生这种情况?一个可用的conda环境为什么会有不兼容的包?

  • 我从一个可用的conda环境开始。
  • 我试图在同一系统上创建环境。
  • 我的YAML文件同时包含了默认和conda-forge渠道。
  • 我尝试从一个--no-build YAML文件和一个完整的YAML文件安装conda环境,但两者都有不兼容的包。
英文:

I have a working conda environment that I want to share with a colleague who is working on the same computer cluster. I exported my environment to a YAML file but then I cannot recreate the environment using that YAML file again due to incompatible packages. The full error message is:


Could not solve for environment specs.

The following packages are incompatible

  • alsa-lib 1.2.6.1** is requested and can be installed;
  • click 8.1.3** is installable with the potential options
    • click 8.1.3, which can be installed;
    • click 8.1.3 would require __win , which is missing on the system;
  • dask 2022.2.0** is uninstallable because it requires
    • distributed >=2022.2.0,<2022.2.1.0a0 , which requires
      • click >=6.6,<8.1.0 , which conflicts with any installable versions previously reported;
      • openssl !=1.1.1e , which can be installed;
        distributed 2022.2.0** , which cannot be installed (as previously explained);
  • gst-plugins-base 1.20.3** is installable with the potential options
    • gst-plugins-base 1.20.3, which can be installed;
    • gst-plugins-base 1.20.3 would require
      • alsa-lib >=1.2.7.2,<1.2.8.0a0 , which conflicts with any installable versions previously reported;
  • icu 69.1** is requested and can be installed;
  • openssl 3.0.5** is requested and can be installed;
  • qt 5.12.9** is uninstallable because there are no viable options
    • qt 5.12.9 would require
      • openssl <3 , which conflicts with any installable versions previously reported;
    • qt 5.12.9 would require
      • icu >=67.1,<68.0a0 , which conflicts with any installable versions previously reported;
    • qt 5.12.9 would require
      • icu >=68.1,<69.0a0 , which conflicts with any installable versions previously reported;
    • qt 5.12.9 would require
      • gst-plugins-base >=1.18.5,<1.19.0a0 but there are no viable options
        • gst-plugins-base 1.18.5 conflicts with any installable versions previously reported;
        • gst-plugins-base 1.18.5 would require
          • alsa-lib >=1.2.3,<1.2.4.0a0 , which conflicts with any installable versions previously reported.

Why is this happening? How can a working conda environment have incompatible packages?

  • I start with a working conda environment.
  • I try to create the environment on the same system.
  • My YAML file has both the default and the conda-forge channel.
  • I tried installing the conda environment both from a --no-build YAML file and a full YAML file but both had incompatible packages.

答案1

得分: 0

conda env export函数不正确地捕获了可能在改变环境时使用了-c标志的临时命令的渠道来源。相反,它只是捕获了当前用户配置的渠道(包括活动环境中的任何环境特定的渠道,这不一定是导出的环境!)。此外,解决方案对channel_priority设置是上下文敏感的。例如,strict模式可能会导致渠道中的软件包被屏蔽。换句话说,通过一系列临时安装命令,可以以这样的方式切换渠道优先级,以至于最终的软件包集不能从只有一个严格渠道优先级的YAML中安装。

由于您在同一系统上,肯定有一条路径可以重建完全相同的环境。

首先,确保channel_priority设置为“flexible”。其次,查看环境的conda list输出,并确保其中列出的所有渠道都包含在YAML中。

在同一系统上工作时可以使用的更严格的重建策略是conda list --explicit


长期来看,可以通过始终从YAML定义开始工作来采用更可重复的软件包管理实践。也就是说,避免对环境进行临时变更,而是始终使用conda env {create,update}命令。

在这个领域的一个较新的解决方案是conda-lock,它可以预先解决环境并只包含下载所需的URL。

英文:

The conda env export function doesn’t capture channels of origin properly for any ad hoc commands that may have used a -c flag when mutating the environment. Rather, it simply captures the current user configuration of channels (including any environment-specific ones in the active environment, which is not necessarily the exported one!). Moreover, solving is context-sensitive to the channel_priority setting. E.g., strict mode can cause packages from channels to be masked. That is, it is entirely feasible to go through a series of ad hoc install commands that switch channel priority in such a way that the final set of packages could not be installed from a single YAML with just one strict channel priority.

Since you are on the same system, there should definitely be a path forward to recreating the exact environment.

First, make sure that channel_priority is “flexible”. Second, have a look through the conda list output for the environment and make sure that all the channels listed there are included in the YAML.

A more strict recreation strategy that can be used when working on the same system is conda list --explicit.


Longer term, one can adopt more reproducible package management practices by always working from a YAML definition from the start. That is, avoid ad hoc mutations of the environment and instead always use conda env {create,update} commands.

A newer solution in this space is conda-lock, which literally presolves environments and includes just the URLs needed for download.

huangapple
  • 本文由 发表于 2023年7月17日 23:03:53
  • 转载请务必保留本文链接:https://go.coder-hub.com/76705783.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定