Python script file missing in Singularity image

huangapple
go评论96阅读模式
英文:

Python script file missing in Singularity image

问题

在AWS中，创建了一个包含Python脚本以打印字符串的Docker镜像（basicprint.py）。

Docker文件：

  1. FROM python
  3. COPY ./basicprint.py ./
  4. CMD ["python", "basicprint.py"]

它正常工作，然后将Docker镜像保存为.tgz文件。

将该.tgz文件复制到本地。

我使用以下命令将Docker镜像（.tgz）转换为Singularity镜像：

  1. singularity build sing_image.sif docker-archive://filename.tgz

成功创建了sing_image.sif。

然后运行：

  1. singularity run sing_image.sif

它报错：basicprint.py: No such file or directory。

有关正确的转换方法而不会丢失文件的建议。

英文:

In AWS, created a docker image with a python script to print a string(basicprint.py)

docker file:

  1. FROM python
  3. COPY ./basicprint.py ./
  4. CMD ["python", "basicprint.py"]

It works fine then saved docker image as .tgz file.

copy that .tgz file in to my local.

I converted docker image(.tgz) into singularity image by using

  1. singularity build sing_image.sif docker-archive://filename.tgz

it was successfully created sing_image.sif

  1. singularity run sing_image.sif

It throws an error : basicprint.py: No such file or directory

Any suggestions on correct method of conversion without missing the file.

答案1

得分: 2

请注意，您将文件放在了根目录下。singularity 在转换和清理过程中删除了您的文件。已经有类似的问题报告，一般情况下，这是在使用容器时预期发生的。

这稍微修改的镜像

  1. FROM python:3.11-slim
  2. COPY ./basicprint.py ./
  3. CMD ["ls"]

演示了文件位于Linux根目录下：

  1. (base) maciej.skorski@kaggle-cpu-maciej:~/docker-debug$ docker build . --tag test-sing
  2. (base) maciej.skorski@kaggle-cpu-maciej:~/docker-debug$ docker run -it test-sing
  3. root@ed8e3545a00b:/# ls
  4. basicprint.py  bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var

现在，让我演示工作版本。重要的是，遵循最佳的Docker实践并使用工作目录

  1. # Dockerfile
  2. FROM python:3.11-slim-bullseye
  4. RUN mkdir -p /usr/src
  5. WORKDIR /usr/src
  6. COPY ./basicprint.py /usr/src/
  8. CMD ["python", "/usr/src/basicprint.py"]

其中示例Python脚本是

  1. # basicprint.py
  2. print('Welcome to my Docker!')

然后我在Debian GNU/Linux 10上使用Docker 20.10.17和Singularity 3.11进行构建和测试：

  1. (base) maciej.skorski@kaggle-cpu-maciej:~/docker-debug$ singularity --version
  2. singularity-ce version 3.11.0+277-gcd6fa5c0d
  4. (base) maciej.skorski@kaggle-cpu-maciej:~/docker-debug$ sudo docker build . --tag test-sing:latest
  5. Successfully tagged test-sing:latest
  7. (base) maciej.skorski@kaggle-cpu-maciej:~/docker-debug$ docker save test-sing:latest --output test-sing.tgz
  9. (base) maciej.skorski@kaggle-cpu-maciej:~/docker-debug$ sudo singularity build test-sing.sif docker-archive:test-sing.tgz
  10. ...
  11. INFO:    Build complete: test-sing.sif
  12. (base) maciej.skorski@kaggle-cpu-maciej:~/docker-debug$ sudo singularity run test-sing.sif
  13. Welcome to my Docker!

在相同的软件环境下，我重现了您的错误。所以我认为它已经解决了！如果您需要进一步的帮助，我们可以设置一个共享的虚拟机，使用GitHub Codespace来使其完全可复制。

英文:

Note that you put your file under the very root. singularity removed your file during conversion and cleaning. Similar issues have been reported and are, in general, expected when working with containers.

This slightly modified image

  1. FROM python:3.11-slim
  2. COPY ./basicprint.py ./
  3. CMD ["ls"]

demonstrates that the file lands under the linux root:

  1. (base) maciej.skorski@kaggle-cpu-maciej:~/docker-debug$ docker build . --tag test-sing
  2. (base) maciej.skorski@kaggle-cpu-maciej:~/docker-debug$ docker run -it test-sing
  3. root@ed8e3545a00b:/# ls
  4. basicprint.py  bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var

Now, let me demonstrate the working version. Importantly, follow best Docker practices and use the working dir

  1. # Dockerfile
  2. FROM python:3.11-slim-bullseye
  4. RUN mkdir -p /usr/src
  5. WORKDIR /usr/src
  6. COPY ./basicprint.py /usr/src/
  8. CMD ["python", "/usr/src/basicprint.py"]

where the sample Python script is

  1. # basicprint.py
  2. print('Welcome to my Docker!')

Then I build and test on Debian GNU/Linux 10 with Docker 20.10.17 and Singularity 3.11:

  1. (base) maciej.skorski@kaggle-cpu-maciej:~/docker-debug$ singularity --version
  2. singularity-ce version 3.11.0+277-gcd6fa5c0d
  4. (base) maciej.skorski@kaggle-cpu-maciej:~/docker-debug$ sudo docker build . --tag test-sing:latest
  5. Successfully tagged test-sing:latest
  7. (base) maciej.skorski@kaggle-cpu-maciej:~/docker-debug$ docker save test-sing:latest --output test-sing.tgz
  9. (base) maciej.skorski@kaggle-cpu-maciej:~/docker-debug$ sudo singularity build test-sing.sif docker-archive:test-sing.tgz
  10. ...
  11. INFO:    Build complete: test-sing.sif
  12. (base) maciej.skorski@kaggle-cpu-maciej:~/docker-debug$ sudo singularity run test-sing.sif
  13. Welcome to my Docker!

Under the same software, I reproduced your error. So I think it has been solved! If you need any further assistance, we can set up a shared Virtual Machine with GitHub Codespace to make it fully reproducible.

答案2

得分: 1

以下是翻译好的部分：

用于测试，您可以使用 def 文件，使用 %file 代替 Dockerfile：

> %files 部分允许您以比使用 %setup 部分更安全的方式将文件复制到容器中。

在您的情况下，创建一个名为 Singularity.def 的文件，其中包含：

  1. Bootstrap: docker
  2. Registry: docker.io
  3. Namespace: library
  4. From: python
  6. %files
  7.     basicprint.py /opt/
  9. %runscript
  10.     exec python /opt/basicprint.py

检查该方法是否有效：

  1. singularity build sing_image.sif Singularity.def
  2. singularity run sing_image.sif

关于您的 Dockerfile，请尝试使用不同的文件夹，再次用于测试：

  1. FROM python
  3. COPY ./basicprint.py /app/
  4. CMD ["python", "/app/basicprint.py"]

然后：

  1. docker build -t my-python-image .
  2. docker save my-python-image:latest | gzip > my-python-image.tgz
  3. singularity build sing_image.sif docker-archive://my-python-image.tgz
  4. singularity run sing_image.sif

还要检查 .sif 文件是否包含您的文件：singularity exec sing_image.sif ls /app。

英文:

For testing, you could use a def file, using %file instead of the Dockerfile:

> The %files section allows you to copy files into the container with greater safety than using the %setup section.

In your case, create a Singularity.def file with:

  1. Bootstrap: docker
  2. Registry: docker.io
  3. Namespace: library
  4. From: python
  6. %files
  7.     basicprint.py /opt/
  9. %runscript
  10.     exec python /opt/basicprint.py

Check if that approach is working with:

  1. singularity build sing_image.sif Singularity.def
  2. singularity run sing_image.sif

Regarding your Dockerfile, try and use a different folder, again for testing:

  1. FROM python
  3. COPY ./basicprint.py /app/
  4. CMD ["python", "/app/basicprint.py"]

Then:

  1. docker build -t my-python-image .
  2. docker save my-python-image:latest | gzip > my-python-image.tgz
  3. singularity build sing_image.sif docker-archive://my-python-image.tgz
  4. singularity run sing_image.sif

Check also the .sif file includes your file: singularity exec sing_image.sif ls /app.

huangapple
  • 本文由 发表于 2023年5月10日 15:03:46
  • 转载请务必保留本文链接：https://go.coder-hub.com/76215725.html
  • amazon-web-services
  • dockerfile
  • python
  • singularity-container
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定