无法将文件夹复制到Docker容器中。

huangapple go评论89阅读模式
英文:

can't copy folder into docker container

问题

我有一个名为images的本地文件夹,其中包含许多文件夹和文件。当我运行容器时,出现以下错误:
我执行的命令是:docker run -t -i file-uploader -token=abcdefgh

panic: failed Walk: Failed to walk directory: *fs.PathError lstat ./images/: no such file or directory

goroutine 1 [running]:
main.main()
        /src/main.go:57 +0x357

这是我创建的Dockerfile

FROM golang:1.16
WORKDIR /src
COPY go.sum go.mod ./
RUN go mod download
COPY ./images/ images/
COPY . .
RUN CGO_ENABLED=0 go build -o /bin/app .
ENTRYPOINT ["/bin/app"]

FROM scratch
COPY --from=0 /bin/app /bin/app
ENTRYPOINT ["/bin/app"]

这是程序中的代码:

var (
	token = flag.String("token", "", "user's token for application")
	rootpath = flag.String("rootpath", "./images/","folder path to be uploaded")
)

func main() {
	flag.Parse()

	if *token == "" {
		log.Fatal(Red + "please provide a client token => -token={$token}")
	}

	tokenSource := oauth2.StaticTokenSource(&oauth2.Token{AccessToken: *token})
	oauthClient := oauth2.NewClient(context.TODO(), tokenSource)
	client := putio.NewClient(oauthClient)

	paths := make(chan string)
	var wg = new(sync.WaitGroup)
	for i := 0; i < 20; i++ {
		wg.Add(1)
		go worker(paths, wg, client)
	}
	if err := filepath.Walk(*rootpath, func(path string, info os.FileInfo, err error) error {
		if err != nil {
			return fmt.Errorf("Failed to walk directory: %T %w", err, err)
		}
		if !info.IsDir() {
			paths <- path
		}
		return nil
	}); err != nil {
		panic(fmt.Errorf("failed Walk: %w", err))
	}
	close(paths)
	wg.Wait()
}

如果未提供标志,则其默认值为文件夹本身,即./images/。当我像这样正常运行它:go run main.go -token="abcde"时,它可以正常工作。我对Dockerfile进行了一些更改,多次尝试了以下更改:

  • COPY . .替换为COPY ./images/ /images。它应该自动在/src内创建一个名为/src/images的文件夹,并将主机上的本地文件夹放入其中。但它没有起作用。
  • 我还尝试了COPY . .,相信它会将主机上的所有内容复制到Docker容器中。它也没有起作用。
  • 我将两个COPY命令放在一起。也没有起作用。
  • COPY . ./也没有起作用。

我的项目结构如下:

/file-uploader-cli
     /images
     Dockerfile
     file-uploader-cli(二进制文件)
     带有go.sum的go.mod
     main.go

如何将/images文件夹放入容器并正确运行它?
额外问题:/images文件夹大约有500 MB左右。将该文件夹放入容器中是否是一个好的做法?

我猜可以像docker cp {$folder_name} ${container_id}:/{$path}这样复制一个文件夹,但是它必须是一个正在运行的容器或其他什么东西?我尝试过使用image_id替换container_id进行此操作,但是我得到了一个错误,类似于No such container:path: 12312312312:/

编辑:
问题出在scratch镜像上,这是为了减小大小。然而,当我删除scratch部分时,镜像的大小变为1.1 GB。有没有更简单或更方便的方法来使用images文件夹而不会占用太多空间?

英文:

I have a local folder called images that contains bunch of folders and files within. When I run the container I get the following error:
the command I execute: docker run -t -i file-uploader -token=abcdefgh

panic: failed Walk: Failed to walk directory: *fs.PathError lstat ./images/: no such file or directory

goroutine 1 [running]:
main.main()
        /src/main.go:57 +0x357

Here is the Dockerfile I created:

FROM golang:1.16
WORKDIR /src
COPY go.sum go.mod ./
RUN go mod download
COPY ./images/ images/
COPY . .
RUN CGO_ENABLED=0 go build -o /bin/app .
ENTRYPOINT [&quot;/bin/app&quot;]

FROM scratch
COPY --from=0 /bin/app /bin/app
ENTRYPOINT [&quot;/bin/app&quot;]

And, here is the code in the program:

var (
	token = flag.String(&quot;token&quot;, &quot;&quot;, &quot;user&#39;s token for application&quot;)
	rootpath = flag.String(&quot;rootpath&quot;, &quot;./images/&quot;,&quot;folder path to be uploaded&quot;)
)

func main() {
	flag.Parse()

	if *token == &quot;&quot; {
		log.Fatal(Red + &quot;please provide a client token =&gt; -token={$token}&quot;)
	}

	tokenSource := oauth2.StaticTokenSource(&amp;oauth2.Token{AccessToken: *token})
	oauthClient := oauth2.NewClient(context.TODO(), tokenSource)
	client := putio.NewClient(oauthClient)

	paths := make(chan string)
	var wg = new(sync.WaitGroup)
	for i := 0; i &lt; 20; i++ {
		wg.Add(1)
		go worker(paths, wg, client)
	}
	if err := filepath.Walk(*rootpath, func(path string, info os.FileInfo, err error) error {
		if err != nil {
			return fmt.Errorf(&quot;Failed to walk directory: %T %w&quot;, err, err)
		}
		if !info.IsDir() {
			paths &lt;- path
		}
		return nil
	}); err != nil {
		panic(fmt.Errorf(&quot;failed Walk: %w&quot;, err))
	}
	close(paths)
	wg.Wait()
}
}

If flag is not provided, its default value is the folder itself which is ./images/. When I run this normally like: go run main.go -token=&quot;abcde&quot;, it works properly. I did some changes on Dockerfile. Some of the changes I made and tried again and again.:

  • replacing COPY . . with COPY ./images/ /images. It should automatically creates a folder inside /src like /src/images and get the local folder from host and put into it. It didn't work.
  • I also did try COPY . ., believing that it will copy everything from host into docker container. It didn't work either.
  • I did put 2 COPY command together. Didn't work.
  • COPY . ./ didn't work either.

My structure of the project is as follows:

/file-uploader-cli
     /images
     Dockerfile
     file-uploader-cli (binary)
     go.mod with go.sum
     main.go

How can I put /images folder into container and run it properly?
Extra question: /images folder is approx. 500 MB or something. Is it a good practice to put that folder into a container?

I guess it is possible to copy a folder like docker cp {$folder_name} ${container_id}:/{$path}, but it must be a running container or something? I did try this using image_id replacing the container_id but I got an error like No such container:path: 12312312312:/.

EDIT:
the problem is the scratch image which was done in order to reduce the size. However, when I delete the scratch thing, the size of the image became 1.1 GB. Any easier or convenient way to utilize the images folder without having too much size?

答案1

得分: 1

在构建应用程序时,不需要images,而是在执行应用程序时需要它。将images添加到第一个镜像而不是最终镜像中:

FROM golang:1.16 AS builder
WORKDIR /src
COPY go.sum go.mod ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /bin/app .

FROM scratch
WORKDIR /
COPY images .
COPY --from=builder /bin/app /bin/app
ENTRYPOINT ["/bin/app"]

或者,如果您想动态提供图像,可以在运行时挂载它。

我还建议自动删除Docker容器,除非您确实希望它们保留下来。否则,您最终会有大量的Exited容器。

docker run --rm -it -v /my/path/to/images:/images:ro file-uploader -token=abcdefgh

我还建议将令牌放在环境变量中,以便不会保存在bash历史记录和Docker运行时信息中。

所以,你的意思是不要将应用程序容器化?

我通常不会将Go程序容器化,除非容器适合部署和操作(例如Kubernetes)。大多数语言(如C/C++、Java、Erlang、Python、JavaScript)都需要文件系统提供的重要运行时组件。编译后的程序通常与操作系统的共享库动态链接,基于虚拟机的语言(如Java或Erlang)需要安装和配置虚拟机(它还可能具有运行时依赖项),而解释性语言(如Python、Ruby或JavaScript)则需要整个解释器运行时以及语言库链接到的任何共享库。

然而,Go是一个例外。忽略CGo(我建议尽量避免使用),Go二进制文件是静态链接的,并且对用户空间运行时要求很少。这就是为什么Go二进制文件是少数几个可以在使用FROM scratch构建的容器中正常工作的东西之一。唯一的例外是,Go程序将需要操作系统的CA证书以验证HTTPS服务器和其他使用TLS保护的协议的证书。

除非Docker有助于分发或操作应用程序,否则建议不要将应用程序容器化。

英文:

You don't need images when building your app, you need it when executing your app. Instead of adding images to the first image, add it to the final image:

FROM golang:1.16 AS builder
WORKDIR /src
COPY go.sum go.mod ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /bin/app .

FROM scratch
WORKDIR /
copy images .
COPY --from=builder /bin/app /bin/app
ENTRYPOINT [&quot;/bin/app&quot;]

Or, if you want to provide images dynamically, you could mount it at runtime.

I also recommend removing docker containers automatically unless you actually want them sticking around. Otherwise you end up with lots and lots of Exited containers.

docker run --rm -it -v /my/path/to/images:/images:ro file-uploader -token=abcdefgh

I would also recommend you put your token in an environment variable so its not saved in bash history and docker runtime information.

> So, you're saying don't dockerize the app?

I don't usually containerize Go programs unless containers are a good fit for deployment and operations (eg kubernetes). Most languages like C/C++, Java, Erlang, Python, Javascript, all require significant runtime components provided by the filesystem - compiled ones are usually dynamically linked with shared libraries from the operating system, VM based languages like Java or Erlang require the VM to be installed and configured (and it will likely also have runtime dependencies), and interpreted languages like Python, Ruby, or Javascript require the entire interpreter runtime, as well as any shared libraries the language's libraries are linked to.

Go is an exception to this though. Ignoring CGo (which I recommend avoiding whenever possible), Go binaries are statically linked and have minimal userspace runtime requirements. This is why a Go binary is one of the few things that can acutally work in a container built FROM scratch. The one exception to this, is that the Go program will need CA Certificates from the operating system to validate the certificates on HTTPS servers and other protocols secured with TLS.

I recommend that you don't dockerize the app unless docker is helping you distribute or operate it.

答案2

得分: 1

问题在于第二个构建阶段没有包含images目录。我认为你不能使用scratch基础镜像来实现这个目的,所以我们需要进行更改。如果你担心镜像大小,可以考虑使用Alpine Linux。我已经将你的Dockerfile转换为使用Alpine。

FROM golang:1.16-alpine
WORKDIR /src
COPY go.sum go.mod ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /bin/app .
ENTRYPOINT ["/bin/app"]

FROM alpine:3.15.0
WORKDIR /opt/app
COPY --from=0 /bin/app app
COPY images .
ENTRYPOINT ["/opt/app/app"]

请注意,我将第二个构建阶段中的应用程序路径更改为/opt/app。我这样做是因为在/bin下有一个图像文件夹会很奇怪,而/opt是存储用户应用程序的常见位置。

至于是否应该将代码容器化,这取决于你。Go的一个优点是静态编译和轻松的交叉编译。因此,你可以将二进制文件(和图像)原样分发。

英文:

The problem is that the second build stage does not include the images directory. I don't think you can use the scratch base for this purpose, so we will have to change that. If you are concerned about image size, you should look into into alpine linux. I have converted your Dockerfile to use alpine.

FROM golang:1.16-alpine
WORKDIR /src
COPY go.sum go.mod ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /bin/app .
ENTRYPOINT [&quot;/bin/app&quot;]

FROM alpine:3.15.0
WORKDIR /opt/app
COPY --from=0 /bin/app app
COPY images .
ENTRYPOINT [&quot;/opt/app/app&quot;]

Note that I changed the app path in the second build stage to /opt/app. I did this because it would be odd to have an image folder under /bin. And /opt is a common place to store user applications.

As for whether or not you should containerize your code, that is up to you. One of Go's advantages is static compilation and easy cross-compiling. So you could distribute your binary (and images) as is.

huangapple
  • 本文由 发表于 2021年12月5日 01:20:12
  • 转载请务必保留本文链接:https://go.coder-hub.com/70227858.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定