英文:
can't copy folder into docker container
问题
我有一个名为images
的本地文件夹,其中包含许多文件夹和文件。当我运行容器时,出现以下错误:
我执行的命令是:docker run -t -i file-uploader -token=abcdefgh
panic: failed Walk: Failed to walk directory: *fs.PathError lstat ./images/: no such file or directory
goroutine 1 [running]:
main.main()
/src/main.go:57 +0x357
这是我创建的Dockerfile
:
FROM golang:1.16
WORKDIR /src
COPY go.sum go.mod ./
RUN go mod download
COPY ./images/ images/
COPY . .
RUN CGO_ENABLED=0 go build -o /bin/app .
ENTRYPOINT ["/bin/app"]
FROM scratch
COPY --from=0 /bin/app /bin/app
ENTRYPOINT ["/bin/app"]
这是程序中的代码:
var (
token = flag.String("token", "", "user's token for application")
rootpath = flag.String("rootpath", "./images/","folder path to be uploaded")
)
func main() {
flag.Parse()
if *token == "" {
log.Fatal(Red + "please provide a client token => -token={$token}")
}
tokenSource := oauth2.StaticTokenSource(&oauth2.Token{AccessToken: *token})
oauthClient := oauth2.NewClient(context.TODO(), tokenSource)
client := putio.NewClient(oauthClient)
paths := make(chan string)
var wg = new(sync.WaitGroup)
for i := 0; i < 20; i++ {
wg.Add(1)
go worker(paths, wg, client)
}
if err := filepath.Walk(*rootpath, func(path string, info os.FileInfo, err error) error {
if err != nil {
return fmt.Errorf("Failed to walk directory: %T %w", err, err)
}
if !info.IsDir() {
paths <- path
}
return nil
}); err != nil {
panic(fmt.Errorf("failed Walk: %w", err))
}
close(paths)
wg.Wait()
}
如果未提供标志,则其默认值为文件夹本身,即./images/
。当我像这样正常运行它:go run main.go -token="abcde"
时,它可以正常工作。我对Dockerfile
进行了一些更改,多次尝试了以下更改:
- 将
COPY . .
替换为COPY ./images/ /images
。它应该自动在/src
内创建一个名为/src/images
的文件夹,并将主机上的本地文件夹放入其中。但它没有起作用。 - 我还尝试了
COPY . .
,相信它会将主机上的所有内容复制到Docker容器中。它也没有起作用。 - 我将两个
COPY
命令放在一起。也没有起作用。 COPY . ./
也没有起作用。
我的项目结构如下:
/file-uploader-cli
/images
Dockerfile
file-uploader-cli(二进制文件)
带有go.sum的go.mod
main.go
如何将/images
文件夹放入容器并正确运行它?
额外问题:/images
文件夹大约有500 MB左右。将该文件夹放入容器中是否是一个好的做法?
我猜可以像docker cp {$folder_name} ${container_id}:/{$path}
这样复制一个文件夹,但是它必须是一个正在运行的容器或其他什么东西?我尝试过使用image_id
替换container_id
进行此操作,但是我得到了一个错误,类似于No such container:path: 12312312312:/
。
编辑:
问题出在scratch
镜像上,这是为了减小大小。然而,当我删除scratch
部分时,镜像的大小变为1.1 GB。有没有更简单或更方便的方法来使用images
文件夹而不会占用太多空间?
英文:
I have a local folder called images
that contains bunch of folders and files within. When I run the container I get the following error:
the command I execute: docker run -t -i file-uploader -token=abcdefgh
panic: failed Walk: Failed to walk directory: *fs.PathError lstat ./images/: no such file or directory
goroutine 1 [running]:
main.main()
/src/main.go:57 +0x357
Here is the Dockerfile
I created:
FROM golang:1.16
WORKDIR /src
COPY go.sum go.mod ./
RUN go mod download
COPY ./images/ images/
COPY . .
RUN CGO_ENABLED=0 go build -o /bin/app .
ENTRYPOINT ["/bin/app"]
FROM scratch
COPY --from=0 /bin/app /bin/app
ENTRYPOINT ["/bin/app"]
And, here is the code in the program:
var (
token = flag.String("token", "", "user's token for application")
rootpath = flag.String("rootpath", "./images/","folder path to be uploaded")
)
func main() {
flag.Parse()
if *token == "" {
log.Fatal(Red + "please provide a client token => -token={$token}")
}
tokenSource := oauth2.StaticTokenSource(&oauth2.Token{AccessToken: *token})
oauthClient := oauth2.NewClient(context.TODO(), tokenSource)
client := putio.NewClient(oauthClient)
paths := make(chan string)
var wg = new(sync.WaitGroup)
for i := 0; i < 20; i++ {
wg.Add(1)
go worker(paths, wg, client)
}
if err := filepath.Walk(*rootpath, func(path string, info os.FileInfo, err error) error {
if err != nil {
return fmt.Errorf("Failed to walk directory: %T %w", err, err)
}
if !info.IsDir() {
paths <- path
}
return nil
}); err != nil {
panic(fmt.Errorf("failed Walk: %w", err))
}
close(paths)
wg.Wait()
}
}
If flag is not provided, its default value is the folder itself which is ./images/
. When I run this normally like: go run main.go -token="abcde"
, it works properly. I did some changes on Dockerfile
. Some of the changes I made and tried again and again.:
- replacing
COPY . .
withCOPY ./images/ /images
. It should automatically creates a folder inside/src
like/src/images
and get the local folder from host and put into it. It didn't work. - I also did try
COPY . .
, believing that it will copy everything from host into docker container. It didn't work either. - I did put 2
COPY
command together. Didn't work. COPY . ./
didn't work either.
My structure of the project is as follows:
/file-uploader-cli
/images
Dockerfile
file-uploader-cli (binary)
go.mod with go.sum
main.go
How can I put /images
folder into container and run it properly?
Extra question: /images
folder is approx. 500 MB or something. Is it a good practice to put that folder into a container?
I guess it is possible to copy a folder like docker cp {$folder_name} ${container_id}:/{$path}
, but it must be a running container or something? I did try this using image_id
replacing the container_id
but I got an error like No such container:path: 12312312312:/
.
EDIT:
the problem is the scratch
image which was done in order to reduce the size. However, when I delete the scratch
thing, the size of the image became 1.1 GB. Any easier or convenient way to utilize the images
folder without having too much size?
答案1
得分: 1
在构建应用程序时,不需要images
,而是在执行应用程序时需要它。将images
添加到第一个镜像而不是最终镜像中:
FROM golang:1.16 AS builder
WORKDIR /src
COPY go.sum go.mod ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /bin/app .
FROM scratch
WORKDIR /
COPY images .
COPY --from=builder /bin/app /bin/app
ENTRYPOINT ["/bin/app"]
或者,如果您想动态提供图像,可以在运行时挂载它。
我还建议自动删除Docker容器,除非您确实希望它们保留下来。否则,您最终会有大量的Exited
容器。
docker run --rm -it -v /my/path/to/images:/images:ro file-uploader -token=abcdefgh
我还建议将令牌放在环境变量中,以便不会保存在bash历史记录和Docker运行时信息中。
所以,你的意思是不要将应用程序容器化?
我通常不会将Go程序容器化,除非容器适合部署和操作(例如Kubernetes)。大多数语言(如C/C++、Java、Erlang、Python、JavaScript)都需要文件系统提供的重要运行时组件。编译后的程序通常与操作系统的共享库动态链接,基于虚拟机的语言(如Java或Erlang)需要安装和配置虚拟机(它还可能具有运行时依赖项),而解释性语言(如Python、Ruby或JavaScript)则需要整个解释器运行时以及语言库链接到的任何共享库。
然而,Go是一个例外。忽略CGo(我建议尽量避免使用),Go二进制文件是静态链接的,并且对用户空间运行时要求很少。这就是为什么Go二进制文件是少数几个可以在使用FROM scratch
构建的容器中正常工作的东西之一。唯一的例外是,Go程序将需要操作系统的CA证书以验证HTTPS服务器和其他使用TLS保护的协议的证书。
除非Docker有助于分发或操作应用程序,否则建议不要将应用程序容器化。
英文:
You don't need images
when building your app, you need it when executing your app. Instead of adding images
to the first image, add it to the final image:
FROM golang:1.16 AS builder
WORKDIR /src
COPY go.sum go.mod ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /bin/app .
FROM scratch
WORKDIR /
copy images .
COPY --from=builder /bin/app /bin/app
ENTRYPOINT ["/bin/app"]
Or, if you want to provide images dynamically, you could mount it at runtime.
I also recommend removing docker containers automatically unless you actually want them sticking around. Otherwise you end up with lots and lots of Exited
containers.
docker run --rm -it -v /my/path/to/images:/images:ro file-uploader -token=abcdefgh
I would also recommend you put your token in an environment variable so its not saved in bash history and docker runtime information.
> So, you're saying don't dockerize the app?
I don't usually containerize Go programs unless containers are a good fit for deployment and operations (eg kubernetes). Most languages like C/C++, Java, Erlang, Python, Javascript, all require significant runtime components provided by the filesystem - compiled ones are usually dynamically linked with shared libraries from the operating system, VM based languages like Java or Erlang require the VM to be installed and configured (and it will likely also have runtime dependencies), and interpreted languages like Python, Ruby, or Javascript require the entire interpreter runtime, as well as any shared libraries the language's libraries are linked to.
Go is an exception to this though. Ignoring CGo (which I recommend avoiding whenever possible), Go binaries are statically linked and have minimal userspace runtime requirements. This is why a Go binary is one of the few things that can acutally work in a container built FROM scratch
. The one exception to this, is that the Go program will need CA Certificates from the operating system to validate the certificates on HTTPS servers and other protocols secured with TLS.
I recommend that you don't dockerize the app unless docker is helping you distribute or operate it.
答案2
得分: 1
问题在于第二个构建阶段没有包含images
目录。我认为你不能使用scratch
基础镜像来实现这个目的,所以我们需要进行更改。如果你担心镜像大小,可以考虑使用Alpine Linux。我已经将你的Dockerfile转换为使用Alpine。
FROM golang:1.16-alpine
WORKDIR /src
COPY go.sum go.mod ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /bin/app .
ENTRYPOINT ["/bin/app"]
FROM alpine:3.15.0
WORKDIR /opt/app
COPY --from=0 /bin/app app
COPY images .
ENTRYPOINT ["/opt/app/app"]
请注意,我将第二个构建阶段中的应用程序路径更改为/opt/app
。我这样做是因为在/bin
下有一个图像文件夹会很奇怪,而/opt
是存储用户应用程序的常见位置。
至于是否应该将代码容器化,这取决于你。Go的一个优点是静态编译和轻松的交叉编译。因此,你可以将二进制文件(和图像)原样分发。
英文:
The problem is that the second build stage does not include the images
directory. I don't think you can use the scratch
base for this purpose, so we will have to change that. If you are concerned about image size, you should look into into alpine linux. I have converted your Dockerfile to use alpine.
FROM golang:1.16-alpine
WORKDIR /src
COPY go.sum go.mod ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /bin/app .
ENTRYPOINT ["/bin/app"]
FROM alpine:3.15.0
WORKDIR /opt/app
COPY --from=0 /bin/app app
COPY images .
ENTRYPOINT ["/opt/app/app"]
Note that I changed the app path in the second build stage to /opt/app
. I did this because it would be odd to have an image folder under /bin
. And /opt
is a common place to store user applications.
As for whether or not you should containerize your code, that is up to you. One of Go's advantages is static compilation and easy cross-compiling. So you could distribute your binary (and images) as is.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论