Debug Golang in containers with delve: container_linux.go:380: starting container process caused: exec: "/dlv": stat /dlv: no such file or directory

I am trying to containerize a debug version Golang application of my own with Golang debugging delve build in.
Here is how I can debug my Golang application locally , it is a very simple RSS reader. It retrieves data from RSS feed I am interested in.

$on my local terminal$ dlv debug parsedata-xml-fp.go  # launch my app with delve
Type 'help' for list of commands.
(dlv) b main
Command failed: Location "main" ambiguous: main.main, runtime.main…
(dlv) b main.main
Breakpoint 1 set at 0x760252 for main.main() ./parsedata-xml-fp.go:50
(dlv) c
> main.main() ./parsedata-xml-fp.go:50 (hits goroutine(1):1 total:1) (PC: 0x760252)
=>  50: func main() {
    51:         // [decode from response.Body]
    52:         url := "https://foreignpolicy.com/feed/"
    53:
    54:         var URLset Rss
    55:         if xmlBytes, err := getXML(url); err != nil {
(dlv) l
> main.main() ./parsedata-xml-fp.go:50 (hits goroutine(1):1 total:1) (PC: 0x760252)
=>  50: func main() {
    51:         // [decode from response.Body]
    52:         url := "https://foreignpolicy.com/feed/"
    53:
    54:         var URLset Rss
    55:         if xmlBytes, err := getXML(url); err != nil {
(dlv) 

On my local machine I can set breakpoints and step into functions I am interested in.

I was trying to do the same inside a container I built.

Option #1:
Below is the Dockerfile of my container

#Dockerfile.dlv
FROM golang:1.17 AS build

WORKDIR /
COPY go/app/parsedata-xml-fp.go .
COPY go.mod .
COPY go.sum .
    
RUN go install github.com/go-delve/delve/cmd/dlv@latest
RUN go build -gcflags="all=-N -l" -o /feedme
RUN echo $(ls /go/bin)

# stage 2 build 
FROM ubuntu:18.04
WORKDIR /

EXPOSE 2345 

COPY --from=build /go/bin/dlv /dlv
COPY --from=build /feedme /feedme
COPY --from=build /parsedata-xml-fp.go /parsedata-xml-fp.go
CMD ["/dlv", "--listen=:2345", "--headless=true", "--api-version=2", "--accept-multiclient", "exec", "/feedme"]

When bootup my container and login , I go the error :

exec: "go": executable file not found in $PATH

Below is the complete log in my containter

sudo docker exec -it b1494552ef1d /bin/sh
# which dlv
# ls
bin   dev  etc     home  lib64  mnt  parsedata-xml-fp.go  root  sbin  sys  usr
boot  dlv  feedme  lib   media  opt  proc                 run   srv   tmp  var
# ./dlv
Delve is a source level debugger for Go programs.
  ......... # dismiss delve help info , just to confirm dlv is installed 

Use "dlv [command] --help" for more information about a command.
# ./dlv debug parsedata-xml-fp.go
exec: "go": executable file not found in $PATH
# which go
# (nothing)

My understanding is ubuntu1804 has no go installed ? Then I tried to use go docker image only

Option #2

Updated Dockerfile as below : use golang:1.17 as base image (Go should be there):

# Dockerfile.localmod
FROM golang:1.17 AS build

WORKDIR /
COPY go/app/parsedata-xml-fp.go .
COPY go.mod .
COPY go.sum .

RUN echo $(which shell)

RUN go install github.com/go-delve/delve/cmd/dlv@latest
RUN go build -gcflags="all=-N -l" -o /feedme
RUN echo $(ls /go/bin)

EXPOSE 2345 
CMD ["/dlv", "--listen=:2345", "--headless=true", "--api-version=2", "--accept-multiclient", "exec", "/feedme"]

This time error occurred right at I boot my container

sudo docker run 734129d1b1a2                     
docker: Error response from daemon: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "/dlv": stat /dlv: no such file or directory: unknown.
ERRO[0000] error waiting for container: context canceled

Anyone could suggest what is the right way of intergrate delve into my Go container and debugging it on container terminal exactly as debug locally?

The problem is with your dlv binary being dynamically compiled. When you download the binary with go install, by default it downloads with CGO_ENABLED=1 (unless overriden), requiring most of the runtime libraries (including glibc) to be loaded at runtime. This might not work well in some container images, where the libraries are not present (e.g. images built from scratch/distro-less static images).

So to avoid the dependency with the container image's dependencies, always download a statically compiled one by setting the above flag to 0. Use the downloaded binary in your docker context

CGO_ENABLED=0 go install github.com/go-delve/delve/cmd/dlv@latest

You can also observe the ldd output on dlv between the statically and dynamically compiled versions. The former would list no libraries that would need to be dynamically loaded, the latter would list them.