Golang REST API Deployment on AWS EKS Fails with CrashLoopBackOff

I'm trying to deploy a simple REST API written in Golang to AWS EKS.

I created an EKS cluster on AWS using Terraform and applied the AWS load balancer controller Helm chart to it.

All resources in the cluster look like:

NAMESPACE     NAME                                                READY   STATUS    RESTARTS   AGE
kube-system   pod/aws-load-balancer-controller-5947f7c854-fgwk2   1/1     Running   0          75m
kube-system   pod/aws-load-balancer-controller-5947f7c854-gkttb   1/1     Running   0          75m
kube-system   pod/aws-node-dfc7r                                  1/1     Running   0          120m
kube-system   pod/aws-node-hpn4z                                  1/1     Running   0          120m
kube-system   pod/aws-node-s6mng                                  1/1     Running   0          120m
kube-system   pod/coredns-66cb55d4f4-5l7vm                        1/1     Running   0          127m
kube-system   pod/coredns-66cb55d4f4-frk6p                        1/1     Running   0          127m
kube-system   pod/kube-proxy-6ndf5                                1/1     Running   0          120m
kube-system   pod/kube-proxy-s95qk                                1/1     Running   0          120m
kube-system   pod/kube-proxy-vdrdd                                1/1     Running   0          120m

NAMESPACE     NAME                                        TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)         AGE
default       service/kubernetes                          ClusterIP   10.100.0.1      <none>        443/TCP         127m
kube-system   service/aws-load-balancer-webhook-service   ClusterIP   10.100.202.90   <none>        443/TCP         75m
kube-system   service/kube-dns                            ClusterIP   10.100.0.10     <none>        53/UDP,53/TCP   127m

NAMESPACE     NAME                        DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
kube-system   daemonset.apps/aws-node     3         3         3       3            3           <none>          127m
kube-system   daemonset.apps/kube-proxy   3         3         3       3            3           <none>          127m

NAMESPACE     NAME                                           READY   UP-TO-DATE   AVAILABLE   AGE
kube-system   deployment.apps/aws-load-balancer-controller   2/2     2            2           75m
kube-system   deployment.apps/coredns                        2/2     2            2           127m

NAMESPACE     NAME                                                      DESIRED   CURRENT   READY   AGE
kube-system   replicaset.apps/aws-load-balancer-controller-5947f7c854   2         2         2       75m
kube-system   replicaset.apps/coredns-66cb55d4f4                        2         2         2       127m

I can run the application locally with Go and with Docker. But releasing this on AWS EKS always throws CrashLoopBackOff.

Running kubectl describe pod PODNAME shows:

Name:         go-api-55d74b9546-dkk9g
Namespace:    default
Priority:     0
Node:         ip-172-16-1-191.ec2.internal/172.16.1.191
Start Time:   Tue, 15 Mar 2022 07:04:08 -0700
Labels:       app=go-api
              pod-template-hash=55d74b9546
Annotations:  kubernetes.io/psp: eks.privileged
Status:       Running
IP:           172.16.1.195
IPs:
  IP:           172.16.1.195
Controlled By:  ReplicaSet/go-api-55d74b9546
Containers:
  go-api:
    Container ID:   docker://a4bc07b60c85fd308157d967d2d0d688d8eeccfe4c829102eb929ca82fb25595
    Image:          saurabhmish/golang-hello:latest
    Image ID:       docker-pullable://saurabhmish/golang-hello@sha256:f79a495ad17710b569136f611ae3c8191173400e2cbb9cfe416e75e2af6f7874
    Port:           3000/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Tue, 15 Mar 2022 07:09:50 -0700
      Finished:     Tue, 15 Mar 2022 07:09:50 -0700
    Ready:          False
    Restart Count:  6
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-jt4gp (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  kube-api-access-jt4gp:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason     Age                     From               Message
  ----     ------     ----                    ----               -------
  Normal   Scheduled  7m31s                   default-scheduler  Successfully assigned default/go-api-55d74b9546-dkk9g to ip-172-16-1-191.ec2.internal
  Normal   Pulled     7m17s                   kubelet            Successfully pulled image "saurabhmish/golang-hello:latest" in 12.77458991s
  Normal   Pulled     7m16s                   kubelet            Successfully pulled image "saurabhmish/golang-hello:latest" in 110.127771ms
  Normal   Pulled     7m3s                    kubelet            Successfully pulled image "saurabhmish/golang-hello:latest" in 109.617419ms
  Normal   Created    6m37s (x4 over 7m17s)   kubelet            Created container go-api
  Normal   Started    6m37s (x4 over 7m17s)   kubelet            Started container go-api
  Normal   Pulled     6m37s                   kubelet            Successfully pulled image "saurabhmish/golang-hello:latest" in 218.952336ms
  Normal   Pulling    5m56s (x5 over 7m30s)   kubelet            Pulling image "saurabhmish/golang-hello:latest"
  Normal   Pulled     5m56s                   kubelet            Successfully pulled image "saurabhmish/golang-hello:latest" in 108.105083ms
  Warning  BackOff    2m28s (x24 over 7m15s)  kubelet            Back-off restarting failed container

Running kubectl logs PODNAME and kubectl logs PODNAME -c go-api shows standard_init_linux.go:228: exec user process caused: exec format error

Manifests:

go-deploy.yaml ( This is the Docker Hub Image with documentation )

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: go-api
  labels:
    app: go-api
spec:
  replicas: 2
  selector:
    matchLabels:
      app: go-api
  strategy: {}
  template:
    metadata:
      labels:
        app: go-api
    spec:
      containers:
      - name: go-api
        image: saurabhmish/golang-hello:latest
        ports:
          - containerPort: 3000
        resources: {}

go-service.yaml

---
kind: Service
apiVersion: v1
metadata:
  name: go-api
spec:
  selector:
    app: go-api
  type: NodePort
  ports:
  - protocol: TCP
    port: 80
    targetPort: 3000

How can I fix this error ?

Posting this as Community wiki for better visibility.
Feel free to expand it.

Thanks to @David Maze, who pointed to the solution. There is an article 'Build Intel64-compatible Docker images from Mac M1 (ARM)' (by Beppe Catanese) here.
This article describes the underlying problem well.

You are developing/building on the ARM architecture (Mac M1), but you deploy the docker image to a x86-64 architecture based Kubernetes cluster.

Solution:

Option A: use buildx

Buildx is a Docker plugin that allows, amongst other features, to build images for various target platforms.

$ docker buildx build --platform linux/amd64 -t myapp .

Option B: set DOCKER_DEFAULT_PLATFORM

The DOCKER_DEFAULT_PLATFORM environment variable permits to set the default platform for the commands that take the --platform flag.

export DOCKER_DEFAULT_PLATFORM=linux/amd64

A CrashloopBackOff means that you have a pod starting, crashing, starting again, and then crashing again.

Maybe the error come from the application itself that it can not connect to database, redis,...

You may find something useful here:

https://stackoverflow.com/questions/41604499/my-kubernetes-pods-keep-crashing-with-crashloopbackoff-but-i-cant-find-any-lo