Golang fails to link an aarch64/arm64 binary on an x86_64 machine while cross compiling

I am trying to cross compile https://github.com/joohoi/acme-dns for an aarch64 machine on my x86_64 desktop.

$ CC=aarch64-linux-gnu-gcc GOOS=linux GOARCH=arm64 CGO_ENABLED=1 go build -v -ldflags="-extld=$CC"
# github.com/mattn/go-sqlite3
sqlite3-binding.c: In function ‘sqlite3SelectNew’:
sqlite3-binding.c:125322:10: warning: function may return address of local variable [-Wreturn-local-addr]
125322 |   return pNew;
       |          ^~~~
sqlite3-binding.c:125282:10: note: declared here
125282 |   Select standin;
       |          ^~~~~~~
# github.com/joohoi/acme-dns
/usr/lib/go-1.15/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: /tmp/go-link-266874795/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-266874795/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-266874795/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-266874795/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-266874795/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-266874795/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-266874795/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-266874795/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-266874795/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-266874795/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-266874795/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-266874795/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-266874795/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-266874795/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-266874795/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-266874795/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-266874795/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-266874795/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-266874795/go.o: error adding symbols: file in wrong format
collect2: error: ld returned 1 exit status

CC=aarch64-linux-gnu-gcc GOOS=linux GOARCH=arm64 CGO_ENABLED=1 go env
GO111MODULE=""
GOARCH="arm64"
GOBIN=""
GOCACHE="/home/voltagex/.cache/go-build"
GOENV="/home/voltagex/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/voltagex/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/voltagex/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go-1.15"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.15/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="aarch64-linux-gnu-gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/btrfs/src/acme-dns/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build156138713=/tmp/go-build -gno-record-gcc-switches"

go env looks correct, except for GOTOOLDIR - I understand this is a calculated field.

go-sqlite3 itself seems to cross compile correctly.

I have tried this with golang 1.15 and 1.17.1.

Host OS is Debian 11, gcc 10.2.1

Two questions:

1. How do I get the correct arm64 linker? I guess I'd need to cross compile this too?
2. Why is GOTOOLPATH pointing to the wrong location while cross compiling and how do I fix this?

Problem reproduced, and resolved by replacing -ldflags="-extld=$CC" with -ldflags="-extld=aarch64-linux-gnu-gcc".

Alternatively, you can also export the CC variable beforehand.

The error output was caused by mismatching linker (with your original build command, it was still the x86-64 linker that got invoked).

Tested on two hosts of mine: one Ubuntu 20.04 + go1.13, the other Ubuntu 18.04 + go1.16.

More explanations:

Seems that the in-line CC env variable setting is passed to the go tool, but not used in the shell's parameter substitution. The following output (Bash 5.0) demonstrates this:

anna@LAPTOP-KV4759EJ:~/git/github.com/joohoi/acme-dns$ CC=123 echo $CC

anna@LAPTOP-KV4759EJ:~/git/github.com/joohoi/acme-dns$ export CC=123; echo $CC
123

Note how the first echo does not produce any output.

> Inspired by https://stackoverflow.com/questions/23185341/relocations-in-generic-elf-em-40