cgo cross compile error stdlib.h not found

background

I'm going to cross compile golang app using go-sqlite3, which push me into "cgo cross compile pit fall". In order to cross compile it to aarch64, the command I used is following, but error:

[root@ec5dc2a3bd37 data_collector]# make crosscompiletry
CGO_ENABLED=1 GOOS=linux GOARCH=arm64 CC=aarch64-linux-gnu-gcc CXX=aarch64-linux-gnu-g++ AR=aarch64-linux-gnu-ar CGO_LDFLAGS="--sysroot=/usr/include:/usr/lib/gcc/aarch64-linux-gnu/4.8.5/include -static" /usr/local/go/bin/go build -o ./output/crosscompile/dcagent ./cmd/agent/main.go
# runtime/cgo
_cgo_export.c:3:20: fatal error: stdlib.h: No such file or directory
 #include <stdlib.h>
                    ^
compilation terminated.
make: *** [crosscompiletry] Error 2

question

How can I fix this error?

Did it means that gccgo can't find includes for aarch64?

I have those includes and develop tools, includes are set by --sysroot, did I do right?

I will dig clues as following while waiting help:

1. I found this step which I didn't do, is this related to my problem?
2. After searched internet still not found solution, most similar situation is this blog, but how to install musl-dev in centos?
3. found a repository xcgo, will try it later.

attachment

include_files

[root@ec5dc2a3bd37 data_collector]# find /usr -regex '.*aarch64.*std.*'
/usr/lib/gcc/aarch64-linux-gnu/4.8.5/include/stdnoreturn.h
/usr/lib/gcc/aarch64-linux-gnu/4.8.5/include/stdarg.h
/usr/lib/gcc/aarch64-linux-gnu/4.8.5/include/stdbool.h
/usr/lib/gcc/aarch64-linux-gnu/4.8.5/include/stdint.h
/usr/lib/gcc/aarch64-linux-gnu/4.8.5/include/stdalign.h
/usr/lib/gcc/aarch64-linux-gnu/4.8.5/include/stddef.h
/usr/lib/gcc/aarch64-linux-gnu/4.8.5/include/stdint-gcc.h
/usr/lib/gcc/aarch64-linux-gnu/4.8.5/include/stdfix.h
[root@ec5dc2a3bd37 data_collector]# find /usr -regex '.*stdlib.*'
/usr/include/c++/4.8.2/tr1/stdlib.h
/usr/include/c++/4.8.2/tr1/cstdlib
/usr/include/c++/4.8.2/cstdlib
/usr/include/stdlib.h
/usr/include/bits/stdlib.h
/usr/include/bits/stdlib-float.h
/usr/include/bits/stdlib-ldbl.h
/usr/local/go/src/go/types/stdlib_test.go

gcc_g++_toolchain

[root@ec5dc2a3bd37 data_collector]# yum groupinstall "Development Tools"
Loaded plugins: fastestmirror, ovl
Loading mirror speeds from cached hostfile
 * base: mirrors.163.com
 * epel: mirror.lzu.edu.cn
 * extras: mirrors.163.com
 * updates: mirrors.163.com
Maybe run: yum groups mark install (see man yum)
No packages in any requested group available to install or update

go_env

[root@ec5dc2a3bd37 data_collector]# go env | grep -v -E 'GOMOD|GONOPROXY|GOPRIVATE|GOPROXY'
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOSUMDB="*"
GOOS="linux"
GOPATH="/root/go"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build259696924=/tmp/go-build -gno-record-gcc-switches"

centos7

[root@ec5dc2a3bd37 data_collector]# cat /etc/*release
CentOS Linux release 7.9.2009 (Core)

sudo apt install --reinstall build-essential

resolved the error for me.

I solved my problem using xcgo. My compile command is :

apt install gcc-8-aarch64-linux-gnu

CGO_ENABLED=1 GOOS=linux GOARCH=arm64 CC=aarch64-linux-gnu-gcc-8 CGO_LDFLAGS="-static" $(GO) build

Return to this question, I think there are some subtle bug about cross compile tool installation in centos, which leads to error reported beyond. If some one saw this, It's recommanded to cross compile your app with xcgo, it support go 1.15. Even if you want build your cross compile docker by your hands, based on ubuntu would be a good idea.

Because 'go build' is a high level compile command, I don't know how to pass '-debug-gcc' to 'go tool cgo' in order to print more debug info. Finding a way to print more debuging info should be a good choice to dive deeper, and I would do it later.

Thanks.