Bug 207547

Summary: building on arm64 fails with "runtime.cgoIsGoPointer: nosplit stack overflow"
Product: Tools Reporter: thediveo
Component: libcapAssignee: Andrew G. Morgan (morgan)
Status: RESOLVED CODE_FIX    
Severity: normal CC: morgan
Priority: P1    
Hardware: ARM   
OS: Linux   
Kernel Version: 5.3.0 Subsystem:
Regression: No Bisected commit-id:

Description thediveo 2020-05-02 21:15:52 UTC
* HW: Raspberry Pi 4B/4GB
* Kernel: Linux 5.3.0-1022-raspi2 #24-Ubuntu SMP Fri Mar 27 21:32:13 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux
* Distribution: Ubuntu 19.10 server
* libcap: 2.33
* go: version go1.14.2 linux/arm64

"make" fails with:

CGO_ENABLED="1" CGO_LDFLAGS_ALLOW="-Wl,-wrap,.+" CGO_CFLAGS="-I/home/harald/libcap-2.33/libcap/include" CGO_LDFLAGS="-L/home/harald/libcap-2.33/libcap" GOPATH=/home/harald/libcap-2.33/go go build try-launching.go
# command-line-arguments
runtime.cgoIsGoPointer: nosplit stack overflow
        760     assumed on entry to libcap/cap.(*syscaller).iabSetProc (nosplit)
        568     after libcap/cap.(*syscaller).iabSetProc (nosplit) uses 192
        488     after runtime.deferreturn (nosplit) uses 80
        360     after runtime.freedefer (nosplit) uses 128
        120     after runtime.gcWriteBarrier (nosplit) uses 240
        72      after runtime.wbBufFlush (nosplit) uses 48
        8       after runtime.cgoCheckWriteBarrier (nosplit) uses 64
        -24     after runtime.cgoIsGoPointer (nosplit) uses 32
make[1]: *** [Makefile:62: try-launching] Error 2
Comment 1 Andrew G. Morgan 2020-05-03 02:46:00 UTC
Just noting that setting up a debian rpi3 with arm64 from here:

  https://wiki.debian.org/RaspberryPiImages

apt-get installing gcc and golang gets me:

  go version go1.11.6 linux/arm64
  gcc (Debian 8.3.0-6) 8.3.0

libcap seems to build and test just fine with these.

I'll clone the go sources and explore where things went wrong (and also verify
https://go-review.googlesource.com/c/go/+/210639 builds and tests OK ;)
Comment 2 Andrew G. Morgan 2020-05-03 03:47:58 UTC
Sigh. The default install could not build the go sources [*].
I'm trying again after enabling some swap space:

$ sudo -s
# dd if=/dev/zero of=/root/swapfile bs=1024 count=2097152
# chmod 600 /root/swapfile
# mkswap /root/swapfile
# swapon /root/swapfile

[This is a similar workaroud to one I used for the raspbian 32-bit build: https://github.com/golang/go/issues/36309#issuecomment-570957272 ]

[*] rpi3 with the default install cannot build the go sources:

pi@rpi3:~/gits/go/src$ ./all.bash 
Building Go cmd/dist using /usr/lib/go-1.11. (go1.11.6 linux/arm64)
Building Go toolchain1 using /usr/lib/go-1.11.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
# cmd/compile/internal/ssa
fatal error: runtime: out of memory

runtime stack:
runtime.throw(0x53a5f7, 0x16)
        /home/pi/gits/go/src/runtime/panic.go:608 +0x4c
[...]
Comment 3 Andrew G. Morgan 2020-05-03 15:26:57 UTC
So, building from HEAD of the Go sources the libcap build works fine. However, checking out the sources at the go1.14.2 tag, does reproduce this failure.

I'll see if there is a specific fix for the problem in the Go build tree.
Comment 4 Andrew G. Morgan 2020-05-03 18:28:03 UTC
This issue appears to affect all of go1.14, go1.14.1, go1.14.2.

I've not looked further back for where it started, but given that it seems to have gone away in subsequent golang sources, I am tempted to say "golang bug". However, the process of iterating on:

  https://go-review.googlesource.com/c/go/+/210639

has let me learn about a different compilation directive: //go:uintptrescapes, which appears to be more appropriate for my use cases than //go:nosplit in the "libcap/cap package" so I've been looking at using that.

My testing of go1.11.6 (default install on my arm64 rpi3), and go1.14, go1.14.2 and HEAD in the Go sources ( https://go.googlesource.com/go ) suggests this variant of libcap builds reliably on linux-arm64. This change is visible here:

  https://git.kernel.org/pub/scm/libs/libcap/libcap.git/commit/?id=194816c176cf433290c98faeb788516e0d414388

This change will be included in libcap-2.34.
Comment 5 thediveo 2020-05-03 20:50:03 UTC
Compiling the 194816c1 commit also works now on my system. Thank you very much!