Created attachment 300446 [details] kernel config On ia64, after 5f501d555653f8968011a1e65ebb121c8b43c144, the gcc binary crashes with SIGSEGV at startup (i.e., during ELF loading). Only gcc exhibits the crash (including g++, etc), other toolchain components (such as ld, ldd, etc) do not, and neither does any other binary from what I can tell. I also haven't observed the issue on any other architecture. Reverting this commit resolves the issue up to and including git tip, with no (visible) issues. Hardware: HP Integrity rx2800 i2 Kernel config attached.
This is probably yet another case of PT_LOAD segments not being ordered by vaddr. See https://lore.kernel.org/linux-fsdevel/YfF18Dy85mCntXrx@fractal.localdomain/ for the fix. You can confirm this by checking the output of "readelf --headers gcc" for program headers of type PT_LOAD with wrong VirtAddr ordering. It would be interesting though, what kind of build configuration causes this weird ordering.
(In reply to magnus.gross from comment #1) > This is probably yet another case of PT_LOAD segments not being ordered by > vaddr. See > https://lore.kernel.org/linux-fsdevel/YfF18Dy85mCntXrx@fractal.localdomain/ > for the fix. > > You can confirm this by checking the output of "readelf --headers gcc" for > program headers of type PT_LOAD with wrong VirtAddr ordering. It would be > interesting though, what kind of build configuration causes this weird > ordering. Hi Magnus, thank you for looking at this. I just tried your patch, and I also tried Alexey's patch, but neither fixed the issue for me. Below is the ELF info from the GCC binary, and also the build-time configuration information from GCC. Thank you for your help! $ readelf --headers /usr/bin/gcc ELF Header: Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 Class: ELF64 Data: 2's complement, little endian Version: 1 (current) OS/ABI: UNIX - System V ABI Version: 0 Type: EXEC (Executable file) Machine: Intel IA-64 Version: 0x1 Entry point address: 0x40000000000f6da0 Start of program headers: 64 (bytes into file) Start of section headers: 1966568 (bytes into file) Flags: 0x10, 64-bit Size of this header: 64 (bytes) Size of program headers: 56 (bytes) Number of program headers: 9 Size of section headers: 64 (bytes) Number of section headers: 33 Section header string table index: 32 Section Headers: [Nr] Name Type Address Offset Size EntSize Flags Link Info Align [ 0] NULL 0000000000000000 00000000 0000000000000000 0000000000000000 0 0 0 [ 1] .interp PROGBITS 4000000000000238 00000238 0000000000000018 0000000000000000 A 0 0 1 [ 2] .note.ABI-tag NOTE 4000000000000250 00000250 0000000000000020 0000000000000000 A 0 0 4 [ 3] .gnu.hash GNU_HASH 4000000000000270 00000270 0000000000000040 0000000000000000 A 4 0 8 [ 4] .dynsym DYNSYM 40000000000002b0 000002b0 0000000000000ea0 0000000000000018 A 5 1 8 [ 5] .dynstr STRTAB 4000000000001150 00001150 00000000000005f7 0000000000000000 A 0 0 1 [ 6] .gnu.version VERSYM 4000000000001748 00001748 0000000000000138 0000000000000002 A 4 0 2 [ 7] .gnu.version_r VERNEED 4000000000001880 00001880 00000000000000e0 0000000000000000 A 5 2 8 [ 8] .rela.dyn RELA 4000000000001960 00001960 0000000000000120 0000000000000018 A 4 0 8 [ 9] .rela.IA_64.[...] RELA 4000000000001a80 00001a80 0000000000000d20 0000000000000018 AI 4 27 8 [10] .init PROGBITS 40000000000027a0 000027a0 0000000000000040 0000000000000000 AX 0 0 16 [11] .plt PROGBITS 40000000000027e0 000027e0 0000000000001a80 0000000000000000 AX 0 0 32 [12] .text PROGBITS 4000000000004280 00004280 00000000001566e0 0000000000000000 AX 0 0 64 [13] .fini PROGBITS 400000000015a960 0015a960 0000000000000040 0000000000000000 AX 0 0 16 [14] .rodata PROGBITS 400000000015a9a0 0015a9a0 0000000000041f88 0000000000000000 A 0 0 8 [15] .opd PROGBITS 400000000019c930 0019c930 0000000000000fd0 0000000000000000 A 0 0 16 [16] .IA_64.unwin[...] PROGBITS 400000000019d900 0019d900 0000000000007ed0 0000000000000000 A 0 0 8 [17] .IA_64.unwind IA_64_UNWIND 40000000001a57d0 001a57d0 00000000000074e8 0000000000000000 AL 12 12 8 [18] .tbss NOBITS 600000000000ccb8 001accb8 0000000000000010 0000000000000000 WAT 0 0 8 [19] .init_array INIT_ARRAY 600000000000ccb8 001accb8 0000000000000030 0000000000000008 WA 0 0 8 [20] .fini_array FINI_ARRAY 600000000000cce8 001acce8 0000000000000008 0000000000000008 WA 0 0 8 [21] .data.rel.ro PROGBITS 600000000000ccf0 001accf0 00000000000311e0 0000000000000000 WA 0 0 8 [22] .dynamic DYNAMIC 600000000003ded0 001dded0 0000000000000200 0000000000000010 WA 5 0 8 [23] .data PROGBITS 600000000003e0d0 001de0d0 0000000000000a80 0000000000000000 WA 0 0 8 [24] .ctors PROGBITS 600000000003eb50 001deb50 0000000000000010 0000000000000000 WA 0 0 8 [25] .dtors PROGBITS 600000000003eb60 001deb60 0000000000000010 0000000000000000 WA 0 0 8 [26] .got PROGBITS 600000000003eb70 001deb70 0000000000000628 0000000000000000 WAp 0 0 8 [27] .IA_64.pltoff PROGBITS 600000000003f1a0 001df1a0 00000000000008c0 0000000000000000 WAp 0 0 16 [28] .sdata PROGBITS 600000000003fa60 001dfa60 0000000000000638 0000000000000000 WAp 0 0 8 [29] .sbss NOBITS 6000000000040098 001e0098 0000000000000378 0000000000000000 WAp 0 0 8 [30] .bss NOBITS 6000000000040410 001e0098 000000000000f578 0000000000000000 WA 0 0 8 [31] .gnu_debuglink PROGBITS 0000000000000000 001e0098 0000000000000028 0000000000000000 0 0 4 [32] .shstrtab STRTAB 0000000000000000 001e00c0 0000000000000127 0000000000000000 0 0 1 Key to Flags: W (write), A (alloc), X (execute), M (merge), S (strings), I (info), L (link order), O (extra OS processing required), G (group), T (TLS), C (compressed), x (unknown), o (OS specific), E (exclude), D (mbind), p (processor specific) Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flags Align PHDR 0x0000000000000040 0x4000000000000040 0x4000000000000040 0x00000000000001f8 0x00000000000001f8 R 0x8 INTERP 0x0000000000000238 0x4000000000000238 0x4000000000000238 0x0000000000000018 0x0000000000000018 R 0x1 [Requesting program interpreter: /lib/ld-linux-ia64.so.2] LOAD 0x0000000000000000 0x4000000000000000 0x4000000000000000 0x00000000001accb8 0x00000000001accb8 R E 0x10000 LOAD 0x00000000001accb8 0x600000000000ccb8 0x600000000000ccb8 0x00000000000333e0 0x0000000000042cd0 RW 0x10000 DYNAMIC 0x00000000001dded0 0x600000000003ded0 0x600000000003ded0 0x0000000000000200 0x0000000000000200 RW 0x8 NOTE 0x0000000000000250 0x4000000000000250 0x4000000000000250 0x0000000000000020 0x0000000000000020 R 0x4 TLS 0x00000000001accb8 0x600000000000ccb8 0x600000000000ccb8 0x0000000000000000 0x0000000000000010 R 0x8 GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 RW 0x10 IA_64_UNWIND 0x00000000001a57d0 0x40000000001a57d0 0x40000000001a57d0 0x00000000000074e8 0x00000000000074e8 R 0x8 Section to Segment mapping: Segment Sections... 00 01 .interp 02 .interp .note.ABI-tag .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_r .rela.dyn .rela.IA_64.pltoff .init .plt .text .fini .rodata .opd .IA_64.unwind_info .IA_64.unwind 03 .init_array .fini_array .data.rel.ro .dynamic .data .ctors .dtors .got .IA_64.pltoff .sdata .sbss .bss 04 .dynamic 05 .note.ABI-tag 06 .tbss 07 08 .IA_64.unwind $ gcc -v Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/libexec/gcc/ia64-unknown-linux-gnu/11.2.1/lto-wrapper Target: ia64-unknown-linux-gnu Configured with: /var/tmp/portage/sys-devel/gcc-11.2.1_p20220115/work/gcc-11-20220115/configure --host=ia64-unknown-linux-gnu --build=ia64-unknown-linux-gnu --prefix=/usr --bindir=/usr/ia64-unknown-linux-gnu/gcc-bin/11.2.1 --includedir=/usr/lib/gcc/ia64-unknown-linux-gnu/11.2.1/include --datadir=/usr/share/gcc-data/ia64-unknown-linux-gnu/11.2.1 --mandir=/usr/share/gcc-data/ia64-unknown-linux-gnu/11.2.1/man --infodir=/usr/share/gcc-data/ia64-unknown-linux-gnu/11.2.1/info --with-gxx-include-dir=/usr/lib/gcc/ia64-unknown-linux-gnu/11.2.1/include/g++-v11 --with-python-dir=/share/gcc-data/ia64-unknown-linux-gnu/11.2.1/python --enable-languages=c,c++,go --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --disable-libunwind-exceptions --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion='Gentoo 11.2.1_p20220115 p4' --disable-esp --enable-libstdcxx-time --disable-libstdcxx-pch --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --disable-multilib --disable-fixed-point --enable-libgomp --disable-libssp --disable-libada --disable-cet --disable-systemtap --disable-valgrind-annotations --disable-vtable-verify --disable-libvtv --without-zstd --enable-lto --with-isl --disable-isl-version-check --disable-libsanitizer --enable-default-pie --enable-default-ssp Thread model: posix Supported LTO compression algorithms: zlib gcc version 11.2.1 20220115 (Gentoo 11.2.1_p20220115 p4) $ gcc -dumpspecs *asm: -x %{mconstant-gp} %{mauto-pic} %(asm_extra) *asm_debug: %{g*:%{%:debug-level-gt(0):%{%:dwarf-version-gt(4):--gdwarf-5;%:dwarf-version-gt(3):--gdwarf-4;%:dwarf-version-gt(2):--gdwarf-3;:--gdwarf2}}} %{fdebug-prefix-map=*:--debug-prefix-map %*} *asm_debug_option: *asm_final: %{gsplit-dwarf: objcopy --extract-dwo %{c:%{o*:%*}%{!o*:%w%b%O}}%{!c:%U%O} %b.dwo objcopy --strip-dwo %{c:%{o*:%*}%{!o*:%w%b%O}}%{!c:%U%O} } *asm_options: %{-target-help:%:print-asm-header()} %{v} %{w:-W} %{I*} %(asm_debug_option) %{gz|gz=zlib:--compress-debug-sections=zlib} %{gz=none:--compress-debug-sections=none} %{gz=zlib-gnu:--compress-debug-sections=zlib-gnu} %a %Y %{c:%W{o*}%{!o*:-o %w%b%O}}%{!c:-o %d%w%u%O} *invoke_as: %{!fwpa*: %{fcompare-debug=*|fdump-final-insns=*:%:compare-debug-dump-opt()} %{!S:-o %|.s | as %(asm_options) %m.s %A } } *cpp: %{posix:-D_POSIX_SOURCE} %{pthread:-D_REENTRANT} *cpp_options: %(cpp_unique_options) %1 %{m*} %{std*&ansi&trigraphs} %{W*&pedantic*} %{w} %{f*} %{g*:%{%:debug-level-gt(0):%{g*} %{!fno-working-directory:-fworking-directory}}} %{O*} %{undef} %{save-temps*:-fpch-preprocess} *cpp_debug_options: %<dumpdir %<dumpbase %<dumpbase-ext %{d*} %:dumps() *cpp_unique_options: %{!Q:-quiet} %{nostdinc*} %{C} %{CC} %{v} %@{I*&F*} %{P} %I %{MD:-MD %{!o:%b.d}%{o*:%.d%*}} %{MMD:-MMD %{!o:%b.d}%{o*:%.d%*}} %{M} %{MM} %{MF*} %{MG} %{MP} %{MQ*} %{MT*} %{Mmodules} %{Mno-modules} %{!E:%{!M:%{!MM:%{!MT:%{!MQ:%{MD|MMD:%{o*:-MQ %*}}}}}}} %{remap} %{%:debug-level-gt(2):-dD} %{!iplugindir*:%{fplugin*:%:find-plugindir()}} %{H} %C %{D*&U*&A*} %{i*} %Z %i %{E|M|MM:%W{o*}} *trad_capable_cpp: cc1 -E %{traditional|traditional-cpp:-traditional-cpp} *cc1: %{profile:-p} %{G*} *cc1_options: %{pg:%{fomit-frame-pointer:%e-pg and -fomit-frame-pointer are incompatible}} %{!iplugindir*:%{fplugin*:%:find-plugindir()}} %1 %{!Q:-quiet} %(cpp_debug_options) %{m*} %{aux-info*} %{g*} %{O*} %{W*&pedantic*} %{w} %{std*&ansi&trigraphs} %{v:-version} %{pg:-p} %{p} %{f*} %{undef} %{Qn:-fno-ident} %{Qy:} %{-help:--help} %{-target-help:--target-help} %{-version:--version} %{-help=*:--help=%*} %{!fsyntax-only:%{S:%W{o*}%{!o*:-o %w%b.s}}} %{fsyntax-only:-o %j} %{-param*} %{coverage:-fprofile-arcs -ftest-coverage} %{fprofile-arcs|fprofile-generate*|coverage: %{!fprofile-update=single: %{pthread:-fprofile-update=prefer-atomic}}} *cc1plus: *link_gcc_c_sequence: %{static|static-pie:--start-group} %G %{!nolibc:%L} %{static|static-pie:--end-group}%{!static:%{!static-pie:%G}} *link_ssp: %{fstack-protector|fstack-protector-all|fstack-protector-strong|fstack-protector-explicit:} *endfile: %{Ofast|ffast-math|funsafe-math-optimizations:crtfastmath.o%s} %{shared|pie:crtendS.o%s;:crtend.o%s} crtn.o%s *link: %{shared:-shared} %{!shared: %{!static: %{rdynamic:-export-dynamic} -dynamic-linker %{muclibc:/lib/ld-uClibc.so.0;:%{mbionic:/system/bin/linker;:%{mmusl:/dev/null;:/lib/ld-linux-ia64.so.2}}}} %{static:-static}} *lib: %{pthread:-lpthread} %{shared:-lc} %{!shared:%{profile:-lc_p}%{!profile:-lc}} *link_gomp: *libgcc: %{static|static-libgcc|static-pie:-lgcc -lgcc_eh %{!static:%{!static-pie:-Bstatic}} -lunwind %{!static:%{!static-pie:-Bdynamic}}}%{!static:%{!static-libgcc:%{!static-pie:%{!shared-libgcc:-lgcc --push-state --as-needed -lgcc_s -lunwind --pop-state}%{shared-libgcc:-lgcc_s -lunwind%{!shared: -lgcc}}}}} *startfile: %{!shared: %{pg|p|profile:gcrt1.o%s;pie:Scrt1.o%s;:crt1.o%s}} crti.o%s %{shared|pie:crtbeginS.o%s;:crtbegin.o%s} *cross_compile: 0 *version: 11.2.1 *multilib: . ; *multilib_defaults: *multilib_extra: *multilib_matches: *multilib_exclusions: *multilib_options: *multilib_reuse: *linker: collect2 *linker_plugin_file: *lto_wrapper: *lto_gcc: *post_link: *link_libgcc: %D *md_exec_prefix: *md_startfile_prefix: *md_startfile_prefix_1: *startfile_prefix_spec: *sysroot_spec: --sysroot=%R *sysroot_suffix_spec: *sysroot_hdrs_suffix_spec: *self_spec: *asm_extra: *link_command: %{!fsyntax-only:%{!c:%{!M:%{!MM:%{!E:%{!S: %(linker) %{!fno-use-linker-plugin:%{!fno-lto: -plugin %(linker_plugin_file) -plugin-opt=%(lto_wrapper) -plugin-opt=-fresolution=%u.res %{flinker-output=*:-plugin-opt=-linker-output-known} %{!nostdlib:%{!nodefaultlibs:%:pass-through-libs(%(link_gcc_c_sequence))}} }}%{flto|flto=*:%<fcompare-debug*} %{flto} %{fno-lto} %{flto=*} %l %{static|shared|r:;!no-pie:-pie} %{fuse-ld=*:-fuse-ld=%*} %{gz|gz=zlib:--compress-debug-sections=zlib} %{gz=none:--compress-debug-sections=none} %{gz=zlib-gnu:--compress-debug-sections=zlib-gnu} %X %{o*} %{e*} %{N} %{n} %{r} %{s} %{t} %{u*} %{z} %{Z} %{!nostdlib:%{!r:%{!nostartfiles:%S}}} %{static|no-pie|static-pie:} %@{L*} %(mfwrap) %(link_libgcc) %{fvtable-verify=none:} %{fvtable-verify=std: %e-fvtable-verify=std is not supported in this configuration} %{fvtable-verify=preinit: %e-fvtable-verify=preinit is not supported in this configuration} %{!nostdlib:%{!r:%{!nodefaultlibs:%{%:sanitize(address):%{!shared:libasan_preinit%O%s} %{static-libasan:%{!shared:-Bstatic --whole-archive -lasan --no-whole-archive -Bdynamic}}%{!static-libasan:-lasan}} %{%:sanitize(hwaddress):%{static-libhwasan:%{!shared:-Bstatic --whole-archive -lhwasan --no-whole-archive -Bdynamic}}%{!static-libhwasan:-lhwasan}} %{%:sanitize(thread):%{!shared:libtsan_preinit%O%s} %{static-libtsan:%{!shared:-Bstatic --whole-archive -ltsan --no-whole-archive -Bdynamic}}%{!static-libtsan:-ltsan}} %{%:sanitize(leak):%{!shared:liblsan_preinit%O%s} %{static-liblsan:%{!shared:-Bstatic --whole-archive -llsan --no-whole-archive -Bdynamic}}%{!static-liblsan:-llsan}}}}} %o %{fopenacc|fopenmp|%:gt(%{ftree-parallelize-loops=*:%*} 1): %:include(libgomp.spec)%(link_gomp)} %{fgnu-tm:%:include(libitm.spec)%(link_itm)} %(mflib) %{fsplit-stack: --wrap=pthread_create} %{fprofile-arcs|fprofile-generate*|coverage:-lgcov} %{!nostdlib:%{!r:%{!nodefaultlibs:%{%:sanitize(address): %{static-libasan|static:%:include(libsanitizer.spec)%(link_libasan)} %{static:%ecannot specify -static with -fsanitize=address}} %{%:sanitize(hwaddress): %{static-libhwasan|static:%:include(libsanitizer.spec)%(link_libhwasan)} %{static:%ecannot specify -static with -fsanitize=hwaddress}} %{%:sanitize(thread): %{static-libtsan|static:%:include(libsanitizer.spec)%(link_libtsan)} %{static:%ecannot specify -static with -fsanitize=thread}} %{%:sanitize(undefined):%{static-libubsan:-Bstatic} -lubsan %{static-libubsan:-Bdynamic} %{static-libubsan|static:%:include(libsanitizer.spec)%(link_libubsan)}} %{%:sanitize(leak): %{static-liblsan|static:%:include(libsanitizer.spec)%(link_liblsan)}}}}} %{!nostdlib:%{!r:%{!nodefaultlibs:%(link_ssp) %(link_gcc_c_sequence)}}} %{!nostdlib:%{!r:%{!nostartfiles:%E}}} %{T*} %(post_link) }}}}}}
Thanks for the detailed report, so it does look like there is indeed a separate problem introduced by that bisected commit. From a quick glance I don't see anything odd with your ELF headers. Would you mind uploading your compiled gcc binary? Just the single gcc binary file will suffice. I should be able to debug the problem swiftly then, as otherwise I would have to wait hours for gcc to compile on my system, thanks!
(In reply to magnus.gross from comment #3) > Thanks for the detailed report, so it does look like there is indeed a > separate problem introduced by that bisected commit. > From a quick glance I don't see anything odd with your ELF headers. > > Would you mind uploading your compiled gcc binary? Just the single gcc > binary file will suffice. I should be able to debug the problem swiftly > then, as otherwise I would have to wait hours for gcc to compile on my > system, thanks! Sure, I have uploaded the entire binary archive here (~175MB): https://synapse.matoro.tk/_matrix/media/r0/download/synapse.matoro.tk/hoCfYesznAVFkBqbniQCvrMt It is zstd-compressed. However, I can also provide live ssh access to the system booted into a bad kernel, would that be more useful? I've already created an account for Kees, one more would be no problem. If so feel free to send me your pubkey at this email address.
This was fixed yesterday in 439a8468242b [1]. > [1] > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=439a8468242b313486e69b8cc3b45ddcfa898fbf