Bug 72841

Summary: Kernel hangs with "rcu_sched detected stalls on CPUs/tasks" messages if maxcpus > 1 on sparc64
Product: Platform Specific/Hardware Reporter: Adrien Dessemond (adrien.dessemond)
Component: SPARC64Assignee: platform_sparc64
Status: RESOLVED CODE_FIX    
Severity: blocking    
Priority: P1    
Hardware: Sparc64   
OS: Linux   
Kernel Version: All kernels > 3.9.11 including 3.14-rc series (3.14-rc7) Subsystem:
Regression: Yes Bisected commit-id:
Attachments: full startup log
kernel configuration

Description Adrien Dessemond 2014-03-24 01:15:34 UTC 
On a SunFire SPARC T1000 machine the kernel stalls on boot with every kernel > 3.9.11 including the latest 3.14-rc7. Tried with various configurations (SLAB / SLUB, Perf counters inactive, no tickless kernel...). If booted with "maxcpus=1", no stall and the kernel boots normally. 

Synmptoms seems similar to bug #61501. 


SILO Version 1.4.14_git20120819_p1

Allocated 64 Megs of memory at 0x40000000 for kernel
Loaded kernel version 3.14.0

[    0.000000] PROMLIB: Sun IEEE Boot Prom 'OBP 4.30.4.d 2011/07/06 14:27'
[    0.000000] PROMLIB: Root node compatible: sun4v
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 3.14.0-rc7 (root@xenon.universe.lan) (gcc version 4.5.3 (Gentoo 4.5.3 p1.4) ) #4 SMP Sun Mar 23 20:39:45 EDT 2014
[    0.000000] bootconsole [earlyprom0] enabled
[    0.000000] ARCH: SUN4V
[    0.000000] Ethernet address: 00:14:4f:6e:cd:36
[    0.000000] PAGE_OFFSET is 0xffffff8000000000 (max_phys_bits == 39)
[    0.000000] Kernel: Using 4 locked TLB entries for main kernel image.
[    0.000000] Remapping the kernel... done.
[    0.000000] OF stdout device is: /virtual-devices@100/console@1
[    0.000000] PROM: Built device tree with 73491 bytes of memory.
[    0.000000] MDESC: Size is 38928 bytes.
[    0.000000] PLATFORM: banner-name [Sun Fire(TM) T1000]
[    0.000000] PLATFORM: name [SUNW,Sun-Fire-T1000]
[    0.000000] PLATFORM: hostid [846ecd36]
[    0.000000] PLATFORM: serial# [00ab4130]
[    0.000000] PLATFORM: stick-frequency [3b9aca00]
[    0.000000] PLATFORM: mac-address [144f6ecd36]
[    0.000000] PLATFORM: watchdog-resolution [1000 ms]
[    0.000000] PLATFORM: watchdog-max-timeout [31536000000 ms]
[    0.000000] PLATFORM: max-cpus [32]
[    0.000000] Top of RAM: 0x1ffd32000, Total RAM: 0x1f7922000
[    0.000000] Memory hole size: 132MB
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x08400000-0x1ffd31fff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x08400000-0x1ffd01fff]
[    0.000000]   node   0: [mem 0x1ffd12000-0x1ffd31fff]
[    0.000000] Booting Linux...
[    0.000000] CPU CAPS: [flush,stbar,swap,muldiv,v9,blkinit,mul32,div32]
[    0.000000] CPU CAPS: [v8plus,ASIBlkInit]
[    0.000000] PERCPU: Embedded 6 pages/cpu @ffffff81fb400000 s13888 r8192 d27072 u131072
[    0.000000] SUN4V: Mondo queue sizes [cpu(4096) dev(16384) r(8192) nr(256)]
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.  Total pages: 1023255
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: root=/dev/sda4
[    0.000000] PID hash table entries: 4096 (order: 2, 32768 bytes)
[    0.000000] Sorting __ex_table...
[    0.000000] Memory: 8169472K/8250504K available (4159K kernel code, 341K rwdata, 1072K rodata, 192K init, 6691K bss, 81032K reserved)
[    0.000000] Hierarchical RCU implementation.
[    0.000000] NR_IRQS:255
[   65.616593] clocksource: mult[800000] shift[23]
[   65.616955] clockevent: mult[80000000] shift[31]
[   65.617902] Console: colour dummy device 80x25
[   65.618259] console [tty0] enabled
[   65.618577] bootconsole [earlyprom0] disabled
[    0.000000] PROMLIB: Sun IEEE Boot Prom 'OBP 4.30.4.d 2011/07/06 14:27'
[    0.000000] PROMLIB: Root node compatible: sun4v
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 3.14.0-rc7 (root@xenon.universe.lan) (gcc version 4.5.3 (Gentoo 4.5.3 p1.4) ) #4 SMP Sun Mar 23 20:39:45 EDT 2014
[    0.000000] bootconsole [earlyprom0] enabled
[    0.000000] ARCH: SUN4V
[    0.000000] Ethernet address: 00:14:4f:6e:cd:36
[    0.000000] PAGE_OFFSET is 0xffffff8000000000 (max_phys_bits == 39)
[    0.000000] Kernel: Using 4 locked TLB entries for main kernel image.
[    0.000000] Remapping the kernel... done.
[    0.000000] OF stdout device is: /virtual-devices@100/console@1
[    0.000000] PROM: Built device tree with 73491 bytes of memory.
[    0.000000] MDESC: Size is 38928 bytes.
[    0.000000] PLATFORM: banner-name [Sun Fire(TM) T1000]
[    0.000000] PLATFORM: name [SUNW,Sun-Fire-T1000]
[    0.000000] PLATFORM: hostid [846ecd36]
[    0.000000] PLATFORM: serial# [00ab4130]
[    0.000000] PLATFORM: stick-frequency [3b9aca00]
[    0.000000] PLATFORM: mac-address [144f6ecd36]
[    0.000000] PLATFORM: watchdog-resolution [1000 ms]
[    0.000000] PLATFORM: watchdog-max-timeout [31536000000 ms]
[    0.000000] PLATFORM: max-cpus [32]
[    0.000000] Top of RAM: 0x1ffd32000, Total RAM: 0x1f7922000
[    0.000000] Memory hole size: 132MB
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x08400000-0x1ffd31fff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x08400000-0x1ffd01fff]
[    0.000000]   node   0: [mem 0x1ffd12000-0x1ffd31fff]
[    0.000000] Booting Linux...
[    0.000000] CPU CAPS: [flush,stbar,swap,muldiv,v9,blkinit,mul32,div32]
[    0.000000] CPU CAPS: [v8plus,ASIBlkInit]
[    0.000000] PERCPU: Embedded 6 pages/cpu @ffffff81fb400000 s13888 r8192 d27072 u131072
[    0.000000] SUN4V: Mondo queue sizes [cpu(4096) dev(16384) r(8192) nr(256)]
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.  Total pages: 1023255
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: root=/dev/sda4
[    0.000000] PID hash table entries: 4096 (order: 2, 32768 bytes)
[    0.000000] Sorting __ex_table...
[    0.000000] Memory: 8169472K/8250504K available (4159K kernel code, 341K rwdata, 1072K rodata, 192K init, 6691K bss, 81032K reserved)
[    0.000000] Hierarchical RCU implementation.
[    0.000000] NR_IRQS:255
[   65.616593] clocksource: mult[800000] shift[23]
[   65.616955] clockevent: mult[80000000] shift[31]
[   65.617902] Console: colour dummy device 80x25
[   65.618259] console [tty0] enabled
[   65.618577] bootconsole [earlyprom0] disabled
[   65.633039] allocated 16777216 bytes of page_cgroup
[   65.633071] please try 'cgroup_disable=memory' option if you don't want memory cgroups
[   65.693077] Calibrating delay using timer specific routine.. 2012.60 BogoMIPS (lpj=1006302)
[   65.693113] pid_max: default: 32768 minimum: 301
[   65.696087] Dentry cache hash table entries: 1048576 (order: 10, 8388608 bytes)
[   65.728844] Inode-cache hash table entries: 524288 (order: 9, 4194304 bytes)
[   65.744631] Mount-cache hash table entries: 512
[   65.745959] Initializing cgroup subsys memory
[   65.746063] Initializing cgroup subsys devices
[   65.746087] Initializing cgroup subsys freezer
[   65.746107] Initializing cgroup subsys hugetlb
[   65.746541] Performance events: Supported PMU type is 'niagara'
[   65.791078] Brought up 32 CPUs
[   65.791197] Testing NMI watchdog ... OK.
[   65.814060] ldc.c:v1.1 (July 22, 2008)
[   65.814080] ldc: Domaining disabled.
[   65.815693] NET: Registered protocol family 16
[   65.837246] VIO: Adding device channel-devices
[   65.837552] VIO: Adding device vldc-port-0-0
[   65.837835] VIO: Adding device vldc-port-0-1
[   65.838130] VIO: Adding device vldc-port-0-2
[   65.838426] VIO: Adding device vldc-port-1-0
[   65.838712] VIO: Adding device vldc-port-3-0
[   65.839025] VIO: Adding device vldc-port-3-8
[   65.839312] VIO: Adding device ds-1
[   65.839614] VIO: Adding device ds-0
[   65.864033] pci_sun4v: Registered hvapi major[1] minor[0]
[   65.864379] /pci@780: SUN4V PCI Bus Module
[   65.864408] /pci@780: On NUMA node -1
[   65.864438] /pci@780: PCI IO[e810000000] MEM[ea00000000]
[   65.864493] /pci@780: Unable to request IOMMU resource.
[   65.913199] /pci@780: MSI Queue first[0] num[36] count[128] devino[0x18]
[   65.913243] /pci@780: MSI first[0] num[256] mask[0xff] width[32]
[   65.913277] /pci@780: MSI addr32[0x7fff0000:0x10000] addr64[0x3ffff0000:0x10000]
[   65.913321] /pci@780: MSI queues at RA [00000001f8400000]
[   65.913354] PCI: Scanning PBM /pci@780
[   65.913842] pci_sun4v f027e7bc: PCI host bridge to bus 0000:02
[   65.913895] pci_bus 0000:02: root bus resource [io  0xe810000000-0xe81fffffff] (bus address [0x0000-0xfffffff])
[   65.913972] pci_bus 0000:02: root bus resource [mem 0xea00000000-0xea7ffeffff] (bus address [0x00000000-0x7ffeffff])
[   65.914031] pci_bus 0000:02: root bus resource [bus 02]
[   65.914112] /pci@7c0: SUN4V PCI Bus Module
[   65.914141] /pci@7c0: On NUMA node -1
[   65.914171] /pci@7c0: PCI IO[f010000000] MEM[f200000000]
[   65.914208] /pci@7c0: Unable to request IOMMU resource.
[   65.962982] /pci@7c0: MSI Queue first[0] num[36] count[128] devino[0x18]
[   65.963025] /pci@7c0: MSI first[0] num[256] mask[0xff] width[32]
[   65.963059] /pci@7c0: MSI addr32[0x7fff0000:0x10000] addr64[0x3ffff0000:0x10000]
[   65.963102] /pci@7c0: MSI queues at RA [00000001f8480000]
[   65.963134] PCI: Scanning PBM /pci@7c0
[   65.963615] pci_sun4v f0289b94: PCI host bridge to bus 0001:02
[   65.963665] pci_bus 0001:02: root bus resource [io  0xf010000000-0xf01fffffff] (bus address [0x0000-0xfffffff])
[   65.963726] pci_bus 0001:02: root bus resource [mem 0xf200000000-0xf27ffeffff] (bus address [0x00000000-0x7ffeffff])
[   65.963783] pci_bus 0001:02: root bus resource [bus 02-04]
[   65.992410] bio: create slab <bio-0> at 0
[   65.994415] vgaarb: loaded
[   65.995919] SCSI subsystem initialized
[   65.999446] ds.c:v1.0 (Jul 11, 2007)
[   66.000013] Switched to clocksource stick
[   66.022110] NET: Registered protocol family 2
[   66.023700] TCP established hash table entries: 65536 (order: 6, 524288 bytes)
[   66.026151] TCP bind hash table entries: 65536 (order: 7, 1048576 bytes)
[   66.030237] TCP: Hash tables configured (established 65536 bind 65536)
[   66.030426] TCP: reno registered
[   66.030518] UDP hash table entries: 4096 (order: 4, 131072 bytes)
[   66.031230] UDP-Lite hash table entries: 4096 (order: 4, 131072 bytes)
[   66.032926] NET: Registered protocol family 1
[   66.033922] sha1_sparc64: sparc64 sha1 opcode not available.
[   66.033975] sha256_sparc64: sparc64 sha256 opcode not available.
[   66.034008] sha512_sparc64: sparc64 sha512 opcode not available.
[   66.034039] md5_sparc64: sparc64 md5 opcode not available.
[   66.034068] aes_sparc64: sparc64 aes opcodes not available.
[   66.034098] des_sparc64: sparc64 des opcodes not available.
[   66.034128] camellia_sparc64: sparc64 camellia opcodes not available.
[   66.034159] crc32c_sparc64: sparc64 crc32c opcode not available.
[   66.043856] futex hash table entries: 8192 (order: 6, 524288 bytes)
[   66.046629] HugeTLB registered 8 MB page size, pre-allocated 0 pages
[   66.050341] msgmni has been set to 15956
[   66.053245] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
[   66.053299] io scheduler noop registered
[   66.053328] io scheduler deadline registered
[   66.053550] io scheduler cfq registered (default)
[   66.056461] f027a840: ttyS0 at I/O 0x0 (irq = 1, base_baud = 115200) is a SUN4V HCONS
[   66.396184] console [ttyHV0] enabled
[   66.397750] f0288178: ttyS1 at MMIO 0xf820c2c000 (irq = 18, base_baud = 115387) is a ST16650V2
[  126.865208] INFO: rcu_sched detected stalls on CPUs/tasks: { 7} (detected by 0, t=60810 jiffies, g=18446744073709551327, c=18446744073709551326, q=101)
[  126.865561] * CPU[  0]: TSTATE[0000000080001603] TPC[000000000042c174] TNPC[000000000042c178] TASK[swapper/0:0]
[  126.865829]              TPC[arch_cpu_idle+0x74/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[start_kernel+0x3b4/0x3c4]
[  126.866124]   CPU[  1]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/1:0]
[  126.866413]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.866700]   CPU[  2]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/2:0]
[  126.867007]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.867340]   CPU[  3]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/3:0]
[  126.867577]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.867862]   CPU[  4]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/4:0]
[  126.868168]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.868428]   CPU[  5]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/5:0]
[  126.868734]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.869019]   CPU[  6]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/6:0]
[  126.869303]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.869588]   CPU[  7]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/7:0]
[  126.869727]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.869840]   CPU[  8]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/8:0]
[  126.869977]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.870091]   CPU[  9]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/9:0]
[  126.870229]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.870343]   CPU[ 10]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/10:0]
[  126.870480]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.870595]   CPU[ 11]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/11:0]
[  126.870734]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.870847]   CPU[ 12]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/12:0]
[  126.870985]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.871246]   CPU[ 13]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/13:0]
[  126.871408]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.871693]   CPU[ 14]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/14:0]
[  126.871978]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.872265]   CPU[ 15]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/15:0]
[  126.872571]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.872904]   CPU[ 16]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/16:0]
[  126.873141]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.873426]   CPU[ 17]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/17:0]
[  126.873735]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.873994]   CPU[ 18]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/18:0]
[  126.874300]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.874587]   CPU[ 19]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/19:0]
[  126.874874]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.875160]   CPU[ 20]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/20:0]
[  126.875297]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.875411]   CPU[ 21]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/21:0]
[  126.875549]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.875662]   CPU[ 22]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/22:0]
[  126.875800]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.875914]   CPU[ 23]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/23:0]
[  126.876053]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.876166]   CPU[ 24]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/24:0]
[  126.876304]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.876418]   CPU[ 25]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/25:0]
[  126.876557]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.876671]   CPU[ 26]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/26:0]
[  126.876809]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.876923]   CPU[ 27]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/27:0]
[  126.877061]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.877175]   CPU[ 28]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/28:0]
[  126.877313]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.877427]   CPU[ 29]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/29:0]
[  126.877565]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.877678]   CPU[ 30]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/30:0]
[  126.877816]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.877929]   CPU[ 31]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/31:0]
[  126.878068]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  126.887472] Fusion MPT base driver 3.04.20
[  126.887522] Copyright (c) 1999-2008 LSI Corporation
[  126.887598] Fusion MPT SAS Host driver 3.04.20
[  126.888159] mptbase: ioc0: Initiating bringup
[  164.865503] ioc0: LSISAS1064 A3: Capabilities={Initiator}
[  235.920978] scsi0 : ioc0: LSISAS1064 A3, FwRev=010a0000h, Ports=1, MaxQ=511, IRQ=25
[  235.978002] mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 0, phy 0, sas_addr 0x5000c500021670d9
[  235.980742] scsi 0:0:0:0: Direct-Access     SEAGATE  ST973401LSUN72G  0556 PQ: 0 ANSI: 3
[  235.985114] sd 0:0:0:0: [sda] 143374738 512-byte logical blocks: (73.4 GB/68.3 GiB)
[  235.986495] sd 0:0:0:0: [sda] Write Protect is off
[  235.988339] mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 1, phy 1, sas_addr 0x5000c50002166991
[  235.988995] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, supports DPO and FUA
[  235.990995] scsi 0:0:1:0: Direct-Access     SEAGATE  ST973401LSUN72G  0556 PQ: 0 ANSI: 3
[  235.995376] sd 0:0:1:0: [sdb] 143374738 512-byte logical blocks: (73.4 GB/68.3 GiB)
[  235.996682] sd 0:0:1:0: [sdb] Write Protect is off
[  235.998715] Fusion MPT misc device (ioctl) driver 3.04.20
[  235.999092] sd 0:0:1:0: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA
[  235.999096] mptctl: Registered with Fusion MPT base driver
[  235.999102] mptctl: /dev/mptctl @ (major,minor=10,220)
[  236.000651] mousedev: PS/2 mouse device common for all mice
[  236.004773]  sda: sda1 sda2 sda3 sda4
[  236.005257] rtc-sun4v rtc-sun4v: rtc core: registered sun4v as rtc0
[  236.009508] TCP: cubic registered
[  236.009556] NET: Registered protocol family 17
[  236.009687] Key type dns_resolver registered
[  236.010707] registered taskstats version 1
[  236.011302] sd 0:0:0:0: [sda] Attached SCSI disk
[  236.012697] rtc-sun4v rtc-sun4v: setting system clock to 2014-03-24 00:47:59 UTC (1395622079)
[  236.033384]  sdb: sdb1 sdb2 sdb3
[  236.039207] sd 0:0:1:0: [sdb] Attached SCSI disk
[  236.077991] EXT4-fs (sda4): mounted filesystem with ordered data mode. Opts: (null)
[  236.078091] VFS: Mounted root (ext4 filesystem) readonly on device 8:4.
[  296.865605] INFO: rcu_sched detected stalls on CPUs/tasks: { 1 2 3 5 6 8 10 11 12 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31} (detected by 0, t=60552 jiffies, g=18446744073709551337, c=18446744073709551336, q=20)
[  296.867077] * CPU[  0]: TSTATE[0000000080001603] TPC[000000000042c174] TNPC[000000000042c178] TASK[swapper/0:0]
[  296.867394]              TPC[arch_cpu_idle+0x74/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[start_kernel+0x3b4/0x3c4]
[  296.867684]   CPU[  1]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/1:0]
[  296.867825]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
...
[  297.236661]   CPU[ 31]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/31:0]
[  297.250135]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  721.866556] INFO: rcu_sched detected stalls on CPUs/tasks: { 1 2 3 5 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31} (detected by 0, t=60845 jiffies, g=18446744073709551338, c=18446744073709551337, q=35)
[  721.868110] * CPU[  0]: TSTATE[0000000080001603] TPC[000000000042c174] TNPC[000000000042c178] TASK[swapper/0:0]
[  721.868591]              TPC[arch_cpu_idle+0x74/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[start_kernel+0x3b4/0x3c4]
...
[  721.882199]   CPU[ 31]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/31:0]
[  721.882336]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
[  722.061274] random: nonblocking pool is initialized
[  782.866698] INFO: rcu_sched detected stalls on CPUs/tasks: { 1 2 3 4 5 6 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31} (detected by 0, t=60703 jiffies, g=18446744073709551345, c=18446744073709551344, q=42)
[  782.868311] * CPU[  0]: TSTATE[0000000080001603] TPC[000000000042c174] TNPC[000000000042c178] TASK[swapper/0:0]
[  782.868621]              TPC[arch_cpu_idle+0x74/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[start_kernel+0x3b4/0x3c4]
[  782.868912]   CPU[  1]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/1:0]
[  782.869053]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
...
[  782.880891]   CPU[ 31]: TSTATE[0000000080001602] TPC[000000000042c170] TNPC[000000000042c174] TASK[swapper/31:0]
[  782.881029]              TPC[arch_cpu_idle+0x70/0xa0] O7[arch_cpu_idle+0x5c/0xa0] I7[cpu_startup_entry+0x114/0x1a0] RPC[0x949378]
 * Caching service dependencies ...
Comment 1 Adrien Dessemond 2014-03-24 01:19:26 UTC 
Created attachment 130541 [details]
full startup log
Comment 2 Adrien Dessemond 2014-03-24 01:23:33 UTC 
Created attachment 130551 [details]
kernel configuration
Comment 3 Adrien Dessemond 2014-03-25 02:32:54 UTC 
See the discussion on http://www.spinics.net/lists/sparclinux/msg11806.html
Comment 4 Adrien Dessemond 2014-03-25 04:03:02 UTC 
Linux 3.14-rc8 includes the patch proposed in the discussion mentioned one comment above. I tested it and the issue is now solved, kernel works like a charm. I close the bug.

https://www.kernel.org/diff/diffview.cgi?file=%2Fpub%2Flinux%2Fkernel%2Fv3.x%2Ftesting%2Fpatch-3.14-rc8.xz;z=2363