Bug 118831

Summary:	booting sparc64 kernel 4.5.4 with "nosmp" causes OOPS in n2_crypto module
Product:	Drivers	Reporter:	Anatoly Pugachev (matorola)
Component:	Other	Assignee:	drivers_other
Status:	RESOLVED UNREPRODUCIBLE
Severity:	low
Priority:	P1
Hardware:	Sparc64
OS:	Linux
Kernel Version:	4.8.0-rc1+	Subsystem:
Regression:	No	Bisected commit-id:
Attachments:	usual boot log (smp) "nosmp" boot log

Description Anatoly Pugachev 2016-05-24 11:59:40 UTC

Hello!

Tried to boot T5120 with "nosmp" kernel option, gives OOPS in n2_crypto module:


May 24 13:11:48 nvg5120 kernel: Kernel command line: root=/dev/mapper/vg1-root ro nosmp
...
May 24 13:11:48 nvg5120 kernel: Loading compiled-in X.509 certificates
May 24 13:11:48 nvg5120 kernel: Kernel unaligned access at TPC[739430] mpi_read_buffer+0xd0/0x120
May 24 13:11:48 nvg5120 kernel: Loaded X.509 cert 'Debian Project: Ben Hutchings: 008a018dca80932630'
May 24 13:11:48 nvg5120 kernel: aes_sparc64: module verification failed: signature and/or required key missing - tainting kernel
...
May 24 13:11:50 nvg5120 kernel: n2rng.c:v0.2 (July 27, 2011)
May 24 13:11:50 nvg5120 kernel: n2rng f0286a1c: Registered RNG HVAPI major 2 minor 0
May 24 13:11:50 nvg5120 kernel: n2rng f0286a1c: Found single-unit RNG, units: 1
May 24 13:11:50 nvg5120 kernel: n2rng f0286a1c: Selftest passed on unit 0
May 24 13:11:50 nvg5120 kernel: n2rng f0286a1c: RNG ready
May 24 13:11:50 nvg5120 kernel: des_sparc64: sparc64 des opcodes not available.
May 24 13:11:50 nvg5120 kernel: des_sparc64: sparc64 des opcodes not available.
May 24 13:11:50 nvg5120 kernel: des_sparc64: sparc64 des opcodes not available.
May 24 13:11:50 nvg5120 kernel: sha1_sparc64: sparc64 sha1 opcode not available.
May 24 13:11:50 nvg5120 kernel: des_sparc64: sparc64 des opcodes not available.
May 24 13:11:50 nvg5120 kernel: n2_crypto: n2_crypto.c:v0.2 (July 28, 2011)
May 24 13:11:50 nvg5120 kernel: n2_crypto: Found N2CP at /virtual-devices@100/n2cp@7
May 24 13:11:50 nvg5120 kernel: n2_crypto: Registered NCS HVAPI version 2.0
May 24 13:11:50 nvg5120 kernel: genirq: Flags mismatch irq 1. 00000000 (cwq-0) vs. 00000000 (cwq-0)
May 24 13:11:50 nvg5120 kernel: ------------[ cut here ]------------
May 24 13:11:50 nvg5120 kernel: WARNING: CPU: 0 PID: 260 at /build/linux-c06pcb/linux-4.5.4/kernel/irq/manage.c:1449 __free_irq+0xac/0x2a0()
May 24 13:11:50 nvg5120 kernel: Trying to free already-free IRQ 1
May 24 13:11:50 nvg5120 kernel: Modules linked in: n2_crypto(E+) n2_rng(E+) sha512_sparc64(E+) rng_core(E) des_generic(E) autofs4(E) ext4(E) ecb(E)
May 24 13:11:50 nvg5120 kernel: CPU: 0 PID: 260 Comm: systemd-udevd Tainted: G            E   4.5.0-2-sparc64-smp #1 Debian 4.5.4-1
May 24 13:11:50 nvg5120 kernel: Call Trace:
May 24 13:11:50 nvg5120 kernel:  [00000000004669d0] warn_slowpath_common+0x70/0xc0
May 24 13:11:50 nvg5120 kernel:  [0000000000466a50] warn_slowpath_fmt+0x30/0x40
May 24 13:11:50 nvg5120 kernel:  [00000000004bdd0c] __free_irq+0xac/0x2a0
May 24 13:11:50 nvg5120 kernel:  [00000000004bdfa0] free_irq+0x40/0x80
May 24 13:11:50 nvg5120 kernel:  [0000000010aae24c] spu_list_destroy+0xec/0x100 [n2_crypto]
May 24 13:11:50 nvg5120 kernel:  [0000000010aafc98] spu_mdesc_scan+0x298/0x4a0 [n2_crypto]
May 24 13:11:50 nvg5120 kernel:  [0000000010ab0204] n2_crypto_probe+0x1a4/0x680 [n2_crypto]
May 24 13:11:50 nvg5120 kernel:  [00000000007c95f4] platform_drv_probe+0x34/0xc0
May 24 13:11:50 nvg5120 kernel:  [00000000007c708c] driver_probe_device+0x24c/0x460
May 24 13:11:50 nvg5120 kernel:  [00000000007c7328] __driver_attach+0x88/0xa0
May 24 13:11:50 nvg5120 kernel:  [00000000007c497c] bus_for_each_dev+0x5c/0xa0
May 24 13:11:50 nvg5120 kernel:  [00000000007c669c] driver_attach+0x1c/0x40
May 24 13:11:50 nvg5120 kernel:  [00000000007c60b0] bus_add_driver+0x1f0/0x2a0
May 24 13:11:50 nvg5120 kernel:  [00000000007c7db4] driver_register+0x74/0x120
May 24 13:11:50 nvg5120 kernel:  [00000000007c97c4] __platform_register_drivers+0x64/0x160
May 24 13:11:50 nvg5120 kernel:  [0000000010ab6014] n2_init+0x14/0x24 [n2_crypto]
May 24 13:11:50 nvg5120 kernel: ---[ end trace 7aa1f0163177edff ]---
May 24 13:11:50 nvg5120 kernel: camellia_sparc64: sparc64 camellia opcodes not available.





Full boot logs, one with "nosmp" (journalctl -b -k > journalctl-b-k-nosmp.txt) and usual boot (journalctl -k -b -1 > journalctl-b-k-smp.txt) are in attachment.

Comment 1 Anatoly Pugachev 2016-05-24 12:01:38 UTC

Created attachment 217201 [details]
usual boot log (smp)

journalctl -k -b -1 > journalctl-b-1-k-smp.txt

Comment 2 Anatoly Pugachev 2016-05-24 12:02:27 UTC

Created attachment 217211 [details]
"nosmp" boot log

journalctl -b -k > journalctl-b-k-nosmp.txt

Comment 3 Anatoly Pugachev 2016-08-12 13:00:57 UTC

cut from 4.8.0-rc1+ boot log:

[1008744.073627] genirq: Flags mismatch irq 1. 00000000 (cwq-0) vs. 00000000 (cwq-0)
[1008744.166088] ------------[ cut here ]------------
[1008744.225340] WARNING: CPU: 0 PID: 852 at kernel/irq/manage.c:1478 __free_irq+0xa8/0x2a0
[1008744.325135] Trying to free already-free IRQ 1
[1008744.381155] Modules linked in: crc32c_sparc64(-) des_sparc64(-) sha1_sparc64(-) aes_sparc64(-) md5_sparc64(-) sha512_sparc64(-) sha256_sparc64(-) n2_crypto(+) n2_rng sha256_generic rng_core flash des_generic sunrpc autofs4 ext4 crc16 jbd2 mbcache raid10 raid456 libcrc32c crc32c_generic async_raid6_recov async_memcpy async_pq raid6_pq async_xor xor async_tx raid0 multipath linear dm_mod raid1 md_mod sd_mod mptsas scsi_transport_sas mptscsih scsi_mod mptbase
[1008744.896558] CPU: 0 PID: 852 Comm: systemd-udevd Not tainted 4.8.0-rc1+ #65
[1008744.983586] Call Trace:
[1008745.016086]  [0000000000463fc4] __warn+0xa4/0xc0
[1008745.075300]  [0000000000464010] warn_slowpath_fmt+0x30/0x40
[1008745.146300]  [00000000004b49c8] __free_irq+0xa8/0x2a0
[1008745.210864]  [00000000004b4c6c] free_irq+0x4c/0x80
[1008745.272250]  [00000000101b61d0] spu_list_destroy+0x70/0x100 [n2_crypto]
[1008745.356032]  [00000000101b7da0] spu_mdesc_scan+0x420/0x4c0 [n2_crypto]
[1008745.438800]  [00000000101b8314] n2_crypto_probe+0x134/0x680 [n2_crypto]
[1008745.522583]  [000000000073fd74] platform_drv_probe+0x34/0xa0
[1008745.594626]  [000000000073db68] driver_probe_device+0x188/0x400
[1008745.669879]  [000000000073de74] __driver_attach+0x94/0x120
[1008745.739790]  [000000000073b93c] bus_for_each_dev+0x3c/0xa0
[1008745.810046]  [000000000073cd50] bus_add_driver+0xf0/0x280
[1008745.878882]  [000000000073e788] driver_register+0xa8/0x100
[1008745.948811]  [0000000000740008] __platform_register_drivers+0x68/0x160
[1008746.031560]  [0000000000426d00] do_one_initcall+0x80/0x160
[1008746.101514]  [0000000000525558] do_init_module+0x4c/0x1b4
[1008746.170334] ---[ end trace 1c684cc32c67d8e4 ]---


then logs "CPU stall" on console (unable to login):

[1009244.728454] INFO: rcu_sched detected stalls on CPUs/tasks:
[1009244.798408]        (detected by 0, t=115537 jiffies, g=12688, c=12687, q=13)
[1009244.881846] All QSes seen, last rcu_sched kthread activity 115537 (4295031941-4294916404), jiffies_till_next_fqs=1, root ->qsmask 0x0
[1009245.031910] systemd-udevd   R  running task        0   871    840 0x308000306000008
[1009245.129367] Call Trace:
[1009245.161844]  [00000000004c3cfc] update_process_times+0x1c/0x60
[1009245.236027]  [00000000004d1d20] tick_sched_handle.isra.4+0x40/0x60
[1009245.314515]  [00000000004d1d68] tick_sched_timer+0x28/0x60
[1009245.384447]  [00000000004c49c8] __hrtimer_run_queues+0xe8/0x1a0
[1009245.459718]  [00000000004c4c98] hrtimer_interrupt+0x98/0x1a0
[1009245.531792]  [000000000042fa54] timer_interrupt+0x94/0xc0
[1009245.600603]  [00000000004209d4] tl0_irq14+0x14/0x20
[1009245.663004]  [000000001051faf0] 0x1051faf0
[1009245.715790]  [00000000104a1464] 0x104a1464
[1009245.768595]  [00000000005a9e90] do_vfs_ioctl+0x690/0x6e0
[1009245.836409]  [00000000005a9f2c] SyS_ioctl+0x4c/0x80
[1009245.898809]  [00000000004061f4] linux_sparc_syscall+0x34/0x44
[1009245.971956] rcu_sched kthread starved for 115537 jiffies! g12688 c12687 f0x2 RCU_GP_WAIT_FQS(3) ->state=0x0
[1009246.094297] rcu_sched       R  running task        0     7      2 0x06000000
[1009246.184382] Call Trace:
[1009246.216877]  [00000000008adb48] schedule+0x68/0xa0
[1009246.278180]  [00000000008b0860] schedule_timeout+0x240/0x260
[1009246.350245]  [00000000004bef84] rcu_gp_kthread+0x3e4/0x840
[1009246.420130]  [0000000000481230] kthread+0xb0/0xe0
[1009246.480396]  [0000000000406044] ret_from_fork+0x1c/0x2c
[1009246.547467]  [0000000000000000]           (null)

Comment 4 Anatoly Pugachev 2016-08-12 13:01:27 UTC

maybe SPARC should ignore nosmp flag?

Comment 5 Anatoly Pugachev 2017-06-09 11:10:07 UTC

can't reproduce with 4.12.0-rc4-00122-gb29794ec95c6 

root@ttip:/home/mator# cat /proc/cmdline 
BOOT_IMAGE=/vmlinuz-4.12.0-rc4-00122-gb29794ec95c6 root=/dev/vdiska2 ro zswap.enabled=1 noresume nosmp

root@ttip:/home/mator# lscpu 
Architecture:          sparc64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Big Endian
CPU(s):                32
On-line CPU(s) list:   0
Off-line CPU(s) list:  1-31
Thread(s) per core:    1
Core(s) per socket:    1
Socket(s):             1
Model name:            UltraSparc T5 (Niagara5)
Flags:                 sun4v

works ok