Bug 215888

Summary: raw socket test with stress-ng trigger soft lockup
Product: Networking Reporter: Colin Ian King (colin.king)
Component: IPV4Assignee: Stephen Hemminger (stephen)
Status: NEW ---    
Severity: blocking CC: thinhtr
Priority: P1    
Hardware: Intel   
OS: Linux   
Kernel Version: 5.17 Subsystem:
Regression: No Bisected commit-id:

Description Colin Ian King 2022-04-26 09:31:50 UTC 
Running stress-ng [1] with the following raw socket stressor triggers a softlockup on a SMP NUMA x86-64 system:

sudo stress-ng --rawsock 20 -t 60

kernel:watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [stress-ng:49781]

Tested this on 5.17. User has also reported this against the stress-ng project:

https://github.com/ColinIanKing/stress-ng/issues/187

[1] Stress-ng:
https://github.com/ColinIanKing/stress-ng
git clone https://github.com/ColinIanKing/stress-ng
cd stress-ng
make
sudo ./stress-ng --rawsock 0 -t 60
Comment 1 Colin Ian King 2022-04-26 09:52:34 UTC 
running on a 200+ thread system with 5.17 and I lost my ssh connection to the system, so this is a DoS vector.
Comment 2 Colin Ian King 2022-04-27 09:01:07 UTC 
A workaround to stress-ng has been added:

commit 69328da97f04745a9da2890c90c131c2322f81e2 (HEAD -> master)
Author: Colin Ian King <colin.i.king@gmail.com>
Date:   Wed Apr 27 08:49:12 2022 +0000

    stress-rawsock: make client wait for server to start


https://github.com/ColinIanKing/stress-ng/commit/69328da97f04745a9da2890c90c131c2322f81e2

Meanwhile, here are some examples of the softlocks:

[  316.080821] watchdog: BUG: soft lockup - CPU#2 stuck for 26s! [stress-ng:2490]
[  316.096501] watchdog: BUG: soft lockup - CPU#5 stuck for 26s! [stress-ng:2493]
[  316.097799] watchdog: BUG: soft lockup - CPU#6 stuck for 26s! [stress-ng:2494]
[  316.097956] Modules linked in: intel_rapl_msr intel_rapl_common snd_hda_codec_generic ledtrig_audio snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi kvm_intel snd_hda_codec kvm snd_hda_core snd_hwdep snd_pcm rapl snd_timer snd input_leds joydev serio_raw soundcore qemu_fw_cfg mac_hid dm_multipath scsi_dh_rdac scsi_dh_emc sch_fq_codel scsi_dh_alua ipmi_devintf ipmi_msghandler msr virtio_rng ip_tables x_tables autofs4 btrfs blake2b_generic zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear hid_generic usbhid hid qxl drm_ttm_helper ttm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops cec crct10dif_pclmul crc32_pclmul rc_core ghash_clmulni_intel aesni_intel xhci_pci virtio_net i2c_i801 ahci net_failover crypto_simd psmouse drm virtio_blk libahci lpc_ich i2c_smbus xhci_pci_renesas cryptd failover
[  316.098130] CPU: 6 PID: 2494 Comm: stress-ng Tainted: G             L    5.15.0-23-generic #23-Ubuntu
[  316.098133] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.15.0-1 04/01/2014
[  316.098134] RIP: 0010:native_queued_spin_lock_slowpath+0x1d/0x30
[  316.098139] Code: 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 0f 1f 44 00 00 66 90 ba 01 00 00 00 8b 07 85 c0 75 09 f0 0f b1 17 85 c0 75 f2 c3 f3 90 <eb> ed 55 48 89 e5 e8 d8 fd ff ff 5d c3 66 0f 1f 44 00 00 0f 1f 44
[  316.098141] RSP: 0018:ffffa231c4bb7cd8 EFLAGS: 00000202
[  316.098142] RAX: 0000000000000001 RBX: ffffffffb99d1ea0 RCX: 0000000000000000
[  316.098144] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffb99d1ea4
[  316.098145] RBP: ffffa231c4bb7cf0 R08: 0000000000000000 R09: ffff932d8de90b80
[  316.098146] R10: 0000000000000009 R11: 0000000000000006 R12: ffffffffb99d1ea4
[  316.098147] R13: ffffffffb99d1ea0 R14: ffffffffb99d26a0 R15: ffff932d86bd5140
[  316.098148] FS:  00007f4078ed1740(0000) GS:ffff932dfbf80000(0000) knlGS:0000000000000000
[  316.098150] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  316.098151] CR2: 000056142782f340 CR3: 000000001a6ce002 CR4: 0000000000370ee0
[  316.098154] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  316.098155] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  316.098156] Call Trace:
[  316.098158]  <TASK>
[  316.098158]  ? queued_write_lock_slowpath+0x7a/0x80
[  316.098162]  _raw_write_lock_bh+0x27/0x30
[  316.098165]  raw_hash_sk+0x39/0xc0
[  316.098168]  inet_create.part.0.constprop.0+0x1d8/0x4a0
[  316.098172]  inet_create+0x1c/0x30
[  316.098174]  __sock_create+0x12e/0x230
[  316.098176]  __sys_socket+0x5d/0xf0
[  316.098178]  __x64_sys_socket+0x17/0x20
[  316.098179]  do_syscall_64+0x5c/0xc0
[  316.098182]  ? exit_to_user_mode_prepare+0x37/0xb0
[  316.098185]  ? syscall_exit_to_user_mode+0x27/0x50
[  316.098187]  ? __do_sys_getpid+0x1e/0x30
[  316.098189]  ? do_syscall_64+0x69/0xc0
[  316.098192]  ? do_user_addr_fault+0x1e3/0x670
[  316.098194]  ? exit_to_user_mode_prepare+0x37/0xb0
[  316.098196]  ? irqentry_exit_to_user_mode+0x9/0x20
[  316.098198]  ? irqentry_exit+0x19/0x30
[  316.098199]  ? exc_page_fault+0x89/0x160
[  316.098200]  ? asm_exc_page_fault+0x8/0x30
[  316.098203]  entry_SYSCALL_64_after_hwframe+0x44/0xae
[  316.098205] RIP: 0033:0x7f4079aafceb
[  316.098207] Code: 73 01 c3 48 8b 0d 45 11 0f 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 29 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 15 11 0f 00 f7 d8 64 89 01 48
[  316.098209] RSP: 002b:00007ffce4463ea8 EFLAGS: 00000206 ORIG_RAX: 0000000000000029
[  316.098210] RAX: ffffffffffffffda RBX: 00007ffce4463fd0 RCX: 00007f4079aafceb
[  316.098212] RDX: 00000000000000ff RSI: 0000000000000003 RDI: 0000000000000002
[  316.098213] RBP: 00007ffce4464120 R08: 0000561427b1690e R09: 00007f4078ed1740
[  316.098214] R10: 00007f4079a7437b R11: 0000000000000206 R12: 0000561427b1a37b
[  316.098215] R13: 00007f40786ec010 R14: 0000000000000026 R15: 0000000000000000
[  316.098217]  </TASK>
[  316.102224] watchdog: BUG: soft lockup - CPU#4 stuck for 26s! [stress-ng:2492]
[  316.102226] Modules linked in: intel_rapl_msr intel_rapl_common snd_hda_codec_generic ledtrig_audio snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi kvm_intel snd_hda_codec kvm snd_hda_core snd_hwdep snd_pcm rapl snd_timer snd input_leds joydev serio_raw soundcore qemu_fw_cfg mac_hid dm_multipath scsi_dh_rdac scsi_dh_emc sch_fq_codel scsi_dh_alua ipmi_devintf ipmi_msghandler msr virtio_rng ip_tables x_tables autofs4 btrfs blake2b_generic zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear hid_generic usbhid hid qxl drm_ttm_helper ttm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops cec crct10dif_pclmul crc32_pclmul rc_core ghash_clmulni_intel aesni_intel xhci_pci virtio_net i2c_i801 ahci net_failover crypto_simd psmouse drm virtio_blk libahci lpc_ich i2c_smbus xhci_pci_renesas cryptd failover
[  316.103068] CPU: 4 PID: 2492 Comm: stress-ng Tainted: G             L    5.15.0-23-generic #23-Ubuntu
[  316.103071] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.15.0-1 04/01/2014
[  316.103072] RIP: 0010:native_queued_spin_lock_slowpath+0x1d/0x30
[  316.103076] Code: 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 0f 1f 44 00 00 66 90 ba 01 00 00 00 8b 07 85 c0 75 09 f0 0f b1 17 85 c0 75 f2 c3 f3 90 <eb> ed 55 48 89 e5 e8 d8 fd ff ff 5d c3 66 0f 1f 44 00 00 0f 1f 44
[  316.103077] RSP: 0018:ffffa231c4cefd60 EFLAGS: 00000202
[  316.103079] RAX: 0000000000000001 RBX: ffffffffb99d1ea0 RCX: 0000000000000000
[  316.103080] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffb99d1ea4
[  316.103081] RBP: ffffa231c4cefd78 R08: 0000000000000000 R09: ffff932d8391dd20
[  316.103082] R10: 0000000000000009 R11: 0000000000000000 R12: ffffffffb99d1ea4
[  316.103083] R13: ffffffffb99d1ea0 R14: ffffffffb99d26a0 R15: ffff932d8d634e00
[  316.103462] FS:  00007f4078ed1740(0000) GS:ffff932dfbf00000(0000) knlGS:0000000000000000
[  316.103484] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  316.103485] CR2: 000056142782f340 CR3: 00000000215b8004 CR4: 0000000000370ee0
[  316.103522] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  316.103523] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  316.103524] Call Trace:
[  316.103525]  <TASK>
[  316.103525]  ? queued_write_lock_slowpath+0x7a/0x80
[  316.103529]  _raw_write_lock_bh+0x27/0x30
[  316.103531]  raw_hash_sk+0x39/0xc0
[  316.103534]  inet_create.part.0.constprop.0+0x1d8/0x4a0
[  316.103537]  inet_create+0x1c/0x30
[  316.103547]  __sock_create+0x12e/0x230
[  316.103549]  __sys_socket+0x5d/0xf0
[  316.103582]  __x64_sys_socket+0x17/0x20
[  316.103584]  do_syscall_64+0x5c/0xc0
[  316.103586]  ? do_user_addr_fault+0x1e3/0x670
[  316.103588]  ? exit_to_user_mode_prepare+0x37/0xb0
[  316.103591]  ? irqentry_exit_to_user_mode+0x9/0x20
[  316.103593]  ? irqentry_exit+0x19/0x30
[  316.103594]  ? exc_page_fault+0x89/0x160
[  316.103595]  ? asm_exc_page_fault+0x8/0x30
[  316.103598]  entry_SYSCALL_64_after_hwframe+0x44/0xae
[  316.103600] RIP: 0033:0x7f4079aafceb
[  316.103602] Code: 73 01 c3 48 8b 0d 45 11 0f 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 29 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 15 11 0f 00 f7 d8 64 89 01 48
[  316.104141] RSP: 002b:00007ffce4463ea8 EFLAGS: 00000206 ORIG_RAX: 0000000000000029
[  316.104144] RAX: ffffffffffffffda RBX: 00007ffce4463fd0 RCX: 00007f4079aafceb
[  316.104145] RDX: 00000000000000ff RSI: 0000000000000003 RDI: 0000000000000002
[  316.104146] RBP: 00007ffce4464120 R08: 0000561427b1690e R09: 00007f4078ed1740
[  316.104147] R10: 00007f4079a7437b R11: 0000000000000206 R12: 0000561427b1a37b
[  316.104148] R13: 00007f40786eb530 R14: 0000000000000025 R15: 0000000000000000
[  316.104262]  </TASK>
[  316.105976] watchdog: BUG: soft lockup - CPU#8 stuck for 26s! [stress-ng:2496]
[  316.105978] Modules linked in: intel_rapl_msr intel_rapl_common snd_hda_codec_generic ledtrig_audio snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi kvm_intel snd_hda_codec kvm snd_hda_core snd_hwdep snd_pcm rapl snd_timer snd input_leds joydev serio_raw soundcore qemu_fw_cfg mac_hid dm_multipath scsi_dh_rdac scsi_dh_emc sch_fq_codel scsi_dh_alua ipmi_devintf ipmi_msghandler msr virtio_rng ip_tables x_tables autofs4 btrfs blake2b_generic zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear hid_generic usbhid hid qxl drm_ttm_helper ttm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops cec crct10dif_pclmul crc32_pclmul rc_core ghash_clmulni_intel aesni_intel xhci_pci virtio_net i2c_i801 ahci net_failover crypto_simd psmouse drm virtio_blk libahci lpc_ich i2c_smbus xhci_pci_renesas cryptd failover
[  316.106040] CPU: 8 PID: 2496 Comm: stress-ng Tainted: G             L    5.15.0-23-generic #23-Ubuntu
[  316.106042] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.15.0-1 04/01/2014
[  316.106043] RIP: 0010:native_queued_spin_lock_slowpath+0x1d/0x30
[  316.106046] Code: 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 0f 1f 44 00 00 66 90 ba 01 00 00 00 8b 07 85 c0 75 09 f0 0f b1 17 85 c0 75 f2 c3 f3 90 <eb> ed 55 48 89 e5 e8 d8 fd ff ff 5d c3 66 0f 1f 44 00 00 0f 1f 44
[  316.106051] RSP: 0018:ffffa231c5befd30 EFLAGS: 00000202
[  316.106600] RAX: 0000000000000001 RBX: ffffffffb99d1ea0 RCX: 0000000000000000
[  316.106601] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffb99d1ea4
[  316.106602] RBP: ffffa231c5befd48 R08: 0000000000000000 R09: ffff932d84e419e0
[  316.106603] R10: 0000000000000009 R11: 0000000000000000 R12: ffffffffb99d1ea4
[  316.106651] R13: ffffffffb99d1ea0 R14: ffffffffb99d26a0 R15: ffff932d90ee6180
[  316.106679] FS:  00007f4078ed1740(0000) GS:ffff932dfc000000(0000) knlGS:0000000000000000
[  316.106707] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  316.106709] CR2: 000056142782f340 CR3: 0000000014bb2004 CR4: 0000000000370ee0
[  316.106711] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  316.107046] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  316.107047] Call Trace:
[  316.107048]  <TASK>
[  316.107049]  ? queued_write_lock_slowpath+0x7a/0x80
[  316.107088]  _raw_write_lock_bh+0x27/0x30
[  316.107091]  raw_hash_sk+0x39/0xc0
[  316.107093]  inet_create.part.0.constprop.0+0x1d8/0x4a0
[  316.107096]  inet_create+0x1c/0x30
[  316.107347]  __sock_create+0x12e/0x230
[  316.107349]  __sys_socket+0x5d/0xf0
[  316.107358]  __x64_sys_socket+0x17/0x20
[  316.107360]  do_syscall_64+0x5c/0xc0
[  316.107363]  ? do_user_addr_fault+0x1e3/0x670
[  316.107365]  ? exit_to_user_mode_prepare+0x37/0xb0
[  316.107381]  ? syscall_exit_to_user_mode+0x27/0x50
[  316.107382]  ? __do_sys_getpid+0x1e/0x30
[  316.107384]  ? do_syscall_64+0x69/0xc0
[  316.107386]  ? asm_exc_page_fault+0x8/0x30
[  316.107854]  entry_SYSCALL_64_after_hwframe+0x44/0xae
[  316.107898] RIP: 0033:0x7f4079aafceb
[  316.107921] Code: 73 01 c3 48 8b 0d 45 11 0f 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 29 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 15 11 0f 00 f7 d8 64 89 01 48
[  316.107922] RSP: 002b:00007ffce4463ea8 EFLAGS: 00000206 ORIG_RAX: 0000000000000029
[  316.107938] RAX: ffffffffffffffda RBX: 00007ffce4463fd0 RCX: 00007f4079aafceb
[  316.107944] RDX: 00000000000000ff RSI: 0000000000000003 RDI: 0000000000000002
[  316.107945] RBP: 00007ffce4464120 R08: 0000561427b1690e R09: 00007f4078ed1740
[  316.107946] R10: 00007f4079a7437b R11: 0000000000000206 R12: 0000561427b1a37b
[  316.107954] R13: 00007f40786ecaf0 R14: 0000000000000027 R15: 0000000000000000
[  316.107956]  </TASK>
[  316.109517] watchdog: BUG: soft lockup - CPU#0 stuck for 26s! [stress-ng:2489]
[  316.109519] Modules linked in: intel_rapl_msr intel_rapl_common snd_hda_codec_generic ledtrig_audio snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi kvm_intel snd_hda_codec kvm snd_hda_core snd_hwdep snd_pcm rapl snd_timer snd input_leds
[  316.109594] watchdog: BUG: soft lockup - CPU#3 stuck for 26s! [stress-ng:2491]
[  316.109595] Modules linked in: intel_rapl_msr intel_rapl_common snd_hda_codec_generic ledtrig_audio snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi kvm_intel snd_hda_codec kvm snd_hda_core
[  316.109968]  joydev serio_raw soundcore qemu_fw_cfg mac_hid dm_multipath scsi_dh_rdac scsi_dh_emc sch_fq_codel scsi_dh_alua ipmi_devintf ipmi_msghandler msr virtio_rng ip_tables x_tables autofs4 btrfs blake2b_generic zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq
[  316.109600]  snd_hwdep snd_pcm
[  316.110087]  async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear hid_generic usbhid hid
[  316.110150]  rapl snd_timer snd input_leds joydev serio_raw soundcore qemu_fw_cfg
[  316.110197]  qxl drm_ttm_helper ttm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops cec crct10dif_pclmul crc32_pclmul rc_core ghash_clmulni_intel aesni_intel xhci_pci virtio_net i2c_i801 ahci net_failover crypto_simd psmouse drm virtio_blk libahci lpc_ich i2c_smbus xhci_pci_renesas cryptd failover
[  316.110435] CPU: 0 PID: 2489 Comm: stress-ng Tainted: G             L    5.15.0-23-generic #23-Ubuntu
[  316.110438] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.15.0-1 04/01/2014
[  316.110459] RIP: 0010:native_queued_spin_lock_slowpath+0x1d/0x30
[  316.110463] Code: 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 0f 1f 44 00 00 66 90 ba 01 00 00 00 8b 07 85 c0 75 09 f0 0f b1 17 85 c0 75 f2 c3 f3 90 <eb> ed 55 48 89 e5 e8 d8 fd ff ff 5d c3 66 0f 1f 44 00 00 0f 1f 44
[  316.110464] RSP: 0018:ffffa231c5c67d48 EFLAGS: 00000202
[  316.110466] RAX: 0000000000000001 RBX: ffffffffb99d1ea0 RCX: 0000000000000000
[  316.110467] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffb99d1ea4
[  316.110468] RBP: ffffa231c5c67d60 R08: 0000000000000000 R09: ffff932d849f05e0
[  316.110469] R10: 0000000000000009 R11: 0000000000000000 R12: ffffffffb99d1ea4
[  316.110470] R13: ffffffffb99d1ea0 R14: ffffffffb99d26a0 R15: ffff932d814609c0
[  316.110472] FS:  00007f4078ed1740(0000) GS:ffff932dfbe00000(0000) knlGS:0000000000000000
[  316.110473] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  316.110475] CR2: 00007ffce4463d88 CR3: 0000000019a94006 CR4: 0000000000370ef0
[  316.110478] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  316.110478] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  316.110480] Call Trace:
[  316.110480]  <TASK>
[  316.110481]  ? queued_write_lock_slowpath+0x7a/0x80
[  316.110484]  _raw_write_lock_bh+0x27/0x30
[  316.110486]  raw_hash_sk+0x39/0xc0
[  316.110489]  inet_create.part.0.constprop.0+0x1d8/0x4a0
[  316.110154]  mac_hid dm_multipath scsi_dh_rdac scsi_dh_emc sch_fq_codel scsi_dh_alua ipmi_devintf ipmi_msghandler msr virtio_rng ip_tables x_tables autofs4 btrfs blake2b_generic zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear hid_generic usbhid hid qxl drm_ttm_helper ttm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops cec crct10dif_pclmul crc32_pclmul rc_core ghash_clmulni_intel aesni_intel xhci_pci virtio_net i2c_i801 ahci net_failover crypto_simd psmouse drm virtio_blk libahci lpc_ich i2c_smbus xhci_pci_renesas cryptd failover
[  316.110531] CPU: 3 PID: 2491 Comm: stress-ng Tainted: G             L    5.15.0-23-generic #23-Ubuntu
[  316.110533] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.15.0-1 04/01/2014
[  316.110534] RIP: 0010:native_queued_spin_lock_slowpath+0x1d/0x30
[  316.110536] Code: 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 0f 1f 44 00 00 66 90 ba 01 00 00 00 8b 07 85 c0 75 09 f0 0f b1 17 85 c0 75 f2 c3 f3 90 <eb> ed 55 48 89 e5 e8 d8 fd ff ff 5d c3 66 0f 1f 44 00 00 0f 1f 44
[  316.110537] RSP: 0018:ffffa231c5c6fd68 EFLAGS: 00000202
[  316.110539] RAX: 0000000000000001 RBX: ffffffffb99d1ea0 RCX: 0000000000000000
[  316.110540] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffb99d1ea4
[  316.110541] RBP: ffffa231c5c6fd80 R08: 0000000000000000 R09: ffff932d838deda0
[  316.110542] R10: 0000000000000009 R11: 0000000000000000 R12: ffffffffb99d1ea4
[  316.110543] R13: ffffffffb99d1ea0 R14: ffffffffb99d26a0 R15: ffff932d86ad0000
[  316.110544] FS:  00007f4078ed1740(0000) GS:ffff932dfbec0000(0000) knlGS:0000000000000000
[  316.110545] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  316.110546] CR2: 00007ffce4463d88 CR3: 000000001eaec006 CR4: 0000000000370ee0
[  316.110549] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  316.110550] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  316.110551] Call Trace:
[  316.110551]  <TASK>
[  316.110552]  ? queued_write_lock_slowpath+0x7a/0x80
[  316.110554]  _raw_write_lock_bh+0x27/0x30
[  316.110556]  raw_hash_sk+0x39/0xc0
[  316.110558]  inet_create.part.0.constprop.0+0x1d8/0x4a0
[  316.110561]  inet_create+0x1c/0x30
[  316.110563]  __sock_create+0x12e/0x230
[  316.110565]  __sys_socket+0x5d/0xf0
[  316.110567]  ? handle_mm_fault+0xd8/0x2c0
[  316.110569]  __x64_sys_socket+0x17/0x20
[  316.110571]  do_syscall_64+0x5c/0xc0
[  316.110573]  ? do_syscall_64+0x69/0xc0
[  316.110576]  ? exit_to_user_mode_prepare+0x37/0xb0
[  316.110578]  ? irqentry_exit_to_user_mode+0x9/0x20
[  316.110580]  ? irqentry_exit+0x19/0x30
[  316.110581]  ? exc_page_fault+0x89/0x160
[  316.110583]  ? asm_exc_page_fault+0x8/0x30
[  316.110588]  entry_SYSCALL_64_after_hwframe+0x44/0xae
[  316.110590] RIP: 0033:0x7f4079aafceb
[  316.110592] Code: 73 01 c3 48 8b 0d 45 11 0f 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 29 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 15 11 0f 00 f7 d8 64 89 01 48
[  316.110593] RSP: 002b:00007ffce4463ea8 EFLAGS: 00000246 ORIG_RAX: 0000000000000029
[  316.110595] RAX: ffffffffffffffda RBX: 00007ffce4463fd0 RCX: 00007f4079aafceb
[  316.110596] RDX: 00000000000000ff RSI: 0000000000000003 RDI: 0000000000000002
[  316.110597] RBP: 00007ffce4464120 R08: 0000000000000000 R09: 0000000000000000
[  316.110598] R10: 0000000000000008 R11: 0000000000000246 R12: 0000561427b1a37b
[  316.110599] R13: 00007f40786eb530 R14: 0000000000000025 R15: 00000000000009bc
[  316.110753]  inet_create+0x1c/0x30
[  316.110756]  __sock_create+0x12e/0x230
[  316.110758]  __sys_socket+0x5d/0xf0
[  316.110759]  ? syscall_exit_to_user_mode+0x27/0x50
[  316.110761]  __x64_sys_socket+0x17/0x20
[  316.110762]  do_syscall_64+0x5c/0xc0
[  316.110765]  ? syscall_exit_to_user_mode+0x27/0x50
[  316.110766]  ? __x64_sys_setpgid+0x14/0x20
[  316.110771]  ? do_syscall_64+0x69/0xc0
[  316.110773]  ? exit_to_user_mode_prepare+0x37/0xb0
[  316.110775]  ? irqentry_exit_to_user_mode+0x9/0x20
[  316.110800]  ? irqentry_exit+0x19/0x30
[  316.110801]  ? exc_page_fault+0x89/0x160
[  316.110803]  ? asm_exc_page_fault+0x8/0x30
[  316.110805]  entry_SYSCALL_64_after_hwframe+0x44/0xae
[  316.110806] RIP: 0033:0x7f4079aafceb
[  316.110808] Code: 73 01 c3 48 8b 0d 45 11 0f 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 29 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 15 11 0f 00 f7 d8 64 89 01 48
[  316.110809] RSP: 002b:00007ffce4463ea8 EFLAGS: 00000246 ORIG_RAX: 0000000000000029
[  316.110811] RAX: ffffffffffffffda RBX: 00007ffce4463fd0 RCX: 00007f4079aafceb
[  316.110812] RDX: 00000000000000ff RSI: 0000000000000003 RDI: 0000000000000002
[  316.110813] RBP: 00007ffce4464120 R08: 0000000000000000 R09: 0000000000000000
[  316.110814] R10: 0000000000000008 R11: 0000000000000246 R12: 0000561427b1a37b
[  316.110814] R13: 00007f40786eaa50 R14: 0000000000000024 R15: 00000000000009ba
[  316.110820]  </TASK>
[  316.110916]  </TASK>
[  316.129038] Modules linked in: intel_rapl_msr intel_rapl_common snd_hda_codec_generic ledtrig_audio snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi kvm_intel snd_hda_codec kvm snd_hda_core snd_hwdep snd_pcm rapl
[  316.135329] Modules linked in: intel_rapl_msr intel_rapl_common snd_hda_codec_generic ledtrig_audio snd_hda_intel snd_intel_dspcfg
[  316.148408]  snd_timer snd input_leds joydev serio_raw soundcore qemu_fw_cfg mac_hid dm_multipath scsi_dh_rdac scsi_dh_emc sch_fq_codel scsi_dh_alua
[  316.155051]  snd_intel_sdw_acpi kvm_intel snd_hda_codec kvm snd_hda_core snd_hwdep snd_pcm rapl snd_timer snd input_leds joydev serio_raw soundcore qemu_fw_cfg mac_hid dm_multipath scsi_dh_rdac scsi_dh_emc sch_fq_codel scsi_dh_alua ipmi_devintf ipmi_msghandler msr virtio_rng ip_tables x_tables autofs4 btrfs blake2b_generic zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c
[  316.161628] watchdog: BUG: soft lockup - CPU#22 stuck for 26s! [stress-ng:2457]
[  316.161684] Modules linked in: intel_rapl_msr intel_rapl_common snd_hda_codec_generic ledtrig_audio snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi kvm_intel snd_hda_codec kvm snd_hda_core snd_hwdep snd_pcm rapl snd_timer snd input_leds joydev serio_raw soundcore qemu_fw_cfg mac_hid dm_multipath scsi_dh_rdac scsi_dh_emc sch_fq_codel scsi_dh_alua ipmi_devintf ipmi_msghandler msr virtio_rng ip_tables x_tables autofs4 btrfs blake2b_generic zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear hid_generic usbhid hid qxl drm_ttm_helper ttm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops cec crct10dif_pclmul crc32_pclmul rc_core ghash_clmulni_intel aesni_intel xhci_pci virtio_net i2c_i801 ahci net_failover crypto_simd psmouse drm virtio_blk libahci lpc_ich i2c_smbus xhci_pci_renesas cryptd failover
[  316.161732] CPU: 22 PID: 2457 Comm: stress-ng Tainted: G             L    5.15.0-23-generic #23-Ubuntu
[  316.161735] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.15.0-1 04/01/2014
[  316.161736] RIP: 0010:native_queued_spin_lock_slowpath+0x1d/0x30
[  316.161740] Code: 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 0f 1f 44 00 00 66 90 ba 01 00 00 00 8b 07 85 c0 75 09 f0 0f b1 17 85 c0 75 f2 c3 f3 90 <eb> ed 55 48 89 e5 e8 d8 fd ff ff 5d c3 66 0f 1f 44 00 00 0f 1f 44
[  316.161755] RSP: 0018:ffffa231c5c47dc8 EFLAGS: 00000202
[  316.161756] RAX: 0000000000000001 RBX: ffffffffb99d1ea0 RCX: 0000000000000000
[  316.161758] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffb99d1ea4
[  316.161759] RBP: ffffa231c5c47de0 R08: 0000000000000000 R09: ffff932dad977160
[  316.161760] R10: 0000000000000009 R11: 0000000000000006 R12: ffffffffb99d1ea4
[  316.161760] R13: ffffffffb99d1ea0 R14: ffffffffb99d26a0 R15: ffff932dae5dc100
[  316.161762] FS:  00007f4078ed1740(0000) GS:ffff932dfc380000(0000) knlGS:0000000000000000
[  316.161763] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  316.161765] CR2: 00007ffce4463d88 CR3: 000000001630c002 CR4: 0000000000370ee0
[  316.161768] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  316.161769] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  316.161770] Call Trace:
[  316.161771]  <TASK>
[  316.161772]  ? queued_write_lock_slowpath+0x7a/0x80
[  316.161775]  _raw_write_lock_bh+0x27/0x30
[  316.161778]  raw_hash_sk+0x39/0xc0
[  316.161781]  inet_create.part.0.constprop.0+0x1d8/0x4a0
[  316.161785]  inet_create+0x1c/0x30
[  316.161787]  __sock_create+0x12e/0x230
[  316.161789]  __sys_socket+0x5d/0xf0
[  316.161790]  __x64_sys_socket+0x17/0x20
[  316.161792]  do_syscall_64+0x5c/0xc0
[  316.161795]  entry_SYSCALL_64_after_hwframe+0x44/0xae
[  316.161797] RIP: 0033:0x7f4079aafceb
[  316.161799] Code: 73 01 c3 48 8b 0d 45 11 0f 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 29 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 15 11 0f 00 f7 d8 64 89 01 48
[  316.161801] RSP: 002b:00007ffce4463ea8 EFLAGS: 00000246 ORIG_RAX: 0000000000000029
[  316.161802] RAX: ffffffffffffffda RBX: 00007ffce4463fd0 RCX: 00007f4079aafceb
[  316.161804] RDX: 00000000000000ff RSI: 0000000000000003 RDI: 0000000000000002
[  316.161805] RBP: 00007ffce4464120 R08: 0000000000000000 R09: 0000000000000000
[  316.161805] R10: 0000000000000008 R11: 0000000000000246 R12: 0000561427b1a37b
[  316.161806] R13: 00007f40786e7ed0 R14: 0000000000000020 R15: 00000000000009b0
[  316.161809]  </TASK>
[  316.165765] watchdog: BUG: soft lockup - CPU#23 stuck for 26s! [stress-ng:2480]
[  316.165767] Modules linked in: intel_rapl_msr intel_rapl_common snd_hda_codec_generic ledtrig_audio snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi kvm_intel snd_hda_codec kvm snd_hda_core snd_hwdep snd_pcm rapl snd_timer snd input_leds joydev serio_raw soundcore qemu_fw_cfg mac_hid dm_multipath scsi_dh_rdac scsi_dh_emc sch_fq_codel scsi_dh_alua ipmi_devintf ipmi_msghandler msr virtio_rng ip_tables x_tables autofs4 btrfs blake2b_generic zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear hid_generic usbhid hid qxl drm_ttm_helper ttm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops cec crct10dif_pclmul crc32_pclmul rc_core ghash_clmulni_intel aesni_intel xhci_pci virtio_net i2c_i801 ahci net_failover crypto_simd psmouse drm virtio_blk libahci lpc_ich i2c_smbus xhci_pci_renesas cryptd failover
[  316.165999] CPU: 23 PID: 2480 Comm: stress-ng Tainted: G             L    5.15.0-23-generic #23-Ubuntu
[  316.166001] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.15.0-1 04/01/2014
[  316.166002] RIP: 0010:native_queued_spin_lock_slowpath+0x1d/0x30
[  316.166005] Code: 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 0f 1f 44 00 00 66 90 ba 01 00 00 00 8b 07 85 c0 75 09 f0 0f b1 17 85 c0 75 f2 c3 f3 90 <eb> ed 55 48 89 e5 e8 d8 fd ff ff 5d c3 66 0f 1f 44 00 00 0f 1f 44
[  316.166011] RSP: 0018:ffffa231c5ba7dc8 EFLAGS: 00000202
[  316.166012] RAX: 0000000000000001 RBX: ffffffffb99d1ea0 RCX: 0000000000000000
[  316.166013] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffffb99d1ea4
[  316.166014] RBP: ffffa231c5ba7de0 R08: 0000000000000000 R09: ffff932d8900e040
[  316.166015] R10: 0000000000000009 R11: 0000000000000006 R12: ffffffffb99d1ea4
[  316.166088] R13: ffffffffb99d1ea0 R14: ffffffffb99d26a0 R15: ffff932d8bad3a80
[  316.166089] FS:  00007f4078ed1740(0000) GS:ffff932dfc3c0000(0000) knlGS:0000000000000000
[  316.166091] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  316.166092] CR2: 000056142782f340 CR3: 0000000018d0c003 CR4: 0000000000370ee0
[  316.166095] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  316.166096] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  316.166097] Call Trace:
[  316.166097]  <TASK>
[  316.166098]  ? queued_write_lock_slowpath+0x7a/0x80
[  316.166101]  _raw_write_lock_bh+0x27/0x30
[  316.166103]  raw_hash_sk+0x39/0xc0
[  316.166165]  inet_create.part.0.constprop.0+0x1d8/0x4a0
[  316.166168]  inet_create+0x1c/0x30
[  316.166170]  __sock_create+0x12e/0x230
[  316.166171]  __sys_socket+0x5d/0xf0
[  316.166188]  ? __do_sys_getpid+0x1e/0x30
[  316.166191]  __x64_sys_socket+0x17/0x20
[  316.166192]  do_syscall_64+0x5c/0xc0
[  316.166195]  entry_SYSCALL_64_after_hwframe+0x44/0xae
[  316.166197] RIP: 0033:0x7f4079aafceb
[  316.166337] Code: 73 01 c3 48 8b 0d 45 11 0f 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 29 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 15 11 0f 00 f7 d8 64 89 01 48
[  316.166338] RSP: 002b:00007ffce4463ea8 EFLAGS: 00000206 ORIG_RAX: 0000000000000029
[  316.166340] RAX: ffffffffffffffda RBX: 00007ffce4463fd0 RCX: 00007f4079aafceb
[  316.166341] RDX: 00000000000000ff RSI: 0000000000000003 RDI: 0000000000000002
[  316.166342] RBP: 00007ffce4464120 R08: 0000561427b1690e R09: 00007f4078ed1740
[  316.166343] R10: 00007f4079a7437b R11: 0000000000000206 R12: 0000561427b1a37b
[  316.166344] R13: 00007f40786e7ed0 R14: 0000000000000020 R15: 0000000000000000
[  316.166346]  </TASK>
Comment 3 Artem S. Tashkinov 2022-04-27 09:02:53 UTC 
Please email to security at kernel.org as well.