Bug 121101 - task_numa_find_cpu in Ceph OSD, not running CFQ
Summary: task_numa_find_cpu in Ceph OSD, not running CFQ
Status: NEW
Alias: None
Product: IO/Storage
Classification: Unclassified
Component: Block Layer (show other bugs)
Hardware: x86-64 Linux
: P1 normal
Assignee: Jens Axboe
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-06-28 14:29 UTC by Alex Gorbachev
Modified: 2016-06-28 16:29 UTC (History)
1 user (show)

See Also:
Kernel Version: 4.4.13-040413-generic
Subsystem:
Regression: No
Bisected commit-id:


Attachments

Description Alex Gorbachev 2016-06-28 14:29:04 UTC
I found similar looking issue here https://lkml.org/lkml/2016/6/22/102, however, we are not using CFQ, the schedule is set to noop and scsi-mq/blk-mq is not enabled.


Jun 28 09:46:41 roc04r-sca090 kernel: [137912.684974] CPU: 30 PID: 10403 Comm: ceph-osd Not tainted 4.4.13-040413-generic #201606072354
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.684991] Hardware name: Supermicro X9DRi-LN4+/X9DR3-LN4+/X9DRi-LN4+/X9DR3-LN4+, BIOS 3.2 03/04/2015
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685009] task: ffff880f79df8000 ti: ffff880f79fb8000 task.ti: ffff880f79fb8000
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685024] RIP: 0010:[<ffffffff810b416e>]  [<ffffffff810b416e>] task_numa_find_cpu+0x22e/0x6f0
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685051] RSP: 0018:ffff880f79fbb818  EFLAGS: 00010206
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685063] RAX: 0000000000000000 RBX: ffff880f79fbb8b8 RCX: 0000000000000000
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685076] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff8810352d4800
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685107] RBP: ffff880f79fbb880 R08: 00000001020cf87c R09: 0000000000ff00ff
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685150] R10: 0000000000000009 R11: 0000000000000006 R12: ffff8807c3adc4c0
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685194] R13: 0000000000000006 R14: 000000000000033e R15: fffffffffffffec7
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685238] FS:  00007f30e46b8700(0000) GS:ffff88105f580000(0000) knlGS:0000000000000000
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685283] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685310] CR2: 000000001321a000 CR3: 0000000853598000 CR4: 00000000000406e0
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685354] Stack:
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685374]  ffffffff813d050f 000000000000000d 0000000000000045 ffff880f79df8000
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685426]  000000000000033f 0000000000000000 0000000000016b00 000000000000033f
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685477]  ffff880f79df8000 ffff880f79fbb8b8 00000000000001f4 0000000000000054
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685528] Call Trace:
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685555]  [<ffffffff813d050f>] ? cpumask_next_and+0x2f/0x40
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685584]  [<ffffffff810b4a6e>] task_numa_migrate+0x43e/0x9b0
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685613]  [<ffffffff810b3acc>] ? update_cfs_shares+0xbc/0x100
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685642]  [<ffffffff810b5059>] numa_migrate_preferred+0x79/0x80
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685672]  [<ffffffff810b9b94>] task_numa_fault+0x7f4/0xd40
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685700]  [<ffffffff813d9634>] ? timerqueue_del+0x24/0x70
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685729]  [<ffffffff810b9205>] ? should_numa_migrate_memory+0x55/0x130
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685762]  [<ffffffff811bd590>] handle_mm_fault+0xbc0/0x1820
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685793]  [<ffffffff810edc00>] ? __hrtimer_init+0x90/0x90
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685822]  [<ffffffff810c211d>] ? remove_wait_queue+0x4d/0x60
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685853]  [<ffffffff8121e20a>] ? poll_freewait+0x4a/0xa0
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685882]  [<ffffffff8106a537>] __do_page_fault+0x197/0x400
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685910]  [<ffffffff8106a7c2>] do_page_fault+0x22/0x30
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685939]  [<ffffffff8180a878>] page_fault+0x28/0x30
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685967]  [<ffffffff813e4c5f>] ? copy_page_to_iter_iovec+0x5f/0x300
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.685997]  [<ffffffff810b2795>] ? select_task_rq_fair+0x625/0x700
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.686026]  [<ffffffff813e4f16>] copy_page_to_iter+0x16/0xa0
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.686056]  [<ffffffff816f02ad>] skb_copy_datagram_iter+0x14d/0x280
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.686087]  [<ffffffff8174a503>] tcp_recvmsg+0x613/0xbe0
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.686117]  [<ffffffff8177844e>] inet_recvmsg+0x7e/0xb0
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.686146]  [<ffffffff816e0d3b>] sock_recvmsg+0x3b/0x50
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.686173]  [<ffffffff816e0f91>] SYSC_recvfrom+0xe1/0x160
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.686202]  [<ffffffff810f36f5>] ? ktime_get_ts64+0x45/0xf0
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.686230]  [<ffffffff816e239e>] SyS_recvfrom+0xe/0x10
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.686259]  [<ffffffff818086f2>] entry_SYSCALL_64_fastpath+0x16/0x71
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.686287] Code: 55 b0 4c 89 f7 e8 53 cd ff ff 48 8b 55 b0 49 8b 4e 78 48 8b 82 d8 01 00 00 48 83 c1 01 31 d2 49 0f af 86 b0 00 00 00 4c 8b 73 78 <48> f7 f1 48 8b 4b 20 49 89 c0 48 29 c1 48 8b 45 d0 4c 03 43 48
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.686512] RIP  [<ffffffff810b416e>] task_numa_find_cpu+0x22e/0x6f0
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.686544]  RSP <ffff880f79fbb818>
Jun 28 09:46:41 roc04r-sca090 kernel: [137912.686896] ---[ end trace 544cb9f68cb55c93 ]---
Jun 28 09:52:15 roc04r-sca090 kernel: [138246.669713] mpt2sas_cm0: log_info(0x30030101): originator(IOP), code(0x03), sub_code(0x0101)
Jun 28 09:55:01 roc0

Note You need to log in before you can comment on or make changes to this bug.