Bug 13522

Summary:	BUG: scheduling while atomic
Product:	Process Management	Reporter:	Sergey Senozhatsky (sergey.senozhatsky)
Component:	Scheduler	Assignee:	Ingo Molnar (mingo)
Status:	CLOSED CODE_FIX
Severity:	high	CC:	akpm, alan, rjw, sergey.senozhatsky
Priority:	P1
Hardware:	All
OS:	Linux
Kernel Version:	2.6.30-git3 (-rc2-git2)	Subsystem:
Regression:	Yes	Bisected commit-id:
Bug Depends on:
Bug Blocks:	13615
Attachments:	syslog_bug git5 git14 git19 git19 (rtorrent was running)

Description Sergey Senozhatsky 2009-06-12 19:29:12 UTC

Hello.
Hard: ASUS F3Jc
Kernel: Linux 2.6.30-git3

See attached file for details.

Jun 12 21:56:39  kernel: [ 8506.042498] BUG: scheduling while atomic: pptpgw/3167/0x00000400
Jun 12 21:56:39  kernel: [ 8506.042504] Modules linked in: ppp_deflate zlib_deflate ppp_async crc_ccitt ppp_generic slhc ipv6 fuse sbp2 loop snd_hda_codec_si3054 snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device snd soundcore pcspkr i2c_i801 rng_core snd_page_alloc psmouse asus_laptop serio_raw evdev led_class usbhid hid sg sd_mod sr_mod cdrom ata_generic pata_acpi ata_piix ide_pci_generic ohci1394 ieee1394 r8169 mii uhci_hcd ehci_hcd usbcore
Jun 12 21:56:39  kernel: [ 8506.042563] 
Jun 12 21:56:39  kernel: [ 8506.042568] Pid: 3167, comm: pptpgw Tainted: G        W  (2.6.30-nv-git3 #7) F3JC                
Jun 12 21:56:39  kernel: [ 8506.042572] EIP: 0060:[<c13e4386>] EFLAGS: 00200246 CPU: 0
Jun 12 21:56:39  kernel: [ 8506.042580] EIP is at _spin_unlock_irqrestore+0x16/0x30
Jun 12 21:56:39  kernel: [ 8506.042583] EAX: c1660040 EBX: 00200246 ECX: 00000000 EDX: 00200246
Jun 12 21:56:39  kernel: [ 8506.042586] ESI: f66b97e0 EDI: 00000001 EBP: f22ebe5c ESP: f22ebe58
Jun 12 21:56:39  kernel: [ 8506.042590]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
Jun 12 21:56:39  kernel: [ 8506.042593] CR0: 8005003b CR2: b753c000 CR3: 36a91000 CR4: 000006d0
Jun 12 21:56:39  kernel: [ 8506.042596] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
Jun 12 21:56:39  kernel: [ 8506.042599] DR6: ffff0ff0 DR7: 00000400
Jun 12 21:56:39  kernel: [ 8506.042601] Call Trace:
Jun 12 21:56:39  kernel: [ 8506.042609]  [<c12b92cb>] tty_ldisc_deref+0x5b/0x90
Jun 12 21:56:39  kernel: [ 8506.042615]  [<c12b2699>] tty_wakeup+0x79/0x90
Jun 12 21:56:39  kernel: [ 8506.042620]  [<c12baa23>] pty_unthrottle+0x23/0x50
Jun 12 21:56:39  kernel: [ 8506.042624]  [<c12b75f0>] tty_unthrottle+0x40/0x70
Jun 12 21:56:39  kernel: [ 8506.042628]  [<c12b571f>] n_tty_read+0x47f/0x720
Jun 12 21:56:39  kernel: [ 8506.042636]  [<c103a020>] ? default_wake_function+0x0/0x40
Jun 12 21:56:39  kernel: [ 8506.042640]  [<c12b0a28>] tty_read+0x98/0xe0
Jun 12 21:56:39  kernel: [ 8506.042644]  [<c12b52a0>] ? n_tty_read+0x0/0x720
Jun 12 21:56:39  kernel: [ 8506.042651]  [<c10e5dfb>] vfs_read+0xbb/0x1c0
Jun 12 21:56:39  kernel: [ 8506.042655]  [<c12b0990>] ? tty_read+0x0/0xe0
Jun 12 21:56:39  kernel: [ 8506.042659]  [<c10e6008>] sys_read+0x58/0xa0
Jun 12 21:56:39  kernel: [ 8506.042665]  [<c100320b>] sysenter_do_call+0x12/0x22
Jun 12 22:02:59  kernel: [ 8885.673899] make invoked oom-killer: gfp_mask=0xd0, order=1, oomkilladj=0
Jun 12 22:03:01  kernel: [ 8885.673912] make cpuset=/ mems_allowed=0
Jun 12 22:03:01  kernel: [ 8885.673921] Pid: 10957, comm: make Tainted: G        W  2.6.30-nv-git3 #7
Jun 12 22:03:01  kernel: [ 8885.673927] Call Trace:
Jun 12 22:03:01  kernel: [ 8885.673944]  [<c10b6384>] T.392+0xc4/0x250
Jun 12 22:03:01  kernel: [ 8885.673954]  [<c10b61b5>] ? badness+0x135/0x240
Jun 12 22:03:01  kernel: [ 8885.673965]  [<c10b665a>] __out_of_memory+0x14a/0x170
Jun 12 22:03:01  kernel: [ 8885.673975]  [<c10b66d9>] out_of_memory+0x59/0xb0
Jun 12 22:03:01  kernel: [ 8885.673985]  [<c10b98dd>] __alloc_pages_internal+0x47d/0x4d0
Jun 12 22:03:01  kernel: [ 8885.673996]  [<c10b99a8>] __get_free_pages+0x28/0x60
Jun 12 22:03:01  kernel: [ 8885.674007]  [<c103daea>] copy_process+0xca/0x1090
Jun 12 22:03:01  kernel: [ 8885.674017]  [<c103eb34>] do_fork+0x84/0x3a0
Jun 12 22:03:01  kernel: [ 8885.674027]  [<c100b958>] sys_vfork+0x38/0x50
Jun 12 22:03:01  kernel: [ 8885.674036]  [<c10032d8>] syscall_call+0x7/0xb
Jun 12 22:03:01  kernel: [ 8885.674042] Mem-Info:
Jun 12 22:03:01  kernel: [ 8885.674047] DMA per-cpu:
Jun 12 22:03:01  kernel: [ 8885.674053] CPU    0: hi:    0, btch:   1 usd:   0
Jun 12 22:03:01  kernel: [ 8885.674059] CPU    1: hi:    0, btch:   1 usd:   0
Jun 12 22:03:01  kernel: [ 8885.674064] Normal per-cpu:
Jun 12 22:03:01  kernel: [ 8885.674070] CPU    0: hi:  186, btch:  31 usd:   0
Jun 12 22:03:01  kernel: [ 8885.674076] CPU    1: hi:  186, btch:  31 usd:   0
Jun 12 22:03:01  kernel: [ 8885.674081] HighMem per-cpu:
Jun 12 22:03:01  kernel: [ 8885.674087] CPU    0: hi:   42, btch:   7 usd:   0
Jun 12 22:03:01  kernel: [ 8885.674093] CPU    1: hi:   42, btch:   7 usd:   0
Jun 12 22:03:01  kernel: [ 8885.674102] Active_anon:7322 active_file:8880 inactive_anon:9533
Jun 12 22:03:01  kernel: [ 8885.674105]  inactive_file:6083 unevictable:0 dirty:5 writeback:0 unstable:0
Jun 12 22:03:01  kernel: [ 8885.674109]  free:73824 slab:102205 mapped:8845 pagetables:561 bounce:0
Jun 12 22:03:01  kernel: [ 8885.674120] DMA free:3520kB min:64kB low:80kB high:96kB active_anon:0kB inactive_anon:12kB active_file:0kB inactive_file:0kB unevictable:0kB present:15804kB pages_scanned:0 all_unreclaimable? no
Jun 12 22:03:01  kernel: [ 8885.674129] lowmem_reserve[]: 0 865 999 999
Jun 12 22:03:01  kernel: [ 8885.674145] Normal free:269068kB min:3728kB low:4660kB high:5592kB active_anon:32kB inactive_anon:56kB active_file:7112kB inactive_file:7076kB unevictable:0kB present:885944kB pages_scanned:0 all_unreclaimable? no
Jun 12 22:03:01  kernel: [ 8885.674154] lowmem_reserve[]: 0 0 1076 1076
Jun 12 22:03:01  kernel: [ 8885.674170] HighMem free:22708kB min:132kB low:276kB high:420kB active_anon:29256kB inactive_anon:38064kB active_file:28408kB inactive_file:17256kB unevictable:0kB present:137832kB pages_scanned:0 all_unreclaimable? no
Jun 12 22:03:01  kernel: [ 8885.674178] lowmem_reserve[]: 0 0 0 0
Jun 12 22:03:01  kernel: [ 8885.674189] DMA: 6*4kB 395*8kB 21*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 3520kB
Jun 12 22:03:01  kernel: [ 8885.674216] Normal: 66712*4kB 55*8kB 0*16kB 1*32kB 0*64kB 0*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 269112kB
Jun 12 22:03:01  kernel: [ 8885.674243] HighMem: 1126*4kB 1333*8kB 278*16kB 50*32kB 18*64kB 3*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 22752kB
Jun 12 22:03:01  kernel: [ 8885.674271] 15997 total pagecache pages
Jun 12 22:03:01  kernel: [ 8885.674276] 777 pages in swap cache
Jun 12 22:03:01  kernel: [ 8885.674282] Swap cache stats: add 13528, delete 12751, find 702/875
Jun 12 22:03:01  kernel: [ 8885.674287] Free swap  = 826996kB
Jun 12 22:03:01  kernel: [ 8885.674291] Total swap = 875500kB
Jun 12 22:03:01  kernel: [ 8885.679660] 262040 pages RAM
Jun 12 22:03:01  kernel: [ 8885.679665] 34730 pages HighMem
Jun 12 22:03:01  kernel: [ 8885.679670] 3930 pages reserved
Jun 12 22:03:01  kernel: [ 8885.679674] 53968 pages shared
Jun 12 22:03:01  kernel: [ 8885.679678] 134773 pages non-shared
Jun 12 22:03:01  kernel: [ 8885.679686] Out of memory: kill process 3212 (kopete) score 17744 or a child
Jun 12 22:03:01  kernel: [ 8885.679726] Killed process 3212 (kopete)
Jun 12 22:03:01  kernel: [ 8886.099540] sh invoked oom-killer: gfp_mask=0xd0, order=1, oomkilladj=0
Jun 12 22:03:01  kernel: [ 8886.099552] sh cpuset=/ mems_allowed=0
Jun 12 22:03:01  kernel: [ 8886.099561] Pid: 11238, comm: sh Tainted: G        W  2.6.30-nv-git3 #7
Jun 12 22:03:01  kernel: [ 8886.099567] Call Trace:
...
/*
truncated since comment's size is limited to 65536.
see attached file for details.
*/

Jun 12 22:06:39  kernel: [ 9105.449449] SysRq : Keyboard mode set to system default
Jun 12 22:06:39  exiting on signal 15
Jun 12 22:09:04  syslogd 1.5.0#5: restart.

Sergey

Comment 1 Sergey Senozhatsky 2009-06-12 19:30:32 UTC

Created attachment 21886 [details]
syslog_bug

Comment 2 Sergey Senozhatsky 2009-06-13 13:52:36 UTC

Hello.
More bugs (git5).

See attachment for details.

kernel: [ 1424.347514] BUG: scheduling while atomic: ksoftirqd/0/4/0x10000500
kernel: [ 1424.347520] INFO: lockdep is turned off.
kernel: [ 1424.347523] Modules linked in: ppp_deflate zlib_deflate ppp_async crc_ccitt ppp_generic slhc ipv6 fuse sbp2 loop snd_hda_codec_si3054 snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device snd soundcore pcspkr psmouse snd_page_alloc i2c_i801 rng_core serio_raw asus_laptop evdev led_class usbhid hid sg sd_mod sr_mod cdrom ata_generic pata_acpi ohci1394 uhci_hcd ata_piix ide_pci_generic ieee1394 ehci_hcd r8169 mii usbcore
kernel: [ 1424.347600] Pid: 4, comm: ksoftirqd/0 Tainted: G        W  2.6.30-nv-git5 #12
kernel: [ 1424.347603] Call Trace:
kernel: [ 1424.347615]  [<c103766b>] __schedule_bug+0x7b/0x90
kernel: [ 1424.347623]  [<c13fc143>] schedule+0x5e3/0xc10
kernel: [ 1424.347629]  [<c1070465>] ? print_lock_contention_bug+0x25/0xf0
kernel: [ 1424.347635]  [<c103a62b>] ? default_wake_function+0x1b/0x40
kernel: [ 1424.347641]  [<c103af26>] __cond_resched+0x26/0x60
kernel: [ 1424.347646]  [<c13fc8c2>] _cond_resched+0x42/0x50
kernel: [ 1424.347651]  [<c13fd68d>] mutex_lock_nested+0x1d/0x280
kernel: [ 1424.347658]  [<c12cc575>] tty_throttle+0x25/0x70
kernel: [ 1424.347664]  [<c12cae77>] n_tty_receive_buf+0x407/0x1220
kernel: [ 1424.347670]  [<c1046fdd>] ? irq_exit+0x3d/0x90
kernel: [ 1424.347676]  [<c1003bfd>] ? apic_timer_interrupt+0x31/0x38
kernel: [ 1424.347684]  [<f90b191d>] ? ppp_async_push+0x5d/0x4d0 [ppp_async]
kernel: [ 1424.347690]  [<f90b191d>] ? ppp_async_push+0x5d/0x4d0 [ppp_async]
kernel: [ 1424.347696]  [<f90b191d>] ? ppp_async_push+0x5d/0x4d0 [ppp_async]
kernel: [ 1424.347701]  [<c12cf738>] pty_write+0x48/0x80
kernel: [ 1424.347707]  [<f90b1c9a>] ppp_async_push+0x3da/0x4d0 [ppp_async]
kernel: [ 1424.347713]  [<f90b1dd8>] ppp_async_send+0x48/0x70 [ppp_async]
kernel: [ 1424.347722]  [<f9098986>] ppp_push+0x176/0x5f0 [ppp_generic]
kernel: [ 1424.347728]  [<c1361916>] ? skb_dequeue+0x26/0x80
kernel: [ 1424.347733]  [<c1361943>] ? skb_dequeue+0x53/0x80
kernel: [ 1424.347740]  [<f90997ad>] ppp_xmit_process+0x3dd/0x570 [ppp_generic]
kernel: [ 1424.347746]  [<c13617f1>] ? skb_queue_tail+0x41/0x60
kernel: [ 1424.347753]  [<f9099aa7>] ppp_start_xmit+0x167/0x1b0 [ppp_generic]
kernel: [ 1424.347759]  [<c138063a>] ? __qdisc_run+0x18a/0x250
kernel: [ 1424.347765]  [<c136ba40>] dev_hard_start_xmit+0x190/0x390
kernel: [ 1424.347770]  [<c1380654>] __qdisc_run+0x1a4/0x250
kernel: [ 1424.347775]  [<c136c0d7>] dev_queue_xmit+0x367/0x4c0
kernel: [ 1424.347780]  [<c136bdb0>] ? dev_queue_xmit+0x40/0x4c0
kernel: [ 1424.347787]  [<c139927d>] ip_finish_output+0x1dd/0x2b0
kernel: [ 1424.347792]  [<c13993ac>] ip_output+0x5c/0xc0
kernel: [ 1424.347797]  [<c1070465>] ? print_lock_contention_bug+0x25/0xf0
kernel: [ 1424.347802]  [<c13982f8>] ip_local_out+0x28/0x50
kernel: [ 1424.347807]  [<c1398a64>] ip_queue_xmit+0x144/0x390
kernel: [ 1424.347813]  [<c104cb62>] ? lock_timer_base+0x32/0x70
kernel: [ 1424.347820]  [<c13b1048>] ? tcp_v4_send_check+0x58/0x100
kernel: [ 1424.347825]  [<c13acb16>] tcp_transmit_skb+0x3b6/0x750
kernel: [ 1424.347830]  [<c104cc63>] ? mod_timer+0xc3/0x130
kernel: [ 1424.347836]  [<c13aef2e>] tcp_write_xmit+0x19e/0x950
kernel: [ 1424.347842]  [<c10ea6d1>] ? cache_free_debugcheck+0x1b1/0x290
kernel: [ 1424.347848]  [<c13af77c>] __tcp_push_pending_frames+0x3c/0xb0
kernel: [ 1424.347854]  [<c13aaad2>] tcp_rcv_established+0x5c2/0xa00
kernel: [ 1424.347859]  [<c13b2f28>] tcp_v4_do_rcv+0x258/0x390
kernel: [ 1424.347864]  [<c13b35ba>] ? tcp_v4_rcv+0x55a/0x7d0
kernel: [ 1424.347869]  [<c13b35ba>] ? tcp_v4_rcv+0x55a/0x7d0
kernel: [ 1424.347874]  [<c13ff455>] ? _spin_lock_nested+0x65/0x80
kernel: [ 1424.347880]  [<c13b35da>] tcp_v4_rcv+0x57a/0x7d0
kernel: [ 1424.347886]  [<c1393eb7>] ip_local_deliver_finish+0xd7/0x270
kernel: [ 1424.347891]  [<c1393e15>] ? ip_local_deliver_finish+0x35/0x270
kernel: [ 1424.347897]  [<c13940f7>] ip_local_deliver+0xa7/0xb0
kernel: [ 1424.347902]  [<c139388e>] ip_rcv_finish+0xee/0x3a0
kernel: [ 1424.347907]  [<c1393d22>] ip_rcv+0x1e2/0x2a0
kernel: [ 1424.347912]  [<c136a8e8>] ? netif_receive_skb+0x108/0x5a0
kernel: [ 1424.347917]  [<c136aab3>] netif_receive_skb+0x2d3/0x5a0
kernel: [ 1424.347922]  [<c136a8e8>] ? netif_receive_skb+0x108/0x5a0
kernel: [ 1424.347927]  [<c136adec>] process_backlog+0x6c/0xb0
kernel: [ 1424.347932]  [<c1368be9>] net_rx_action+0x149/0x200
kernel: [ 1424.347937]  [<c1368b78>] ? net_rx_action+0xd8/0x200
kernel: [ 1424.347943]  [<c1046d0a>] __do_softirq+0x7a/0x1a0
kernel: [ 1424.347948]  [<c1046e7d>] do_softirq+0x4d/0x60
kernel: [ 1424.347953]  [<c1046efe>] ksoftirqd+0x6e/0x110
kernel: [ 1424.347958]  [<c1046e90>] ? ksoftirqd+0x0/0x110
kernel: [ 1424.347963]  [<c105b8c5>] kthread+0x55/0xa0
kernel: [ 1424.347968]  [<c105b870>] ? kthread+0x0/0xa0
kernel: [ 1424.347973]  [<c1003d5f>] kernel_thread_helper+0x7/0x18

Sergey

Comment 3 Sergey Senozhatsky 2009-06-13 13:53:24 UTC

Created attachment 21897 [details]
git5

Comment 4 Sergey Senozhatsky 2009-06-19 18:02:29 UTC

Hello.

kernel: [27404.945372] BUG: scheduling while atomic: pptpgw/4637/0x00000400
kernel: [27404.945378] INFO: lockdep is turned off.
kernel: [27404.945383] Modules linked in: ppp_deflate zlib_deflate ppp_async crc_ccitt ppp_generic slhc ipv6 fuse sbp2 loop snd_hda_codec_si3054 snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device pcspkr snd i2c_i801 psmouse serio_raw rng_core soundcore snd_page_alloc asus_laptop evdev usbhid hid sg sr_mod cdrom sd_mod sdhci_pci sdhci mmc_core ohci1394 ata_generic pata_acpi ricoh_mmc uhci_hcd led_class ata_piix ieee1394 ide_pci_generic ehci_hcd usbcore r8169 mii
kernel: [27404.945532] 
kernel: [27404.945540] Pid: 4637, comm: pptpgw Not tainted (2.6.30-dbgnv-git14 #4) F3JC                
kernel: [27404.945549] EIP: 0060:[<c104a3d5>] EFLAGS: 00200246 CPU: 0
kernel: [27404.945559] EIP is at __tasklet_schedule+0x65/0xb0
kernel: [27404.945566] EAX: 00000000 EBX: f6077c6c ECX: 00000000 EDX: 00000000
kernel: [27404.945573] ESI: 00200246 EDI: 00000001 EBP: e2807e60 ESP: e2807e54
kernel: [27404.945581]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
kernel: [27404.945589] CR0: 8005003b CR2: b5f027ac CR3: 35993000 CR4: 000006d0
kernel: [27404.945596] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
kernel: [27404.945603] DR6: ffff0ff0 DR7: 00000400
kernel: [27404.945609] Call Trace:
kernel: [27404.945621]  [<f90be38b>] ppp_asynctty_wakeup+0x6b/0x80 [ppp_async]
kernel: [27404.945632]  [<c12de902>] tty_wakeup+0x72/0x90
kernel: [27404.945643]  [<c12e70d3>] pty_unthrottle+0x23/0x50
kernel: [27404.945653]  [<c12e3b22>] tty_unthrottle+0x42/0x70
kernel: [27404.945663]  [<c12e1c2f>] n_tty_read+0x48f/0x730
kernel: [27404.945676]  [<c103e150>] ? default_wake_function+0x0/0x40
kernel: [27404.945688]  [<c12dcc20>] tty_read+0xa0/0xe0
kernel: [27404.945698]  [<c12e17a0>] ? n_tty_read+0x0/0x730
kernel: [27404.945709]  [<c10faf2b>] vfs_read+0xbb/0x1c0
kernel: [27404.945720]  [<c12dcb80>] ? tty_read+0x0/0xe0
kernel: [27404.945730]  [<c10fb138>] sys_read+0x58/0xa0
kernel: [27404.945741]  [<c100325b>] sysenter_do_call+0x12/0x32
...
/USR/SBIN/CRON[10140]: CMD (fetchmail -sk -m '/usr/bin/procmail' > /dev/null 2>&1)
...
kernel: [27729.047046] BUG: spinlock lockup on CPU#1, pppd/4636, f5928dc0
kernel: [27729.047060] Pid: 4636, comm: pppd Not tainted 2.6.30-dbgnv-git14 #4
kernel: [27729.047067] Call Trace:
kernel: [27729.047084]  [<c141786b>] ? printk+0x23/0x36
kernel: [27729.047099]  [<c125da29>] _raw_spin_lock+0x159/0x170
kernel: [27729.047111]  [<c141b7f2>] _spin_lock_bh+0x62/0x90
kernel: [27729.047130]  [<f90aabe7>] ? ppp_channel_push+0x27/0xd0 [ppp_generic]
kernel: [27729.047145]  [<f90aabe7>] ppp_channel_push+0x27/0xd0 [ppp_generic]
kernel: [27729.047159]  [<f90aad67>] ppp_write+0xd7/0xe0 [ppp_generic]
kernel: [27729.047172]  [<c10fac1e>] vfs_write+0xbe/0x1c0
kernel: [27729.047186]  [<f90aac90>] ? ppp_write+0x0/0xe0 [ppp_generic]
kernel: [27729.047197]  [<c10fae28>] sys_write+0x58/0xa0
kernel: [27729.047209]  [<c100325b>] sysenter_do_call+0x12/0x32
kernel: [27787.880283] SysRq : Keyboard mode set to system default

The rest is in attach.

Sergey

Comment 5 Sergey Senozhatsky 2009-06-19 18:03:09 UTC

Created attachment 22010 [details]
git14

Comment 6 Sergey Senozhatsky 2009-06-19 18:19:37 UTC

kernel: [  239.982730] 
kernel: [  239.982736] =================================
kernel: [  239.982745] [ INFO: inconsistent lock state ]
kernel: [  239.982753] 2.6.30-dbgnv-git9 #5
kernel: [  239.982758] ---------------------------------
kernel: [  239.982765] inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
kernel: [  239.982774] ksoftirqd/0/4 [HC0[0]:SC1[5]:HE1:SE0] takes:
kernel: [  239.982781]  (&tty->termios_mutex){+.?...}, at: [<c12d44d5>] tty_throttle+0x25/0x70
kernel: [  239.982805] {SOFTIRQ-ON-W} state was registered at:
kernel: [  239.982811]   [<c107755f>] __lock_acquire+0xacf/0x1840
kernel: [  239.982825]   [<c1078371>] lock_acquire+0xa1/0x120
kernel: [  239.982836]   [<c14099dc>] mutex_lock_nested+0x5c/0x2e0
kernel: [  239.982849]   [<c12d4465>] tty_unthrottle+0x25/0x70
kernel: [  239.982859]   [<c12d28f9>] reset_buffer_flags+0xc9/0x100
kernel: [  239.982870]   [<c12d3ca0>] n_tty_open+0x40/0x160
kernel: [  239.982879]   [<c12d5ae9>] tty_ldisc_open+0x39/0x70
kernel: [  239.982890]   [<c12d5cdb>] tty_ldisc_setup+0x2b/0x90
kernel: [  239.982901]   [<c12d0619>] tty_init_dev+0x99/0x190
kernel: [  239.982910]   [<c12d09cc>] tty_open+0x2bc/0x480
kernel: [  239.982920]   [<c10fdc42>] chrdev_open+0xe2/0x1b0
kernel: [  239.982933]   [<c10f7e14>] __dentry_open+0xd4/0x290
kernel: [  239.982943]   [<c10f80fd>] nameidata_to_filp+0x6d/0x80
kernel: [  239.982954]   [<c1108673>] do_filp_open+0x5f3/0x980
kernel: [  239.982965]   [<c10f7b6d>] do_sys_open+0x5d/0x130
kernel: [  239.982975]   [<c10f7cd4>] sys_open+0x34/0x50
kernel: [  239.982985]   [<c100134b>] init_post+0x4b/0x110
kernel: [  239.982996]   [<c15c84d6>] kernel_init+0x165/0x180
kernel: [  239.983009]   [<c1003edb>] kernel_thread_helper+0x7/0x1c
kernel: [  239.983020]   [<ffffffff>] 0xffffffff
kernel: [  239.983043] irq event stamp: 122452
kernel: [  239.983049] hardirqs last  enabled at (122452): [<c140b6af>] _spin_unlock_irqrestore+0x3f/0x70
kernel: [  239.983063] hardirqs last disabled at (122451): [<c140ba94>] _spin_lock_irqsave+0x24/0xa0
kernel: [  239.983076] softirqs last  enabled at (122294): [<c104ac75>] __do_softirq+0xe5/0x1c0
kernel: [  239.983090] softirqs last disabled at (122305): [<c104adc5>] do_softirq+0x75/0x90
kernel: [  239.983102] 
kernel: [  239.983104] other info that might help us debug this:
kernel: [  239.983112] 9 locks held by ksoftirqd/0/4:
kernel: [  239.983118]  #0:  (rcu_read_lock){.+.+..}, at: [<c1371652>] net_rx_action+0xe2/0x260
kernel: [  239.983139]  #1:  (rcu_read_lock){.+.+..}, at: [<c1373860>] netif_receive_skb+0x140/0x5e0
kernel: [  239.983159]  #2:  (rcu_read_lock){.+.+..}, at: [<c139d5e5>] ip_local_deliver_finish+0x35/0x270
kernel: [  239.983179]  #3:  (slock-AF_INET/1){+.-...}, at: [<c13bcf5a>] tcp_v4_rcv+0x55a/0x7d0
kernel: [  239.983201]  #4:  (rcu_read_lock){.+.+..}, at: [<c1374df0>] dev_queue_xmit+0x40/0x4c0
kernel: [  239.983220]  #5:  (_xmit_PPP#2){+.-...}, at: [<c1389cca>] __qdisc_run+0x18a/0x250
kernel: [  239.983242]  #6:  (&ppp->wlock){+.-...}, at: [<f909f406>] ppp_xmit_process+0x26/0x570 [ppp_generic]
kernel: [  239.983268]  #7:  (&pch->downl){+.-...}, at: [<f909e980>] ppp_push+0x160/0x5f0 [ppp_generic]
kernel: [  239.983290]  #8:  (&ap->xmit_lock){+.-...}, at: [<f90b791d>] ppp_async_push+0x5d/0x4d0 [ppp_async]
kernel: [  239.983312] 
kernel: [  239.983313] stack backtrace:
kernel: [  239.983322] Pid: 4, comm: ksoftirqd/0 Not tainted 2.6.30-dbgnv-git9 #5
kernel: [  239.983329] Call Trace:
kernel: [  239.983340]  [<c1407a47>] ? printk+0x23/0x36
kernel: [  239.983350]  [<c107523e>] print_usage_bug+0x17e/0x1c0
kernel: [  239.983361]  [<c1075640>] mark_lock+0x3c0/0x5c0
kernel: [  239.983372]  [<c1076050>] ? check_usage_forwards+0x0/0xc0
kernel: [  239.983383]  [<c107751c>] __lock_acquire+0xa8c/0x1840
kernel: [  239.983393]  [<c1073bd9>] ? trace_hardirqs_off+0x19/0x40
kernel: [  239.983405]  [<c140b6cf>] ? _spin_unlock_irqrestore+0x5f/0x70
kernel: [  239.983416]  [<c1078371>] lock_acquire+0xa1/0x120
kernel: [  239.983427]  [<c12d44d5>] ? tty_throttle+0x25/0x70
kernel: [  239.983437]  [<c12d44d5>] ? tty_throttle+0x25/0x70
kernel: [  239.983448]  [<c14099dc>] mutex_lock_nested+0x5c/0x2e0
kernel: [  239.983458]  [<c12d44d5>] ? tty_throttle+0x25/0x70
kernel: [  239.983468]  [<c1075c49>] ? trace_hardirqs_on+0x19/0x40
kernel: [  239.983477]  [<c12d44d5>] ? tty_throttle+0x25/0x70
kernel: [  239.983488]  [<c12d44d5>] tty_throttle+0x25/0x70
kernel: [  239.983498]  [<c12d2dd7>] n_tty_receive_buf+0x407/0x1220
kernel: [  239.983509]  [<c1075c49>] ? trace_hardirqs_on+0x19/0x40
kernel: [  239.983520]  [<c1030ae3>] ? __wake_up+0x53/0x70
kernel: [  239.983532]  [<c1076e1c>] ? __lock_acquire+0x38c/0x1840
kernel: [  239.983545]  [<f90b791d>] ? ppp_async_push+0x5d/0x4d0 [ppp_async]
kernel: [  239.983558]  [<f90b791d>] ? ppp_async_push+0x5d/0x4d0 [ppp_async]
kernel: [  239.983569]  [<c12d7698>] pty_write+0x48/0x80
kernel: [  239.983581]  [<f90b7c9a>] ppp_async_push+0x3da/0x4d0 [ppp_async]
kernel: [  239.983595]  [<f90b7dd8>] ppp_async_send+0x48/0x70 [ppp_async]
kernel: [  239.983609]  [<f909e996>] ppp_push+0x176/0x5f0 [ppp_generic]
kernel: [  239.983621]  [<c140b6af>] ? _spin_unlock_irqrestore+0x3f/0x70
kernel: [  239.983632]  [<c1075af8>] ? trace_hardirqs_on_caller+0x68/0x1a0
kernel: [  239.983643]  [<c1075c49>] ? trace_hardirqs_on+0x19/0x40
kernel: [  239.983654]  [<c136a143>] ? skb_dequeue+0x53/0x80
kernel: [  239.983668]  [<f909f7bd>] ppp_xmit_process+0x3dd/0x570 [ppp_generic]
kernel: [  239.983679]  [<c1075c49>] ? trace_hardirqs_on+0x19/0x40
kernel: [  239.983691]  [<c1369ff1>] ? skb_queue_tail+0x41/0x60
kernel: [  239.983705]  [<f909fab7>] ppp_start_xmit+0x167/0x1b0 [ppp_generic]
kernel: [  239.983716]  [<c1389cca>] ? __qdisc_run+0x18a/0x250
kernel: [  239.983728]  [<c1374abc>] dev_hard_start_xmit+0x1ec/0x3b0
kernel: [  239.983739]  [<c140b937>] ? _spin_lock+0x67/0x80
kernel: [  239.983749]  [<c1389ce4>] __qdisc_run+0x1a4/0x250
kernel: [  239.983761]  [<c1375117>] dev_queue_xmit+0x367/0x4c0
kernel: [  239.983772]  [<c1374df0>] ? dev_queue_xmit+0x40/0x4c0
kernel: [  239.983783]  [<c13a2a1d>] ip_finish_output+0x1dd/0x2b0
kernel: [  239.983794]  [<c13a2b52>] ip_output+0x62/0xd0
kernel: [  239.983804]  [<c13a1a88>] ip_local_out+0x28/0x50
kernel: [  239.983814]  [<c13a21f4>] ip_queue_xmit+0x144/0x390
kernel: [  239.983825]  [<c1050a22>] ? lock_timer_base+0x32/0x70
kernel: [  239.983836]  [<c10758a8>] ? mark_held_locks+0x68/0xa0
kernel: [  239.983847]  [<c13ba998>] ? tcp_v4_send_check+0x58/0x100
kernel: [  239.983860]  [<c13b6462>] tcp_transmit_skb+0x3b2/0x750
kernel: [  239.983871]  [<c13b887e>] tcp_write_xmit+0x19e/0x950
kernel: [  239.983882]  [<c13b90cc>] __tcp_push_pending_frames+0x3c/0xb0
kernel: [  239.983894]  [<c13b3fa5>] tcp_rcv_established+0x145/0xa00
kernel: [  239.983905]  [<c13bc8c8>] tcp_v4_do_rcv+0x258/0x390
kernel: [  239.983915]  [<c13bcf5a>] ? tcp_v4_rcv+0x55a/0x7d0
kernel: [  239.983926]  [<c140b8b5>] ? _spin_lock_nested+0x65/0x80
kernel: [  239.983936]  [<c13bcf7a>] tcp_v4_rcv+0x57a/0x7d0
kernel: [  239.983948]  [<c139d687>] ip_local_deliver_finish+0xd7/0x270
kernel: [  239.983959]  [<c139d5e5>] ? ip_local_deliver_finish+0x35/0x270
kernel: [  239.983969]  [<c139d8c7>] ip_local_deliver+0xa7/0xb0
kernel: [  239.983979]  [<c139d00b>] ip_rcv_finish+0xeb/0x3d0
kernel: [  239.983990]  [<c139d4f2>] ip_rcv+0x202/0x2c0
kernel: [  239.984000]  [<c1373860>] ? netif_receive_skb+0x140/0x5e0
kernel: [  239.984012]  [<c1373a33>] netif_receive_skb+0x313/0x5e0
kernel: [  239.984023]  [<c1373860>] ? netif_receive_skb+0x140/0x5e0
kernel: [  239.984033]  [<c1373d6e>] ? process_backlog+0x6e/0xd0
kernel: [  239.984045]  [<c1373d78>] process_backlog+0x78/0xd0
kernel: [  239.984056]  [<c13716d0>] net_rx_action+0x160/0x260
kernel: [  239.984066]  [<c1371652>] ? net_rx_action+0xe2/0x260
kernel: [  239.984077]  [<c104abe0>] ? __do_softirq+0x50/0x1c0
kernel: [  239.984089]  [<c104ac15>] __do_softirq+0x85/0x1c0
kernel: [  239.984100]  [<c104adc5>] do_softirq+0x75/0x90
kernel: [  239.984111]  [<c104ae4e>] ksoftirqd+0x6e/0x110
kernel: [  239.984121]  [<c104ade0>] ? ksoftirqd+0x0/0x110
kernel: [  239.984132]  [<c105f965>] kthread+0x55/0xa0
kernel: [  239.984142]  [<c105f910>] ? kthread+0x0/0xa0
kernel: [  239.984153]  [<c1003edb>] kernel_thread_helper+0x7/0x1c

Comment 7 Sergey Senozhatsky 2009-06-20 13:41:41 UTC

Hello.

[ 3924.650779] =================================
[ 3924.650783] [ INFO: inconsistent lock state ]
[ 3924.650788] 2.6.30-dbgnv-git14 #5
[ 3924.650790] ---------------------------------
[ 3924.650794] inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
[ 3924.650798] ksoftirqd/0/4 [HC0[0]:SC1[5]:HE1:SE0] takes:
[ 3924.650802]  (&tty->termios_mutex){+.?...}, at: [<ffffffff8136bdc3>] tty_throttle+0x33/0x90
[ 3924.650816] {SOFTIRQ-ON-W} state was registered at:
[ 3924.650819]   [<ffffffff8109d8c3>] __lock_acquire+0x7f3/0x1ca0
[ 3924.650827]   [<ffffffff8109ee63>] lock_acquire+0xf3/0x140
[ 3924.650833]   [<ffffffff814dbcef>] mutex_lock_nested+0x5f/0x3f0
[ 3924.650839]   [<ffffffff8136bd33>] tty_unthrottle+0x33/0x90
[ 3924.650845]   [<ffffffff81369f86>] reset_buffer_flags+0xf6/0x140
[ 3924.650850]   [<ffffffff8136b410>] n_tty_open+0x50/0x170
[ 3924.650856]   [<ffffffff8136d4ca>] tty_ldisc_open+0x4a/0x90
[ 3924.650861]   [<ffffffff8136d990>] tty_ldisc_setup+0x40/0xc0
[ 3924.650867]   [<ffffffff81367650>] tty_init_dev+0xd0/0x1e0
[ 3924.650872]   [<ffffffff81367b3d>] tty_open+0x3dd/0x550
[ 3924.650877]   [<ffffffff8113b1cf>] chrdev_open+0x10f/0x200
[ 3924.650883]   [<ffffffff81134925>] __dentry_open+0xf5/0x320
[ 3924.650890]   [<ffffffff81134c97>] nameidata_to_filp+0x77/0x90
[ 3924.650895]   [<ffffffff811476aa>] do_filp_open+0x7ea/0xb90
[ 3924.650902]   [<ffffffff8113469f>] do_sys_open+0x9f/0x160
[ 3924.650907]   [<ffffffff811347ce>] sys_open+0x2e/0x50
[ 3924.650913]   [<ffffffff8100a22a>] init_post+0x4a/0x1c0
[ 3924.650919]   [<ffffffff81762a1d>] kernel_init+0x1e8/0x218
[ 3924.650926]   [<ffffffff8101568a>] child_rip+0xa/0x20
[ 3924.650933]   [<ffffffffffffffff>] 0xffffffffffffffff
[ 3924.650948] irq event stamp: 855504
[ 3924.650951] hardirqs last  enabled at (855504): [<ffffffff814de03d>] _spin_unlock_irqrestore+0x4d/0x90
[ 3924.650958] hardirqs last disabled at (855503): [<ffffffff814de4bc>] _spin_lock_irqsave+0x3c/0xc0
[ 3924.650964] softirqs last  enabled at (855278): [<ffffffff8106be3e>] __do_softirq+0x19e/0x210
[ 3924.650971] softirqs last disabled at (855287): [<ffffffff8101578c>] call_softirq+0x1c/0x30
[ 3924.650978] 
[ 3924.650979] other info that might help us debug this:
[ 3924.650983] 9 locks held by ksoftirqd/0/4:
[ 3924.650986]  #0:  (rcu_read_lock){.+.+..}, at: [<ffffffff814291a0>] net_rx_action+0x120/0x2d0
[ 3924.650998]  #1:  (rcu_read_lock){.+.+..}, at: [<ffffffff8142ba80>] netif_receive_skb+0x160/0x690
[ 3924.651008]  #2:  (rcu_read_lock){.+.+..}, at: [<ffffffff8145c280>] ip_local_deliver_finish+0x50/0x2c0
[ 3924.651020]  #3:  (slock-AF_INET/1){+.-...}, at: [<ffffffff8147ff5c>] tcp_v4_rcv+0x68c/0x8b0
[ 3924.651033]  #4:  (rcu_read_lock){.+.+..}, at: [<ffffffff8142d301>] dev_queue_xmit+0x61/0x520
[ 3924.651043]  #5:  (_xmit_PPP#2){+.-...}, at: [<ffffffff81445aea>] __qdisc_run+0x25a/0x2d0
[ 3924.651055]  #6:  (&ppp->wlock){+.-...}, at: [<ffffffffa03126b6>] ppp_xmit_process+0x36/0x630 [ppp_generic]
[ 3924.651069]  #7:  (&pch->downl){+.-...}, at: [<ffffffffa0311b3c>] ppp_push+0x1dc/0x6b0 [ppp_generic]
[ 3924.651081]  #8:  (&ap->xmit_lock){+.-...}, at: [<ffffffffa031ba49>] ppp_async_push+0x79/0x570 [ppp_async]
[ 3924.651093] 
[ 3924.651094] stack backtrace:
[ 3924.651099] Pid: 4, comm: ksoftirqd/0 Not tainted 2.6.30-dbgnv-git14 #5
[ 3924.651102] Call Trace:
[ 3924.651105]  <IRQ>  [<ffffffff8109b32a>] print_usage_bug+0x1ca/0x220
[ 3924.651116]  [<ffffffff810994e6>] ? save_trace+0x56/0x100
[ 3924.651122]  [<ffffffff8109c510>] ? check_usage_forwards+0x0/0xe0
[ 3924.651127]  [<ffffffff8109b749>] mark_lock+0x3c9/0x720
[ 3924.651133]  [<ffffffff8109d86f>] __lock_acquire+0x79f/0x1ca0
[ 3924.651138]  [<ffffffff8101d1d0>] ? native_sched_clock+0x20/0x70
[ 3924.651144]  [<ffffffff8101d233>] ? sched_clock+0x13/0x30
[ 3924.651149]  [<ffffffff8136bdc3>] ? tty_throttle+0x33/0x90
[ 3924.651154]  [<ffffffff810997b0>] ? trace_hardirqs_off+0x20/0x40
[ 3924.651160]  [<ffffffff8136bdc3>] ? tty_throttle+0x33/0x90
[ 3924.651165]  [<ffffffff8109ee63>] lock_acquire+0xf3/0x140
[ 3924.651170]  [<ffffffff8136bdc3>] ? tty_throttle+0x33/0x90
[ 3924.651176]  [<ffffffff814dbcef>] mutex_lock_nested+0x5f/0x3f0
[ 3924.651181]  [<ffffffff8136bdc3>] ? tty_throttle+0x33/0x90
[ 3924.651186]  [<ffffffff8109be21>] ? trace_hardirqs_on_caller+0x81/0x1f0
[ 3924.651191]  [<ffffffff8136bdc3>] ? tty_throttle+0x33/0x90
[ 3924.651197]  [<ffffffff8109bfb0>] ? trace_hardirqs_on+0x20/0x40
[ 3924.651202]  [<ffffffff8136bdc3>] tty_throttle+0x33/0x90
[ 3924.651208]  [<ffffffff8136a4b8>] n_tty_receive_buf+0x438/0x12c0
[ 3924.651213]  [<ffffffff8109bb18>] ? mark_held_locks+0x78/0xb0
[ 3924.651218]  [<ffffffff814de03d>] ? _spin_unlock_irqrestore+0x4d/0x90
[ 3924.651224]  [<ffffffff8109be21>] ? trace_hardirqs_on_caller+0x81/0x1f0
[ 3924.651229]  [<ffffffff8109bfb0>] ? trace_hardirqs_on+0x20/0x40
[ 3924.651235]  [<ffffffff8136cf49>] ? tty_ldisc_try+0x59/0x80
[ 3924.651241]  [<ffffffff8136fba4>] pty_write+0xa4/0xc0
[ 3924.651246]  [<ffffffff814de3c2>] ? _spin_lock_bh+0x72/0x90
[ 3924.651252]  [<ffffffffa031bdf5>] ppp_async_push+0x425/0x570 [ppp_async]
[ 3924.651259]  [<ffffffffa031bf9a>] ppp_async_send+0x5a/0x80 [ppp_async]
[ 3924.651267]  [<ffffffffa0311b59>] ppp_push+0x1f9/0x6b0 [ppp_generic]
[ 3924.651272]  [<ffffffff8109bfb0>] ? trace_hardirqs_on+0x20/0x40
[ 3924.651278]  [<ffffffff8142066d>] ? skb_dequeue+0x6d/0xa0
[ 3924.651286]  [<ffffffffa0312aa0>] ppp_xmit_process+0x420/0x630 [ppp_generic]
[ 3924.651291]  [<ffffffff814204b7>] ? skb_queue_tail+0x57/0x80
[ 3924.651298]  [<ffffffffa0312e10>] ppp_start_xmit+0x160/0x210 [ppp_generic]
[ 3924.651304]  [<ffffffff8142d05c>] dev_hard_start_xmit+0x2ec/0x3d0
[ 3924.651309]  [<ffffffff814de32c>] ? _spin_lock+0x6c/0x90
[ 3924.651315]  [<ffffffff81445b0e>] __qdisc_run+0x27e/0x2d0
[ 3924.651320]  [<ffffffff814de32c>] ? _spin_lock+0x6c/0x90
[ 3924.651325]  [<ffffffff8142d5e8>] dev_queue_xmit+0x348/0x520
[ 3924.651330]  [<ffffffff8142d301>] ? dev_queue_xmit+0x61/0x520
[ 3924.651336]  [<ffffffff814623bd>] ip_finish_output+0x2cd/0x320
[ 3924.651342]  [<ffffffff81462491>] ip_output+0x81/0xf0
[ 3924.651348]  [<ffffffff8146112f>] ip_local_out+0x2f/0x60
[ 3924.651353]  [<ffffffff81461a7b>] ip_queue_xmit+0x20b/0x410
[ 3924.651358]  [<ffffffff814de03d>] ? _spin_unlock_irqrestore+0x4d/0x90
[ 3924.651361]  [<ffffffff8141f10c>] ? __skb_clone+0x3c/0x130
[ 3924.651361]  [<ffffffff8147871b>] tcp_transmit_skb+0x4fb/0x7b0
[ 3924.651361]  [<ffffffff8109be21>] ? trace_hardirqs_on_caller+0x81/0x1f0
[ 3924.651361]  [<ffffffff8147ae23>] tcp_write_xmit+0x213/0xad0
[ 3924.651361]  [<ffffffff8147778a>] ? tcp_established_options+0x4a/0x100
[ 3924.651361]  [<ffffffff8147b78a>] __tcp_push_pending_frames+0x3a/0xb0
[ 3924.651361]  [<ffffffff81475bce>] tcp_rcv_established+0x19e/0xbd0
[ 3924.651361]  [<ffffffff8147f780>] tcp_v4_do_rcv+0x2a0/0x3f0
[ 3924.651361]  [<ffffffff812c77f0>] ? _raw_spin_trylock+0x30/0x80
[ 3924.651361]  [<ffffffff8147ff5c>] ? tcp_v4_rcv+0x68c/0x8b0
[ 3924.651361]  [<ffffffff8147ff7c>] tcp_v4_rcv+0x6ac/0x8b0
[ 3924.651361]  [<ffffffff8145c351>] ip_local_deliver_finish+0x121/0x2c0
[ 3924.651361]  [<ffffffff8145c280>] ? ip_local_deliver_finish+0x50/0x2c0
[ 3924.651361]  [<ffffffffa0315b69>] ? ppp_input+0x129/0x1f0 [ppp_generic]
[ 3924.651361]  [<ffffffff8145c59d>] ip_local_deliver+0xad/0xc0
[ 3924.651361]  [<ffffffff8145bba2>] ip_rcv_finish+0x152/0x490
[ 3924.651361]  [<ffffffff8145c199>] ip_rcv+0x2b9/0x350
[ 3924.651361]  [<ffffffff8142bc64>] netif_receive_skb+0x344/0x690
[ 3924.651361]  [<ffffffff8142ba80>] ? netif_receive_skb+0x160/0x690
[ 3924.651361]  [<ffffffff8142c033>] ? process_backlog+0x83/0x100
[ 3924.651361]  [<ffffffff8142c045>] process_backlog+0x95/0x100
[ 3924.651361]  [<ffffffff8142922f>] net_rx_action+0x1af/0x2d0
[ 3924.651361]  [<ffffffff814291a0>] ? net_rx_action+0x120/0x2d0
[ 3924.651361]  [<ffffffff8106bda2>] __do_softirq+0x102/0x210
[ 3924.651361]  [<ffffffff8106b100>] ? ksoftirqd+0x0/0x150
[ 3924.651361]  [<ffffffff8101578c>] call_softirq+0x1c/0x30
[ 3924.651361]  <EOI>  [<ffffffff81017fbd>] do_softirq+0xad/0x100
[ 3924.651361]  [<ffffffff8106b188>] ksoftirqd+0x88/0x150
[ 3924.651361]  [<ffffffff810837ee>] kthread+0xbe/0xd0
[ 3924.651361]  [<ffffffff8101568a>] child_rip+0xa/0x20
[ 3924.651361]  [<ffffffff81015010>] ? restore_args+0x0/0x30
[ 3924.651361]  [<ffffffff81083730>] ? kthread+0x0/0xd0
[ 3924.651361]  [<ffffffff81015680>] ? child_rip+0x0/0x20
[ 4126.989269] r8169: eth0: link up


cpu info:
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Core(TM)2 CPU          6400  @ 2.13GHz
(x86_x64)

Comment 8 Andrew Morton 2009-06-22 21:40:50 UTC

Thanks.  I think Alan might have just fixed this - can you retest Linus's
current tree?

(I marked this as a regression too)

Comment 9 Sergey Senozhatsky 2009-06-22 22:58:28 UTC

On (06/22/09 21:40), bugzilla-daemon@bugzilla.kernel.org wrote:
> --- Comment #8 from Andrew Morton <akpm@linux-foundation.org>  2009-06-22
> 21:40:50 ---
> Thanks.  I think Alan might have just fixed this - can you retest Linus's
> current tree?
> 
> (I marked this as a regression too)
> 
> -- 

Hello Andrew. 
I was writing `resolved` when kernel died.

//commit a6540f731d506d9e82444cf0020e716613d4c46c
//Author: Alan Cox <alan@linux.intel.com>
//Date:   Mon Jun 22 18:42:29 2009 +0100
//
//    ppp: Fix throttling bugs


Stack (I'll attach it)
kernel: [ 1373.158304] 
kernel: [ 1373.158308] =================================
kernel: [ 1373.158313] [ INFO: inconsistent lock state ]
kernel: [ 1373.158317] 2.6.30-dbgnv-git19 #1
kernel: [ 1373.158320] ---------------------------------
kernel: [ 1373.158324] inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
kernel: [ 1373.158328] ksoftirqd/0/4 [HC0[0]:SC1[5]:HE1:SE0] takes:
kernel: [ 1373.158332]  (&tty->termios_mutex){+.?...}, at: [<c12e4675>] tty_throttle+0x25/0x70
kernel: [ 1373.158346] {SOFTIRQ-ON-W} state was registered at:
kernel: [ 1373.158349]   [<c107792f>] __lock_acquire+0xacf/0x1840
kernel: [ 1373.158357]   [<c1078741>] lock_acquire+0xa1/0x120
kernel: [ 1373.158362]   [<c141a08c>] mutex_lock_nested+0x5c/0x2e0
kernel: [ 1373.158370]   [<c12e4605>] tty_unthrottle+0x25/0x70
kernel: [ 1373.158375]   [<c12e2a99>] reset_buffer_flags+0xc9/0x100
kernel: [ 1373.158381]   [<c12e3e40>] n_tty_open+0x40/0x160
kernel: [ 1373.158386]   [<c12e5a79>] tty_ldisc_open+0x39/0x70
kernel: [ 1373.158391]   [<c12e5e8b>] tty_ldisc_setup+0x2b/0x90
kernel: [ 1373.158397]   [<c12e07a9>] tty_init_dev+0x99/0x190
kernel: [ 1373.158402]   [<c12e0b8c>] tty_open+0x2ec/0x490
kernel: [ 1373.158407]   [<c10fe4e2>] chrdev_open+0xe2/0x1b0
kernel: [ 1373.158415]   [<c10f86d4>] __dentry_open+0xd4/0x290
kernel: [ 1373.158420]   [<c10f89bd>] nameidata_to_filp+0x6d/0x80
kernel: [ 1373.158426]   [<c1108f13>] do_filp_open+0x5f3/0x980
kernel: [ 1373.158432]   [<c10f842d>] do_sys_open+0x5d/0x130
kernel: [ 1373.158438]   [<c10f8594>] sys_open+0x34/0x50
kernel: [ 1373.158443]   [<c100134b>] init_post+0x4b/0x110
kernel: [ 1373.158449]   [<c15e04fc>] kernel_init+0x18b/0x1a6
kernel: [ 1373.158456]   [<c1003edb>] kernel_thread_helper+0x7/0x1c
kernel: [ 1373.158462]   [<ffffffff>] 0xffffffff
kernel: [ 1373.158473] irq event stamp: 1125702
kernel: [ 1373.158476] hardirqs last  enabled at (1125702): [<c141bd5f>] _spin_unlock_irqrestore+0x3f/0x70
kernel: [ 1373.158483] hardirqs last disabled at (1125701): [<c141c144>] _spin_lock_irqsave+0x24/0xa0
kernel: [ 1373.158490] softirqs last  enabled at (1125486): [<c104b21d>] __do_softirq+0x10d/0x1f0
kernel: [ 1373.158498] softirqs last disabled at (1125491): [<c104b375>] do_softirq+0x75/0x90
kernel: [ 1373.158504] 
kernel: [ 1373.158505] other info that might help us debug this:
kernel: [ 1373.158509] 9 locks held by ksoftirqd/0/4:
kernel: [ 1373.158512]  #0:  (rcu_read_lock){.+.+..}, at: [<c13819a2>] net_rx_action+0xe2/0x260
kernel: [ 1373.158524]  #1:  (rcu_read_lock){.+.+..}, at: [<c1383b30>] netif_receive_skb+0x140/0x5e0
kernel: [ 1373.158534]  #2:  (rcu_read_lock){.+.+..}, at: [<c13ad8a5>] ip_local_deliver_finish+0x35/0x270
kernel: [ 1373.158545]  #3:  (slock-AF_INET/1){+.-...}, at: [<c13cd21a>] tcp_v4_rcv+0x55a/0x7d0
kernel: [ 1373.158557]  #4:  (rcu_read_lock){.+.+..}, at: [<c13850c0>] dev_queue_xmit+0x40/0x4d0
kernel: [ 1373.158567]  #5:  (_xmit_PPP#2){+.-...}, at: [<c1399faa>] __qdisc_run+0x18a/0x250
kernel: [ 1373.158579]  #6:  (&ppp->wlock){+.-...}, at: [<f9099406>] ppp_xmit_process+0x26/0x570 [ppp_generic]
kernel: [ 1373.158593]  #7:  (&pch->downl){+.-...}, at: [<f9098980>] ppp_push+0x160/0x5f0 [ppp_generic]
kernel: [ 1373.158604]  #8:  (&ap->xmit_lock){+.-...}, at: [<f90b190d>] ppp_async_push+0x5d/0x4d0 [ppp_async]
kernel: [ 1373.158615] 
kernel: [ 1373.158616] stack backtrace:
kernel: [ 1373.158621] Pid: 4, comm: ksoftirqd/0 Not tainted 2.6.30-dbgnv-git19 #1
kernel: [ 1373.158625] Call Trace:
kernel: [ 1373.158631]  [<c14180dc>] ? printk+0x23/0x36
kernel: [ 1373.158636]  [<c107560e>] print_usage_bug+0x17e/0x1c0
kernel: [ 1373.158642]  [<c1075a10>] mark_lock+0x3c0/0x5c0
kernel: [ 1373.158647]  [<c1076420>] ? check_usage_forwards+0x0/0xc0
kernel: [ 1373.158653]  [<c10778ec>] __lock_acquire+0xa8c/0x1840
kernel: [ 1373.158658]  [<c1073fa9>] ? trace_hardirqs_off+0x19/0x40
kernel: [ 1373.158664]  [<c141bd7f>] ? _spin_unlock_irqrestore+0x5f/0x70
kernel: [ 1373.158670]  [<c1078741>] lock_acquire+0xa1/0x120
kernel: [ 1373.158675]  [<c12e4675>] ? tty_throttle+0x25/0x70
kernel: [ 1373.158680]  [<c12e4675>] ? tty_throttle+0x25/0x70
kernel: [ 1373.158686]  [<c141a08c>] mutex_lock_nested+0x5c/0x2e0
kernel: [ 1373.158691]  [<c12e4675>] ? tty_throttle+0x25/0x70
kernel: [ 1373.158696]  [<c1076019>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1373.158702]  [<c12e4675>] ? tty_throttle+0x25/0x70
kernel: [ 1373.158707]  [<c12e4675>] tty_throttle+0x25/0x70
kernel: [ 1373.158712]  [<c12e2f77>] n_tty_receive_buf+0x407/0x1220
kernel: [ 1373.158719]  [<c1075c78>] ? mark_held_locks+0x68/0xa0
kernel: [ 1373.158724]  [<c141bd5f>] ? _spin_unlock_irqrestore+0x3f/0x70
kernel: [ 1373.158730]  [<c1075ec8>] ? trace_hardirqs_on_caller+0x68/0x1a0
kernel: [ 1373.158736]  [<c1076019>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1373.158741]  [<c12e55f2>] ? tty_ldisc_try+0x42/0x60
kernel: [ 1373.158747]  [<c12e7add>] pty_write+0x7d/0x90
kernel: [ 1373.158753]  [<f90b1c8a>] ppp_async_push+0x3da/0x4d0 [ppp_async]
kernel: [ 1373.158760]  [<f90b1dc8>] ppp_async_send+0x48/0x70 [ppp_async]
kernel: [ 1373.158767]  [<f9098996>] ppp_push+0x176/0x5f0 [ppp_generic]
kernel: [ 1373.158773]  [<c141bd5f>] ? _spin_unlock_irqrestore+0x3f/0x70
kernel: [ 1373.158779]  [<c1075ec8>] ? trace_hardirqs_on_caller+0x68/0x1a0
kernel: [ 1373.158784]  [<c1076019>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1373.158790]  [<c137a443>] ? skb_dequeue+0x53/0x80
kernel: [ 1373.158797]  [<f90997bd>] ppp_xmit_process+0x3dd/0x570 [ppp_generic]
kernel: [ 1373.158803]  [<c1076019>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1373.158808]  [<c137a2f1>] ? skb_queue_tail+0x41/0x60
kernel: [ 1373.158815]  [<f9099ab7>] ppp_start_xmit+0x167/0x1b0 [ppp_generic]
kernel: [ 1373.158820]  [<c1399faa>] ? __qdisc_run+0x18a/0x250
kernel: [ 1373.158826]  [<c1384d8c>] dev_hard_start_xmit+0x1ec/0x3b0
kernel: [ 1373.158832]  [<c141bfe7>] ? _spin_lock+0x67/0x80
kernel: [ 1373.158837]  [<c1399fc4>] __qdisc_run+0x1a4/0x250
kernel: [ 1373.158843]  [<c13853f7>] dev_queue_xmit+0x377/0x4d0
kernel: [ 1373.158849]  [<c13850c0>] ? dev_queue_xmit+0x40/0x4d0
kernel: [ 1373.158855]  [<c13b2cdd>] ip_finish_output+0x1dd/0x2b0
kernel: [ 1373.158860]  [<c13b2e12>] ip_output+0x62/0xd0
kernel: [ 1373.158866]  [<c13b1d48>] ip_local_out+0x28/0x50
kernel: [ 1373.158871]  [<c13b24b4>] ip_queue_xmit+0x144/0x390
kernel: [ 1373.158877]  [<c10649ec>] ? hrtimer_interrupt+0x1bc/0x270
kernel: [ 1373.158883]  [<c1050ee2>] ? lock_timer_base+0x32/0x70
kernel: [ 1373.158888]  [<c1075c78>] ? mark_held_locks+0x68/0xa0
kernel: [ 1373.158894]  [<c13cac58>] ? tcp_v4_send_check+0x58/0x100
kernel: [ 1373.158899]  [<c13c6722>] tcp_transmit_skb+0x3b2/0x750
kernel: [ 1373.158905]  [<c13c8b3e>] tcp_write_xmit+0x19e/0x950
kernel: [ 1373.158911]  [<c13c938c>] __tcp_push_pending_frames+0x3c/0xb0
kernel: [ 1373.158916]  [<c13c4265>] tcp_rcv_established+0x145/0xa00
kernel: [ 1373.158922]  [<c13ccb88>] tcp_v4_do_rcv+0x258/0x390
kernel: [ 1373.158927]  [<c13cd21a>] ? tcp_v4_rcv+0x55a/0x7d0
kernel: [ 1373.158933]  [<c141bf65>] ? _spin_lock_nested+0x65/0x80
kernel: [ 1373.158939]  [<c13cd23a>] tcp_v4_rcv+0x57a/0x7d0
kernel: [ 1373.158950]  [<c13ad947>] ip_local_deliver_finish+0xd7/0x270
kernel: [ 1373.158955]  [<c13ad8a5>] ? ip_local_deliver_finish+0x35/0x270
kernel: [ 1373.158961]  [<c13adb87>] ip_local_deliver+0xa7/0xb0
kernel: [ 1373.158966]  [<c13ad2cb>] ip_rcv_finish+0xeb/0x3d0
kernel: [ 1373.158972]  [<c13ad7b2>] ip_rcv+0x202/0x2c0
kernel: [ 1373.158978]  [<c1383b30>] ? netif_receive_skb+0x140/0x5e0
kernel: [ 1373.158983]  [<c1383d03>] netif_receive_skb+0x313/0x5e0
kernel: [ 1373.158989]  [<c1383b30>] ? netif_receive_skb+0x140/0x5e0
kernel: [ 1373.158995]  [<c138403e>] ? process_backlog+0x6e/0xd0
kernel: [ 1373.159001]  [<c1384048>] process_backlog+0x78/0xd0
kernel: [ 1373.159006]  [<c1381a20>] net_rx_action+0x160/0x260
kernel: [ 1373.159012]  [<c13819a2>] ? net_rx_action+0xe2/0x260
kernel: [ 1373.159018]  [<c104b170>] ? __do_softirq+0x60/0x1f0
kernel: [ 1373.159023]  [<c104b1ba>] __do_softirq+0xaa/0x1f0
kernel: [ 1373.159029]  [<c104b375>] do_softirq+0x75/0x90
kernel: [ 1373.159035]  [<c104b3fe>] ksoftirqd+0x6e/0x110
kernel: [ 1373.159040]  [<c104b390>] ? ksoftirqd+0x0/0x110
kernel: [ 1373.159046]  [<c105f734>] kthread+0x84/0x90
kernel: [ 1373.159051]  [<c105f6b0>] ? kthread+0x0/0x90
kernel: [ 1373.159057]  [<c1003edb>] kernel_thread_helper+0x7/0x1c

/USR/SBIN/CRON[8943]: CMD (fetchmail -sk -m '/usr/bin/procmail' > /dev/null 2>&1)

/*
*here we go
*/
kernel: [ 1715.273674] BUG: scheduling while atomic: ksoftirqd/0/4/0x10000500
kernel: [ 1715.273679] INFO: lockdep is turned off.
kernel: [ 1715.273682] Modules linked in: ppp_deflate zlib_deflate ppp_async crc_ccitt ppp_generic slhc ipv6 fuse sbp2 loop snd_hda_codec_si3054 snd_hda_codec_realtek
snd_hda_intel snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq pcspkr snd_timer snd_seq_device psmouse i2c_i801 snd soundcore rng_core snd_page_alloc
serio_raw asus_laptop evdev usbhid hid sg sr_mod cdrom sd_mod ata_generic pata_acpi ricoh_mmc uhci_hcd ata_piix sdhci_pci sdhci ohci1394 ieee1394 mmc_core led_class
ide_pci_generic ehci_hcd r8169 mii usbcore
kernel: [ 1715.273768] Pid: 4, comm: ksoftirqd/0 Not tainted 2.6.30-dbgnv-git19 #1
kernel: [ 1715.273771] Call Trace:
kernel: [ 1715.273784]  [<c103b615>] __schedule_bug+0x95/0xa0
kernel: [ 1715.273791]  [<c14189f3>] schedule+0x5e3/0xc10
kernel: [ 1715.273798]  [<c1073fa9>] ? trace_hardirqs_off+0x19/0x40
kernel: [ 1715.273804]  [<c141bd7f>] ? _spin_unlock_irqrestore+0x5f/0x70
kernel: [ 1715.273810]  [<c1078845>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1715.273816]  [<c103e64b>] ? default_wake_function+0x1b/0x40
kernel: [ 1715.273821]  [<c103ef46>] __cond_resched+0x26/0x60
kernel: [ 1715.273826]  [<c1419172>] _cond_resched+0x42/0x50
kernel: [ 1715.273832]  [<c141a04d>] mutex_lock_nested+0x1d/0x2e0
kernel: [ 1715.273837]  [<c1076019>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1715.273843]  [<c12e4675>] tty_throttle+0x25/0x70
kernel: [ 1715.273849]  [<c12e2f77>] n_tty_receive_buf+0x407/0x1220
kernel: [ 1715.273854]  [<c1076019>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1715.273860]  [<c1078845>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1715.273865]  [<c1078845>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1715.273871]  [<c12e55d5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1715.273876]  [<c12e55d5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1715.273881]  [<c12e55d5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1715.273886]  [<c1076019>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1715.273891]  [<c12e55f2>] ? tty_ldisc_try+0x42/0x60
kernel: [ 1715.273897]  [<c12e7add>] pty_write+0x7d/0x90
kernel: [ 1715.273905]  [<f90b1c8a>] ppp_async_push+0x3da/0x4d0 [ppp_async]
kernel: [ 1715.273912]  [<f90b1dc8>] ppp_async_send+0x48/0x70 [ppp_async]
kernel: [ 1715.273920]  [<f9098996>] ppp_push+0x176/0x5f0 [ppp_generic]
kernel: [ 1715.273926]  [<c137a416>] ? skb_dequeue+0x26/0x80
kernel: [ 1715.273932]  [<c1076019>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1715.273937]  [<c137a443>] ? skb_dequeue+0x53/0x80
kernel: [ 1715.273944]  [<f90997bd>] ppp_xmit_process+0x3dd/0x570 [ppp_generic]
kernel: [ 1715.273950]  [<c1076019>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1715.273955]  [<c137a2f1>] ? skb_queue_tail+0x41/0x60
kernel: [ 1715.273962]  [<f9099ab7>] ppp_start_xmit+0x167/0x1b0 [ppp_generic]
kernel: [ 1715.273968]  [<c1399faa>] ? __qdisc_run+0x18a/0x250
kernel: [ 1715.273974]  [<c1384d8c>] dev_hard_start_xmit+0x1ec/0x3b0
kernel: [ 1715.273980]  [<c141bfe7>] ? _spin_lock+0x67/0x80
kernel: [ 1715.273985]  [<c1399fc4>] __qdisc_run+0x1a4/0x250
kernel: [ 1715.273991]  [<c13853f7>] dev_queue_xmit+0x377/0x4d0
kernel: [ 1715.273996]  [<c13850c0>] ? dev_queue_xmit+0x40/0x4d0
kernel: [ 1715.274008]  [<c13b2cdd>] ip_finish_output+0x1dd/0x2b0
kernel: [ 1715.274013]  [<c13b2e12>] ip_output+0x62/0xd0
kernel: [ 1715.274019]  [<c1078845>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1715.274025]  [<c13b1d48>] ip_local_out+0x28/0x50
kernel: [ 1715.274030]  [<c13b24b4>] ip_queue_xmit+0x144/0x390
kernel: [ 1715.274036]  [<c1050ee2>] ? lock_timer_base+0x32/0x70
kernel: [ 1715.274043]  [<c13cac58>] ? tcp_v4_send_check+0x58/0x100
kernel: [ 1715.274048]  [<c13c6722>] tcp_transmit_skb+0x3b2/0x750
kernel: [ 1715.274054]  [<c10f2960>] ? cache_free_debugcheck+0x210/0x290
kernel: [ 1715.274060]  [<c13c8b3e>] tcp_write_xmit+0x19e/0x950
kernel: [ 1715.274066]  [<c13c938c>] __tcp_push_pending_frames+0x3c/0xb0
kernel: [ 1715.274071]  [<c13c4265>] tcp_rcv_established+0x145/0xa00
kernel: [ 1715.274077]  [<c13ccb88>] tcp_v4_do_rcv+0x258/0x390
kernel: [ 1715.274082]  [<c13cd21a>] ? tcp_v4_rcv+0x55a/0x7d0
kernel: [ 1715.274087]  [<c13cd21a>] ? tcp_v4_rcv+0x55a/0x7d0
kernel: [ 1715.274092]  [<c141bf65>] ? _spin_lock_nested+0x65/0x80
kernel: [ 1715.274098]  [<c13cd23a>] tcp_v4_rcv+0x57a/0x7d0
kernel: [ 1715.274104]  [<c13ad947>] ip_local_deliver_finish+0xd7/0x270
kernel: [ 1715.274109]  [<c13ad8a5>] ? ip_local_deliver_finish+0x35/0x270
kernel: [ 1715.274114]  [<c13adb87>] ip_local_deliver+0xa7/0xb0
kernel: [ 1715.274120]  [<c10309dd>] ? cpuacct_charge+0x9d/0xc0
kernel: [ 1715.274125]  [<c1383b30>] ? netif_receive_skb+0x140/0x5e0
kernel: [ 1715.274131]  [<c13ad2cb>] ip_rcv_finish+0xeb/0x3d0
kernel: [ 1715.274136]  [<c13ad7b2>] ip_rcv+0x202/0x2c0
kernel: [ 1715.274141]  [<c1383b30>] ? netif_receive_skb+0x140/0x5e0
kernel: [ 1715.274147]  [<c1383d03>] netif_receive_skb+0x313/0x5e0
kernel: [ 1715.274152]  [<c1383b30>] ? netif_receive_skb+0x140/0x5e0
kernel: [ 1715.274158]  [<c1384048>] process_backlog+0x78/0xd0
kernel: [ 1715.274164]  [<c1381a20>] net_rx_action+0x160/0x260
kernel: [ 1715.274169]  [<c13819a2>] ? net_rx_action+0xe2/0x260
kernel: [ 1715.274176]  [<c104b1ba>] __do_softirq+0xaa/0x1f0
kernel: [ 1715.274182]  [<c104b375>] do_softirq+0x75/0x90
kernel: [ 1715.274187]  [<c104b3fe>] ksoftirqd+0x6e/0x110
kernel: [ 1715.274192]  [<c104b390>] ? ksoftirqd+0x0/0x110
kernel: [ 1715.274198]  [<c105f734>] kthread+0x84/0x90
kernel: [ 1715.274204]  [<c105f6b0>] ? kthread+0x0/0x90
kernel: [ 1715.274210]  [<c1003edb>] kernel_thread_helper+0x7/0x1c

	Sergey

Comment 10 Sergey Senozhatsky 2009-06-22 23:00:29 UTC

Created attachment 22057 [details]
git19

Comment 11 Andrew Morton 2009-06-22 23:23:55 UTC

Do you mean that you're running a kernel with Alan's patch but that kernel still
gets the above traces?

Comment 12 Sergey Senozhatsky 2009-06-23 09:13:19 UTC

On (06/22/09 23:23), bugzilla-daemon@bugzilla.kernel.org wrote:
> --- Comment #11 from Andrew Morton <akpm@linux-foundation.org>  2009-06-22
> 23:23:55 ---
> Do you mean that you're running a kernel with Alan's patch but that kernel
> still
> gets the above traces?
> 

Yes.

	Sergey

Comment 13 Sergey Senozhatsky 2009-06-23 10:14:18 UTC

Please watch attached file.
Kernel: 7e0338c..d888a4c  master     -> origin/master (-git19)

kernel: [  272.683938] Call Trace:
kernel: [  272.683938]  [<c14186dc>] ? printk+0x23/0x36
kernel: [  272.683938]  [<c1076cb0>] check_usage+0x380/0x420
kernel: [  272.683938]  [<c1076da9>] check_irq_usage+0x59/0xf0
kernel: [  272.683938]  [<c1077c94>] __lock_acquire+0xe54/0x1840
kernel: [  272.683938]  [<c1073f89>] ? trace_hardirqs_off+0x19/0x40
kernel: [  272.683938]  [<c141c37f>] ? _spin_unlock_irqrestore+0x5f/0x70
kernel: [  272.683938]  [<c1078721>] lock_acquire+0xa1/0x120
kernel: [  272.683938]  [<c12e4c75>] ? tty_throttle+0x25/0x70
kernel: [  272.683938]  [<c12e4c75>] ? tty_throttle+0x25/0x70
kernel: [  272.683938]  [<c141a68c>] mutex_lock_nested+0x5c/0x2e0
kernel: [  272.683938]  [<c12e4c75>] ? tty_throttle+0x25/0x70
kernel: [  272.683938]  [<c1075ff9>] ? trace_hardirqs_on+0x19/0x40
kernel: [  272.683938]  [<c12e4c75>] ? tty_throttle+0x25/0x70
kernel: [  272.683938]  [<c12e4c75>] tty_throttle+0x25/0x70
kernel: [  272.683938]  [<c12e3577>] n_tty_receive_buf+0x407/0x1220
kernel: [  272.683938]  [<c1075c58>] ? mark_held_locks+0x68/0xa0
kernel: [  272.683938]  [<c141c35f>] ? _spin_unlock_irqrestore+0x3f/0x70
kernel: [  272.683938]  [<c1075ea8>] ? trace_hardirqs_on_caller+0x68/0x1a0
kernel: [  272.683938]  [<c1075ff9>] ? trace_hardirqs_on+0x19/0x40
kernel: [  272.683938]  [<c12e5bf2>] ? tty_ldisc_try+0x42/0x60
kernel: [  272.683938]  [<c12e80dd>] pty_write+0x7d/0x90
kernel: [  272.683938]  [<f80afc8a>] ppp_async_push+0x3da/0x4d0 [ppp_async]
kernel: [  272.683938]  [<f80afdc8>] ppp_async_send+0x48/0x70 [ppp_async]
kernel: [  272.683938]  [<f809c996>] ppp_push+0x176/0x5f0 [ppp_generic]
kernel: [  272.683938]  [<c141c35f>] ? _spin_unlock_irqrestore+0x3f/0x70
kernel: [  272.683938]  [<c1075ea8>] ? trace_hardirqs_on_caller+0x68/0x1a0
kernel: [  272.683938]  [<c1075ff9>] ? trace_hardirqs_on+0x19/0x40
kernel: [  272.683938]  [<c137aa43>] ? skb_dequeue+0x53/0x80
kernel: [  272.683938]  [<f809d7bd>] ppp_xmit_process+0x3dd/0x570 [ppp_generic]
kernel: [  272.683938]  [<c1075ff9>] ? trace_hardirqs_on+0x19/0x40
kernel: [  272.683938]  [<c137a8f1>] ? skb_queue_tail+0x41/0x60
kernel: [  272.683938]  [<f809dab7>] ppp_start_xmit+0x167/0x1b0 [ppp_generic]
kernel: [  272.683938]  [<c139a5aa>] ? __qdisc_run+0x18a/0x250
kernel: [  272.683938]  [<c138538c>] dev_hard_start_xmit+0x1ec/0x3b0
kernel: [  272.683938]  [<c141c5e7>] ? _spin_lock+0x67/0x80
kernel: [  272.683938]  [<c139a5c4>] __qdisc_run+0x1a4/0x250
kernel: [  272.683938]  [<c13859f7>] dev_queue_xmit+0x377/0x4d0
kernel: [  272.683938]  [<c13856c0>] ? dev_queue_xmit+0x40/0x4d0
kernel: [  272.683938]  [<c13b32dd>] ip_finish_output+0x1dd/0x2b0
kernel: [  272.683938]  [<c13b3412>] ip_output+0x62/0xd0
kernel: [  272.683938]  [<c1073ffd>] ? lock_release_holdtime+0x4d/0x210
kernel: [  272.683938]  [<c13b2348>] ip_local_out+0x28/0x50
kernel: [  272.683938]  [<c13b2ab4>] ip_queue_xmit+0x144/0x390
kernel: [  272.683938]  [<c10c3d7c>] ? unlock_page+0x4c/0x70
kernel: [  272.683938]  [<c13cb258>] ? tcp_v4_send_check+0x58/0x100
kernel: [  272.683938]  [<c13c6d22>] tcp_transmit_skb+0x3b2/0x750
kernel: [  272.683938]  [<c1073ffd>] ? lock_release_holdtime+0x4d/0x210
kernel: [  272.683938]  [<c13c913e>] tcp_write_xmit+0x19e/0x950
kernel: [  272.683938]  [<c13c992c>] tcp_push_one+0x3c/0x60
kernel: [  272.683938]  [<c13bc511>] tcp_sendmsg+0x961/0xa20
kernel: [  272.683938]  [<c13732bf>] sock_sendmsg+0x10f/0x130
kernel: [  272.683938]  [<c105fac0>] ? autoremove_wake_function+0x0/0x60
kernel: [  272.683938]  [<c13741ce>] sys_sendto+0x10e/0x150
kernel: [  272.683938]  [<c1073ffd>] ? lock_release_holdtime+0x4d/0x210
kernel: [  272.683938]  [<c10792b3>] ? lock_release_non_nested+0xa3/0x2a0
kernel: [  272.683938]  [<c10db973>] ? might_fault+0x43/0xb0
kernel: [  272.683938]  [<c10db973>] ? might_fault+0x43/0xb0
kernel: [  272.683938]  [<c1374251>] sys_send+0x41/0x60
kernel: [  272.683938]  [<c1374ec8>] sys_socketcall+0x1c8/0x290
kernel: [  272.683938]  [<c100328a>] ? sysenter_exit+0xf/0x1a
kernel: [  272.683938]  [<c100325b>] sysenter_do_call+0x12/0x32

	Sergey

Comment 14 Sergey Senozhatsky 2009-06-23 10:15:35 UTC

Created attachment 22065 [details]
git19 (rtorrent was running)

Comment 15 Sergey Senozhatsky 2009-06-23 10:21:40 UTC

kernel: [ 1631.073371] BUG: scheduling while atomic: pptpgw/3169/0x10000500
kernel: [ 1631.073380] INFO: lockdep is turned off.
kernel: [ 1631.073386] Modules linked in: ppp_deflate zlib_deflate ppp_async crc_ccitt ppp_generic slhc ipv6 fuse sbp2 loop snd_hda_codec_si3054 snd_hda_codec_realtek
snd_hda_intel snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device i2c_i801 pcspkr snd psmouse serio_raw rng_core soundcore
snd_page_alloc asus_laptop evdev usbhid hid sg sr_mod cdrom sd_mod ata_generic pata_acpi sdhci_pci sdhci ohci1394 ata_piix ide_pci_generic uhci_hcd ricoh_mmc mmc_core led_class
ieee1394 ehci_hcd r8169 mii usbcore
kernel: [ 1631.073544] 
kernel: [ 1631.073554] Pid: 3169, comm: pptpgw Not tainted (2.6.30-dbgnv-git19 #1) F3JC                
kernel: [ 1631.073563] EIP: 0060:[<c141c361>] EFLAGS: 00200246 CPU: 0
kernel: [ 1631.073578] EIP is at _spin_unlock_irqrestore+0x41/0x70
kernel: [ 1631.073586] EAX: 00000000 EBX: f5ae2620 ECX: 00000000 EDX: 0000008f
kernel: [ 1631.073593] ESI: 00200246 EDI: 00000001 EBP: f63bded8 ESP: f63bdecc
kernel: [ 1631.073601]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
kernel: [ 1631.073609] CR0: 8005003b CR2: b73979d5 CR3: 36236000 CR4: 000006d0
kernel: [ 1631.073616] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
kernel: [ 1631.073623] DR6: ffff0ff0 DR7: 00000400
kernel: [ 1631.073629] Call Trace:
kernel: [ 1631.073644]  [<c1031083>] __wake_up+0x53/0x70
kernel: [ 1631.073656]  [<c12df2cb>] tty_write_unlock+0x3b/0x60
kernel: [ 1631.073666]  [<c12df7a6>] tty_write+0x1d6/0x250
kernel: [ 1631.073679]  [<c10fa84f>] ? rw_verify_area+0x6f/0xf0
kernel: [ 1631.073689]  [<c12e2380>] ? n_tty_write+0x0/0x420
kernel: [ 1631.073701]  [<c10fb43e>] vfs_write+0xbe/0x1c0
kernel: [ 1631.073710]  [<c12df5d0>] ? tty_write+0x0/0x250
kernel: [ 1631.073721]  [<c10fb648>] sys_write+0x58/0xa0
kernel: [ 1631.073733]  [<c100325b>] sysenter_do_call+0x12/0x32


	Sergey

Comment 16 Sergey Senozhatsky 2009-06-24 22:27:42 UTC

git22
kernel: [ 2284.749937] BUG: scheduling while atomic: pptpgw/3091/0x10000500
kernel: [ 2284.749946] Modules linked in: ppp_deflate zlib_deflate ppp_async crc_ccitt ppp_generic slhc ipv6 fuse sbp2 loop snd_hda_codec_si3054 snd_hda_codec_realtek
snd_hda_intel snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device snd rng_core soundcore i2c_i801 snd_page_alloc psmouse asus_laptop
pcspkr evdev serio_raw usbhid hid sg sr_mod sd_mod cdrom sdhci_pci sdhci mmc_core ata_generic pata_acpi ohci1394 uhci_hcd led_class ricoh_mmc ieee1394 ehci_hcd ata_piix
ide_pci_generic usbcore r8169 mii
kernel: [ 2284.750059] 
kernel: [ 2284.750067] Pid: 3091, comm: pptpgw Not tainted (2.6.30-git22 #1) F3JC                
kernel: [ 2284.750075] EIP: 0060:[<c1048f43>] EFLAGS: 00200246 CPU: 0
kernel: [ 2284.750089] EIP is at __tasklet_schedule+0x53/0x80
kernel: [ 2284.750096] EAX: 00000000 EBX: 00200246 ECX: 00000000 EDX: 00000000
kernel: [ 2284.750102] ESI: 00000000 EDI: f6736000 EBP: f629de70 ESP: f629de68
kernel: [ 2284.750109]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
kernel: [ 2284.750115] CR0: 8005003b CR2: b6f0c000 CR3: 36888000 CR4: 000006d0
kernel: [ 2284.750122] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
kernel: [ 2284.750128] DR6: ffff0ff0 DR7: 00000400
kernel: [ 2284.750133] Call Trace:
kernel: [ 2284.750148]  [<f809e76c>] ppp_asynctty_receive+0x3dc/0x510 [ppp_async]
kernel: [ 2284.750162]  [<c12b434d>] pty_write+0x7d/0x90
kernel: [ 2284.750172]  [<c12aea43>] n_tty_write+0x303/0x410
kernel: [ 2284.750185]  [<c103cfa0>] ? default_wake_function+0x0/0x40
kernel: [ 2284.750194]  [<c12abd83>] tty_write+0x173/0x250
kernel: [ 2284.750206]  [<c10e845f>] ? rw_verify_area+0x6f/0xf0
kernel: [ 2284.750215]  [<c12ae740>] ? n_tty_write+0x0/0x410
kernel: [ 2284.750224]  [<c10e904e>] vfs_write+0xbe/0x1c0
kernel: [ 2284.750233]  [<c12abc10>] ? tty_write+0x0/0x250
kernel: [ 2284.750242]  [<c10e9258>] sys_write+0x58/0xa0
kernel: [ 2284.750252]  [<c100324b>] sysenter_do_call+0x12/0x22


	Sergey

Comment 17 Sergey Senozhatsky 2009-06-29 20:18:24 UTC

Hello.
rc1-git5.

kernel: [  984.626653] BUG: scheduling while atomic: pptpgw/3347/0x10000500
kernel: [  984.626661] Modules linked in: ppp_deflate zlib_deflate ppp_async crc_ccitt ppp_generic slhc ipv6 fuse loop snd_hda_codec_si3054 snd_hda_codec_realtek snd_hda_intel
snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device snd soundcore pcspkr psmouse rng_core serio_raw i2c_i801 snd_page_alloc
asus_laptop evdev usbhid hid sg sr_mod cdrom sd_mod ata_generic pata_acpi ricoh_mmc sdhci_pci uhci_hcd sdhci mmc_core led_class ehci_hcd r8169 mii ata_piix ide_pci_generic usbcore
kernel: [  984.626763] 
kernel: [  984.626771] Pid: 3347, comm: pptpgw Not tainted (2.6.31-rc1-dbgnv-git5 #2) F3JC                
kernel: [  984.626780] EIP: 0060:[<c103a1fc>] EFLAGS: 00200246 CPU: 0
kernel: [  984.626794] EIP is at finish_task_switch+0x4c/0xc0
kernel: [  984.626800] EAX: 00000000 EBX: c1eaa380 ECX: 00000001 EDX: c1407ee0
kernel: [  984.626807] ESI: f5c72200 EDI: f72fc070 EBP: e4f87f20 ESP: e4f87f0c
kernel: [  984.626813]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
kernel: [  984.626819] CR0: 8005003b CR2: b6f399ae CR3: 24f80000 CR4: 000006d0
kernel: [  984.626825] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
kernel: [  984.626831] DR6: ffff0ff0 DR7: 00000400
kernel: [  984.626836] Call Trace:
kernel: [  984.626848]  [<c13fcfef>] schedule+0x41f/0xb20
kernel: [  984.626859]  [<c10ecdb8>] ? sys_read+0x58/0xa0
kernel: [  984.626869]  [<c10033a2>] work_resched+0x5/0x1a
kernel: [ 1051.216759] BUG: scheduling while atomic: pptpgw/3347/0x10000500
kernel: [ 1051.216768] Modules linked in: ppp_deflate zlib_deflate ppp_async crc_ccitt ppp_generic slhc ipv6 fuse loop snd_hda_codec_si3054 snd_hda_codec_realtek snd_hda_intel
snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device snd soundcore pcspkr psmouse rng_core serio_raw i2c_i801 snd_page_alloc
asus_laptop evdev usbhid hid sg sr_mod cdrom sd_mod ata_generic pata_acpi ricoh_mmc sdhci_pci uhci_hcd sdhci mmc_core led_class ehci_hcd r8169 mii ata_piix ide_pci_generic usbcore
kernel: [ 1051.216869] 
kernel: [ 1051.216878] Pid: 3347, comm: pptpgw Not tainted (2.6.31-rc1-dbgnv-git5 #2) F3JC                
kernel: [ 1051.216886] EIP: 0060:[<c1049883>] EFLAGS: 00200246 CPU: 0
kernel: [ 1051.216901] EIP is at __tasklet_schedule+0x53/0x80
kernel: [ 1051.216907] EAX: 00000000 EBX: 00200246 ECX: 00000000 EDX: 00000000
kernel: [ 1051.216914] ESI: 00000000 EDI: e556dd10 EBP: e4f87e70 ESP: e4f87e68
kernel: [ 1051.216921]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
kernel: [ 1051.216927] CR0: 8005003b CR2: b7085000 CR3: 24f80000 CR4: 000006d0
kernel: [ 1051.216934] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
kernel: [ 1051.216939] DR6: ffff0ff0 DR7: 00000400
kernel: [ 1051.216945] Call Trace:
kernel: [ 1051.216960]  [<f80b376c>] ppp_asynctty_receive+0x3dc/0x510 [ppp_async]
kernel: [ 1051.216975]  [<c12d161d>] pty_write+0x7d/0x90
kernel: [ 1051.216985]  [<c12cbd13>] n_tty_write+0x303/0x410
kernel: [ 1051.216996]  [<c103d8f0>] ? default_wake_function+0x0/0x40
kernel: [ 1051.217005]  [<c12c8ef3>] tty_write+0x173/0x250
kernel: [ 1051.217015]  [<c10ebcaf>] ? rw_verify_area+0x6f/0xf0
kernel: [ 1051.217025]  [<c12cba10>] ? n_tty_write+0x0/0x410
kernel: [ 1051.217034]  [<c10ec89e>] vfs_write+0xbe/0x1c0
kernel: [ 1051.217043]  [<c12c8d80>] ? tty_write+0x0/0x250
kernel: [ 1051.217052]  [<c10ecaa8>] sys_write+0x58/0xa0
kernel: [ 1051.217062]  [<c100324b>] sysenter_do_call+0x12/0x22




kernel: [  984.626848]  [<c13fcfef>] schedule+0x41f/0xb20
kernel: [  984.626859]  [<c10ecdb8>] ? sys_read+0x58/0xa0
kernel: [  984.626869]  [<c10033a2>] work_resched+0x5/0x1a
is very suspicious.

	Sergey

Comment 18 Rafael J. Wysocki 2009-06-29 23:26:10 UTC

References : http://lkml.org/lkml/2009/6/29/237

Comment 19 Sergey Senozhatsky 2009-06-30 08:33:43 UTC

kernel: [ 1474.020101] BUG: scheduling while atomic: pptpgw/3180/0x10000500
kernel: [ 1474.020111] INFO: lockdep is turned off.
kernel: [ 1474.020117] Modules linked in: ppp_deflate zlib_deflate ppp_async crc_ccitt ppp_generic slhc ipv6 fuse loop snd_hda_codec_si3054 snd_hda_codec_realtek snd_hda_intel
snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device psmouse i2c_i801 pcspkr serio_raw rng_core snd soundcore snd_page_alloc
asus_laptop evdev usbhid hid sg sr_mod cdrom sd_mod ata_generic pata_acpi uhci_hcd ata_piix sdhci_pci sdhci mmc_core ide_pci_generic led_class ricoh_mmc ehci_hcd usbcore r8169 mii
kernel: [ 1474.020256] 
kernel: [ 1474.020265] Pid: 3180, comm: pptpgw Not tainted (2.6.31-rc1-dbgnv-git6 #1) F3JC                
kernel: [ 1474.020274] EIP: 0060:[<c11300c7>] EFLAGS: 00200286 CPU: 0
kernel: [ 1474.020289] EIP is at inotify_inode_queue_event+0x17/0x110
kernel: [ 1474.020297] EAX: 00000000 EBX: c2eb30b8 ECX: 00000000 EDX: 00000002
kernel: [ 1474.020304] ESI: 0000003d EDI: f6d5de38 EBP: ed529f54 ESP: ed529f18
kernel: [ 1474.020312]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
kernel: [ 1474.020319] CR0: 8005003b CR2: b7c28000 CR3: 02665000 CR4: 000006d0
kernel: [ 1474.020327] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
kernel: [ 1474.020334] DR6: ffff0ff0 DR7: 00000400
kernel: [ 1474.020340] Call Trace:
kernel: [ 1474.020355]  [<c12de81f>] ? tty_write+0x1df/0x250
kernel: [ 1474.020367]  [<c10fae6f>] ? rw_verify_area+0x6f/0xf0
kernel: [ 1474.020377]  [<c12e1860>] ? n_tty_write+0x0/0x420
kernel: [ 1474.020387]  [<c10faff9>] vfs_write+0x109/0x1c0
kernel: [ 1474.020398]  [<c12de640>] ? tty_write+0x0/0x250
kernel: [ 1474.020408]  [<c10fba38>] sys_write+0x58/0xa0
kernel: [ 1474.020420]  [<c100339b>] sysenter_do_call+0x12/0x32

EIP is at inotify_inode_queue_event+0x17/0x110
...

	Sergey

Comment 20 Sergey Senozhatsky 2009-06-30 21:59:40 UTC

Hello,
rc1-git6.

dmesg | ./scripts/markup_oops.pl

 	*__get_cpu_var(tasklet_vec).tail = t; 
 	__get_cpu_var(tasklet_vec).tail = &(t->next); 
 	raise_softirq_irqoff(TASKLET_SOFTIRQ); 
 	local_irq_restore(flags); 
 c104c4fb:	f7 c6 00 02 00 00    	test   $0x200,%esi 
 c104c501:	74 1d                	je     c104c520 <__tasklet_schedule+0x80> 
 c104c503:	e8 78 b5 02 00       	call   c1077a80 <trace_hardirqs_on> 
 	return flags; 
 } 
  
 static inline void native_restore_fl(unsigned long flags) 
 { 
 	asm volatile("push %0 ; popf" 
 c104c508:	56                   	push   %esi                |  %esi => 200246 
 c104c509:	9d                   	popf    
 } 
*c104c50a:	8b 45 f4             	mov    -0xc(%ebp),%eax     |  %eax = 0 <--- faulting instruction
 c104c50d:	65 33 05 14 00 00 00 	xor    %gs:0x14,%eax 
 c104c514:	75 13                	jne    c104c529 <__tasklet_schedule+0x89> 
 c104c516:	83 c4 04             	add    $0x4,%esp 
 c104c519:	5b                   	pop    %ebx 
 c104c51a:	5e                   	pop    %esi 
 c104c51b:	c9                   	leave   
 c104c51c:	c3                   	ret     
 c104c51d:	8d 76 00             	lea    0x0(%esi),%esi 
 c104c520:	56                   	push   %esi 
 c104c521:	9d                   	popf    
 	local_irq_save(flags); 
 	t->next = NULL; 
 	*__get_cpu_var(tasklet_vec).tail = t; 
 	__get_cpu_var(tasklet_vec).tail = &(t->next); 
 	raise_softirq_irqoff(TASKLET_SOFTIRQ); 
 	local_irq_restore(flags); 
 c104c522:	e8 29 7d 02 00       	call   c1074250 <trace_hardirqs_off> 
 c104c527:	eb e1                	jmp    c104c50a <__tasklet_schedule+0x6a> 
 } 
 c104c529:	e8 c2 85 ff ff       	call   c1044af0 <__stack_chk_fail> 
 c104c52e:	66 90                	xchg   %ax,%ax 
  
 c104c530 <raise_softirq>: 
 	if (!in_interrupt())




syslog:

kernel: [47080.294012] BUG: scheduling while atomic: pptpgw/6053/0x10000400
kernel: [47080.294022] INFO: lockdep is turned off.
kernel: [47080.294028] Modules linked in: ppp_deflate zlib_deflate ppp_async crc_ccitt ppp_generic slhc ipv6 fuse loop snd_hda_codec_si3054 snd_hda_codec_realtek snd_hda_intel
snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device pcspkr psmouse snd i2c_i801 rng_core serio_raw soundcore snd_page_alloc
asus_laptop evdev sg usbhid sr_mod hid cdrom sd_mod ata_generic pata_acpi ata_piix ricoh_mmc sdhci_pci sdhci ide_pci_generic uhci_hcd mmc_core led_class ehci_hcd usbcore r8169 mii
kernel: [47080.294165] 
kernel: [47080.294175] Pid: 6053, comm: pptpgw Not tainted (2.6.31-rc1-dbgnv-git6 #1) F3JC                
kernel: [47080.294184] EIP: 0060:[<c104c50a>] EFLAGS: 00200246 CPU: 0
kernel: [47080.294198] EIP is at __tasklet_schedule+0x6a/0x90
kernel: [47080.294205] EAX: 00000000 EBX: c31cf17c ECX: 00000000 EDX: 00000000
kernel: [47080.294213] ESI: 00200246 EDI: 00000fff EBP: c3271e60 ESP: c3271e54
kernel: [47080.294220]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
kernel: [47080.294228] CR0: 8005003b CR2: b7462000 CR3: 29de4000 CR4: 000006d0
kernel: [47080.294235] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
kernel: [47080.294242] DR6: ffff0ff0 DR7: 00000400
kernel: [47080.294248] Call Trace:
kernel: [47080.294263]  [<f80b238b>] ppp_asynctty_wakeup+0x6b/0x80 [ppp_async]
kernel: [47080.294276]  [<c12dea72>] tty_wakeup+0x72/0x90
kernel: [47080.294287]  [<c12e7603>] pty_unthrottle+0x23/0x50
kernel: [47080.294296]  [<c12e4072>] tty_unthrottle+0x42/0x70
kernel: [47080.294306]  [<c12e254f>] n_tty_read+0x48f/0x730
kernel: [47080.294320]  [<c1039f20>] ? default_wake_function+0x0/0x40
kernel: [47080.294332]  [<c12dd1b0>] tty_read+0xa0/0xe0
kernel: [47080.294341]  [<c12e20c0>] ? n_tty_read+0x0/0x730
kernel: [47080.294352]  [<c10fb30b>] vfs_read+0xbb/0x1c0
kernel: [47080.294362]  [<c12dd110>] ? tty_read+0x0/0xe0
kernel: [47080.294372]  [<c10fbad8>] sys_read+0x58/0xa0
kernel: [47080.294383]  [<c100339b>] sysenter_do_call+0x12/0x32
kernel: [47080.294401] BUG: scheduling while atomic: pptpgw/6053/0x00000400
kernel: [47080.294408] INFO: lockdep is turned off.
kernel: [47080.294413] Modules linked in: ppp_deflate zlib_deflate ppp_async crc_ccitt ppp_generic slhc ipv6 fuse loop snd_hda_codec_si3054 snd_hda_codec_realtek snd_hda_intel
snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device pcspkr psmouse snd i2c_i801 rng_core serio_raw soundcore snd_page_alloc
asus_laptop evdev sg usbhid sr_mod hid cdrom sd_mod ata_generic pata_acpi ata_piix ricoh_mmc sdhci_pci sdhci ide_pci_generic uhci_hcd mmc_core led_class ehci_hcd usbcore r8169 mii
kernel: [47080.294548] 
kernel: [47080.294555] Pid: 6053, comm: pptpgw Not tainted (2.6.31-rc1-dbgnv-git6 #1) F3JC                
kernel: [47080.294564] EIP: 0060:[<c104c50a>] EFLAGS: 00200246 CPU: 0
kernel: [47080.294572] EIP is at __tasklet_schedule+0x6a/0x90
kernel: [47080.294579] EAX: 00000000 EBX: c31cf17c ECX: 00000000 EDX: 00000000
kernel: [47080.294586] ESI: 00200246 EDI: 00000fff EBP: c3271e60 ESP: c3271e54
kernel: [47080.294593]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
kernel: [47080.294600] CR0: 8005003b CR2: b7462000 CR3: 29de4000 CR4: 000006d0
kernel: [47080.294607] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
kernel: [47080.294614] DR6: ffff0ff0 DR7: 00000400
kernel: [47080.294619] Call Trace:
kernel: [47080.294631]  [<f80b238b>] ppp_asynctty_wakeup+0x6b/0x80 [ppp_async]
kernel: [47080.294641]  [<c12dea72>] tty_wakeup+0x72/0x90
kernel: [47080.294651]  [<c12e7603>] pty_unthrottle+0x23/0x50
kernel: [47080.294661]  [<c12e4072>] tty_unthrottle+0x42/0x70
kernel: [47080.294670]  [<c12e254f>] n_tty_read+0x48f/0x730
kernel: [47080.294683]  [<c1039f20>] ? default_wake_function+0x0/0x40
kernel: [47080.294695]  [<c12dd1b0>] tty_read+0xa0/0xe0
kernel: [47080.294704]  [<c12e20c0>] ? n_tty_read+0x0/0x730
kernel: [47080.294714]  [<c10fb30b>] vfs_read+0xbb/0x1c0
kernel: [47080.294724]  [<c12dd110>] ? tty_read+0x0/0xe0
kernel: [47080.294733]  [<c10fbad8>] sys_read+0x58/0xa0
kernel: [47080.294744]  [<c100339b>] sysenter_do_call+0x12/0x32
pptp[6064]: anon log[logecho:pptp_ctrl.c:677]: Echo Reply received.
kernel: [47113.649464] INFO: RCU detected CPU 1 stall (t=14044096/3000 jiffies)
kernel: [47113.649464] Pid: 6052, comm: pppd Not tainted 2.6.31-rc1-dbgnv-git6 #1
kernel: [47113.649464] Call Trace:
kernel: [47113.649464]  [<c1417a58>] ? printk+0x23/0x36
kernel: [47113.649464]  [<c10a8220>] __rcu_pending+0x140/0x210
kernel: [47113.649464]  [<c10a831a>] rcu_pending+0x2a/0x70
kernel: [47113.649464]  [<c1051bdf>] update_process_times+0x3f/0x80
kernel: [47113.649464]  [<c10714df>] tick_sched_timer+0x6f/0xf0
kernel: [47113.649464]  [<c10640f6>] __run_hrtimer+0x56/0xe0
kernel: [47113.649464]  [<c1071470>] ? tick_sched_timer+0x0/0xf0
kernel: [47113.649464]  [<c1071470>] ? tick_sched_timer+0x0/0xf0
kernel: [47113.649464]  [<c1064af5>] hrtimer_interrupt+0x145/0x270
kernel: [47113.649464]  [<c1070ea6>] ? tick_program_event+0x36/0x60
kernel: [47113.649464]  [<c101c4bc>] smp_apic_timer_interrupt+0x5c/0xb0
kernel: [47113.649464]  [<c12582b8>] ? trace_hardirqs_off_thunk+0xc/0x14
kernel: [47113.649464]  [<c1003e36>] apic_timer_interrupt+0x36/0x3c
kernel: [47113.649464]  [<c1257c46>] ? delay_tsc+0x26/0x80
kernel: [47113.649464]  [<c1257ba8>] __delay+0x18/0x30
kernel: [47113.649464]  [<c125d956>] _raw_spin_lock+0xf6/0x170
kernel: [47113.649464]  [<f8097e94>] ? ppp_channel_push+0x24/0xd0 [ppp_generic]
kernel: [47113.649464]  [<c141b3d2>] _spin_lock_bh+0x62/0x90
kernel: [47113.649464]  [<f8097e94>] ? ppp_channel_push+0x24/0xd0 [ppp_generic]
kernel: [47113.649464]  [<f8097e94>] ppp_channel_push+0x24/0xd0 [ppp_generic]
kernel: [47113.649464]  [<f8098057>] ppp_write+0xd7/0xe0 [ppp_generic]
kernel: [47113.649464]  [<c10fafae>] vfs_write+0xbe/0x1c0
kernel: [47113.649464]  [<f8097f80>] ? ppp_write+0x0/0xe0 [ppp_generic]
kernel: [47113.649464]  [<c10fba38>] sys_write+0x58/0xa0
kernel: [47113.649464]  [<c100339b>] sysenter_do_call+0x12/0x32
/USR/SBIN/CRON[6450]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
kernel: [47143.649461] INFO: RCU detected CPU 1 stall (t=14053096/12000 jiffies)
kernel: [47143.649461] Pid: 6052, comm: pppd Not tainted 2.6.31-rc1-dbgnv-git6 #1
kernel: [47143.649461] Call Trace:
kernel: [47143.649461]  [<c1417a58>] ? printk+0x23/0x36
kernel: [47143.649461]  [<c10a8220>] __rcu_pending+0x140/0x210
kernel: [47143.649461]  [<c10a831a>] rcu_pending+0x2a/0x70
kernel: [47143.649461]  [<c1051bdf>] update_process_times+0x3f/0x80
kernel: [47143.649461]  [<c10714df>] tick_sched_timer+0x6f/0xf0
kernel: [47143.649461]  [<c10640f6>] __run_hrtimer+0x56/0xe0
kernel: [47143.649461]  [<c1071470>] ? tick_sched_timer+0x0/0xf0
kernel: [47143.649461]  [<c1071470>] ? tick_sched_timer+0x0/0xf0
kernel: [47143.649461]  [<c1064af5>] hrtimer_interrupt+0x145/0x270
kernel: [47143.649461]  [<c1070ea6>] ? tick_program_event+0x36/0x60
kernel: [47143.649461]  [<c101c4bc>] smp_apic_timer_interrupt+0x5c/0xb0
kernel: [47143.649461]  [<c12582b8>] ? trace_hardirqs_off_thunk+0xc/0x14
kernel: [47143.649461]  [<c1003e36>] apic_timer_interrupt+0x36/0x3c
kernel: [47143.649461]  [<c1257c65>] ? delay_tsc+0x45/0x80
kernel: [47143.649461]  [<c1257ba8>] __delay+0x18/0x30
kernel: [47143.649461]  [<c125d956>] _raw_spin_lock+0xf6/0x170
kernel: [47143.649461]  [<f8097e94>] ? ppp_channel_push+0x24/0xd0 [ppp_generic]
kernel: [47143.649461]  [<c141b3d2>] _spin_lock_bh+0x62/0x90
kernel: [47143.649461]  [<f8097e94>] ? ppp_channel_push+0x24/0xd0 [ppp_generic]
kernel: [47143.649461]  [<f8097e94>] ppp_channel_push+0x24/0xd0 [ppp_generic]
kernel: [47143.649461]  [<f8098057>] ppp_write+0xd7/0xe0 [ppp_generic]
kernel: [47143.649461]  [<c10fafae>] vfs_write+0xbe/0x1c0
kernel: [47143.649461]  [<f8097f80>] ? ppp_write+0x0/0xe0 [ppp_generic]
kernel: [47143.649461]  [<c10fba38>] sys_write+0x58/0xa0
kernel: [47143.649461]  [<c100339b>] sysenter_do_call+0x12/0x32
pptp[6064]: anon log[logecho:pptp_ctrl.c:677]: Echo Reply received.
pptp[6064]: anon log[logecho:pptp_ctrl.c:679]: no more Echo Reply/Request packets will be reported.
kernel: [47173.649458] INFO: RCU detected CPU 1 stall (t=14062096/21000 jiffies)
kernel: [47173.649458] Pid: 6052, comm: pppd Not tainted 2.6.31-rc1-dbgnv-git6 #1
kernel: [47173.649458] Call Trace:
kernel: [47173.649458]  [<c1417a58>] ? printk+0x23/0x36
kernel: [47173.649458]  [<c10a8220>] __rcu_pending+0x140/0x210
kernel: [47173.649458]  [<c10a831a>] rcu_pending+0x2a/0x70
kernel: [47173.649458]  [<c1051bdf>] update_process_times+0x3f/0x80
kernel: [47173.649458]  [<c10714df>] tick_sched_timer+0x6f/0xf0
kernel: [47173.649458]  [<c10640f6>] __run_hrtimer+0x56/0xe0
kernel: [47173.649458]  [<c1071470>] ? tick_sched_timer+0x0/0xf0
kernel: [47173.649458]  [<c1071470>] ? tick_sched_timer+0x0/0xf0
kernel: [47173.649458]  [<c1064af5>] hrtimer_interrupt+0x145/0x270
kernel: [47173.649458]  [<c1070ea6>] ? tick_program_event+0x36/0x60
kernel: [47173.649458]  [<c101c4bc>] smp_apic_timer_interrupt+0x5c/0xb0
kernel: [47173.649458]  [<c12582b8>] ? trace_hardirqs_off_thunk+0xc/0x14
kernel: [47173.649458]  [<c1003e36>] apic_timer_interrupt+0x36/0x3c
kernel: [47173.649458]  [<c1257c65>] ? delay_tsc+0x45/0x80
kernel: [47173.649458]  [<c1257ba8>] __delay+0x18/0x30
kernel: [47173.649458]  [<c125d956>] _raw_spin_lock+0xf6/0x170
kernel: [47173.649458]  [<f8097e94>] ? ppp_channel_push+0x24/0xd0 [ppp_generic]
kernel: [47173.649458]  [<c141b3d2>] _spin_lock_bh+0x62/0x90
kernel: [47173.649458]  [<f8097e94>] ? ppp_channel_push+0x24/0xd0 [ppp_generic]
kernel: [47173.649458]  [<f8097e94>] ppp_channel_push+0x24/0xd0 [ppp_generic]
kernel: [47173.649458]  [<f8098057>] ppp_write+0xd7/0xe0 [ppp_generic]
kernel: [47173.649458]  [<c10fafae>] vfs_write+0xbe/0x1c0
kernel: [47173.649458]  [<f8097f80>] ? ppp_write+0x0/0xe0 [ppp_generic]
kernel: [47173.649458]  [<c10fba38>] sys_write+0x58/0xa0
kernel: [47173.649458]  [<c100339b>] sysenter_do_call+0x12/0x32
kernel: [47203.649454] INFO: RCU detected CPU 1 stall (t=14071096/30000 jiffies)
kernel: [47203.649456] Pid: 6052, comm: pppd Not tainted 2.6.31-rc1-dbgnv-git6 #1
kernel: [47203.649456] Call Trace:
kernel: [47203.649456]  [<c1417a58>] ? printk+0x23/0x36
kernel: [47203.649456]  [<c10a8220>] __rcu_pending+0x140/0x210
kernel: [47203.649456]  [<c10a831a>] rcu_pending+0x2a/0x70
kernel: [47203.649456]  [<c1051bdf>] update_process_times+0x3f/0x80
kernel: [47203.649456]  [<c10714df>] tick_sched_timer+0x6f/0xf0
kernel: [47203.649456]  [<c10640f6>] __run_hrtimer+0x56/0xe0
kernel: [47203.649456]  [<c1071470>] ? tick_sched_timer+0x0/0xf0
kernel: [47203.649456]  [<c1071470>] ? tick_sched_timer+0x0/0xf0
kernel: [47203.649456]  [<c1064af5>] hrtimer_interrupt+0x145/0x270
kernel: [47203.649456]  [<c101c4bc>] smp_apic_timer_interrupt+0x5c/0xb0
kernel: [47203.649456]  [<c12582b8>] ? trace_hardirqs_off_thunk+0xc/0x14
kernel: [47203.649456]  [<c1003e36>] apic_timer_interrupt+0x36/0x3c
kernel: [47203.649456]  [<c1257c65>] ? delay_tsc+0x45/0x80
kernel: [47203.649456]  [<c1257ba8>] __delay+0x18/0x30
kernel: [47203.649456]  [<c125d956>] _raw_spin_lock+0xf6/0x170
kernel: [47203.649456]  [<f8097e94>] ? ppp_channel_push+0x24/0xd0 [ppp_generic]
kernel: [47203.649456]  [<c141b3d2>] _spin_lock_bh+0x62/0x90
kernel: [47203.649456]  [<f8097e94>] ? ppp_channel_push+0x24/0xd0 [ppp_generic]
kernel: [47203.649456]  [<f8097e94>] ppp_channel_push+0x24/0xd0 [ppp_generic]
kernel: [47203.649456]  [<f8098057>] ppp_write+0xd7/0xe0 [ppp_generic]
kernel: [47203.649456]  [<c10fafae>] vfs_write+0xbe/0x1c0
kernel: [47203.649456]  [<f8097f80>] ? ppp_write+0x0/0xe0 [ppp_generic]
kernel: [47203.649456]  [<c10fba38>] sys_write+0x58/0xa0
kernel: [47203.649456]  [<c100339b>] sysenter_do_call+0x12/0x32
kernel: [47233.649451] INFO: RCU detected CPU 1 stall (t=14080096/39000 jiffies)
kernel: [47233.649452] Pid: 6052, comm: pppd Not tainted 2.6.31-rc1-dbgnv-git6 #1
kernel: [47233.649452] Call Trace:
kernel: [47233.649452]  [<c1417a58>] ? printk+0x23/0x36
kernel: [47233.649452]  [<c10a8220>] __rcu_pending+0x140/0x210
kernel: [47233.649452]  [<c10a831a>] rcu_pending+0x2a/0x70
kernel: [47233.649452]  [<c1051bdf>] update_process_times+0x3f/0x80
kernel: [47233.649452]  [<c10714df>] tick_sched_timer+0x6f/0xf0
kernel: [47233.649452]  [<c10640f6>] __run_hrtimer+0x56/0xe0
kernel: [47233.649452]  [<c1071470>] ? tick_sched_timer+0x0/0xf0
kernel: [47233.649452]  [<c1071470>] ? tick_sched_timer+0x0/0xf0
kernel: [47233.649452]  [<c1064af5>] hrtimer_interrupt+0x145/0x270
kernel: [47233.649452]  [<c1070ea6>] ? tick_program_event+0x36/0x60
kernel: [47233.649452]  [<c101c4bc>] smp_apic_timer_interrupt+0x5c/0xb0
kernel: [47233.649452]  [<c12582b8>] ? trace_hardirqs_off_thunk+0xc/0x14
kernel: [47233.649452]  [<c1003e36>] apic_timer_interrupt+0x36/0x3c
kernel: [47233.649452]  [<c1257c63>] ? delay_tsc+0x43/0x80
kernel: [47233.649452]  [<c1257ba8>] __delay+0x18/0x30
kernel: [47233.649452]  [<c125d956>] _raw_spin_lock+0xf6/0x170
kernel: [47233.649452]  [<f8097e94>] ? ppp_channel_push+0x24/0xd0 [ppp_generic]
kernel: [47233.649452]  [<c141b3d2>] _spin_lock_bh+0x62/0x90
kernel: [47233.649452]  [<f8097e94>] ? ppp_channel_push+0x24/0xd0 [ppp_generic]
kernel: [47233.649452]  [<f8097e94>] ppp_channel_push+0x24/0xd0 [ppp_generic]
kernel: [47233.649452]  [<f8098057>] ppp_write+0xd7/0xe0 [ppp_generic]
kernel: [47233.649452]  [<c10fafae>] vfs_write+0xbe/0x1c0
kernel: [47233.649452]  [<f8097f80>] ? ppp_write+0x0/0xe0 [ppp_generic]
kernel: [47233.649452]  [<c10fba38>] sys_write+0x58/0xa0
kernel: [47233.649452]  [<c100339b>] sysenter_do_call+0x12/0x32
kernel: [47263.649448] INFO: RCU detected CPU 1 stall (t=14089096/48000 jiffies)
kernel: [47263.649449] Pid: 6052, comm: pppd Not tainted 2.6.31-rc1-dbgnv-git6 #1
kernel: [47263.649449] Call Trace:
kernel: [47263.649449]  [<c1417a58>] ? printk+0x23/0x36
kernel: [47263.649449]  [<c10a8220>] __rcu_pending+0x140/0x210
kernel: [47263.649449]  [<c10a831a>] rcu_pending+0x2a/0x70
kernel: [47263.649449]  [<c1051bdf>] update_process_times+0x3f/0x80
kernel: [47263.649449]  [<c10714df>] tick_sched_timer+0x6f/0xf0
kernel: [47263.649449]  [<c10640f6>] __run_hrtimer+0x56/0xe0
kernel: [47263.649449]  [<c1071470>] ? tick_sched_timer+0x0/0xf0
kernel: [47263.649449]  [<c1071470>] ? tick_sched_timer+0x0/0xf0
kernel: [47263.649449]  [<c1064af5>] hrtimer_interrupt+0x145/0x270
kernel: [47263.649449]  [<c1070ea6>] ? tick_program_event+0x36/0x60
kernel: [47263.649449]  [<c101c4bc>] smp_apic_timer_interrupt+0x5c/0xb0
kernel: [47263.649449]  [<c12582b8>] ? trace_hardirqs_off_thunk+0xc/0x14
kernel: [47263.649449]  [<c1003e36>] apic_timer_interrupt+0x36/0x3c
kernel: [47263.649449]  [<c1257c65>] ? delay_tsc+0x45/0x80
kernel: [47263.649449]  [<c1257ba8>] __delay+0x18/0x30
kernel: [47263.649449]  [<c125d956>] _raw_spin_lock+0xf6/0x170
kernel: [47263.649449]  [<f8097e94>] ? ppp_channel_push+0x24/0xd0 [ppp_generic]
kernel: [47263.649449]  [<c141b3d2>] _spin_lock_bh+0x62/0x90
kernel: [47263.649449]  [<f8097e94>] ? ppp_channel_push+0x24/0xd0 [ppp_generic]
kernel: [47263.649449]  [<f8097e94>] ppp_channel_push+0x24/0xd0 [ppp_generic]
kernel: [47263.649449]  [<f8098057>] ppp_write+0xd7/0xe0 [ppp_generic]
kernel: [47263.649449]  [<c10fafae>] vfs_write+0xbe/0x1c0
kernel: [47263.649449]  [<f8097f80>] ? ppp_write+0x0/0xe0 [ppp_generic]
kernel: [47263.649449]  [<c10fba38>] sys_write+0x58/0xa0
kernel: [47263.649449]  [<c100339b>] sysenter_do_call+0x12/0x32
kernel: [47293.649445] INFO: RCU detected CPU 1 stall (t=14098096/57000 jiffies)
kernel: [47293.649446] Pid: 6052, comm: pppd Not tainted 2.6.31-rc1-dbgnv-git6 #1
kernel: [47293.649446] Call Trace:
kernel: [47293.649446]  [<c1417a58>] ? printk+0x23/0x36
kernel: [47293.649446]  [<c10a8220>] __rcu_pending+0x140/0x210
kernel: [47293.649446]  [<c10a831a>] rcu_pending+0x2a/0x70
kernel: [47293.649446]  [<c1051bdf>] update_process_times+0x3f/0x80
kernel: [47293.649446]  [<c10714df>] tick_sched_timer+0x6f/0xf0
kernel: [47293.649446]  [<c10640f6>] __run_hrtimer+0x56/0xe0
kernel: [47293.649446]  [<c1071470>] ? tick_sched_timer+0x0/0xf0
kernel: [47293.649446]  [<c1071470>] ? tick_sched_timer+0x0/0xf0
kernel: [47293.649446]  [<c1064af5>] hrtimer_interrupt+0x145/0x270
kernel: [47293.649446]  [<c1070ea6>] ? tick_program_event+0x36/0x60
kernel: [47293.649446]  [<c101c4bc>] smp_apic_timer_interrupt+0x5c/0xb0
kernel: [47293.649446]  [<c12582b8>] ? trace_hardirqs_off_thunk+0xc/0x14
kernel: [47293.649446]  [<c1003e36>] apic_timer_interrupt+0x36/0x3c
kernel: [47293.649446]  [<c1257c48>] ? delay_tsc+0x28/0x80
kernel: [47293.649446]  [<c1257ba8>] __delay+0x18/0x30
kernel: [47293.649446]  [<c125d956>] _raw_spin_lock+0xf6/0x170
kernel: [47293.649446]  [<f8097e94>] ? ppp_channel_push+0x24/0xd0 [ppp_generic]
kernel: [47293.649446]  [<c141b3d2>] _spin_lock_bh+0x62/0x90
kernel: [47293.649446]  [<f8097e94>] ? ppp_channel_push+0x24/0xd0 [ppp_generic]
kernel: [47293.649446]  [<f8097e94>] ppp_channel_push+0x24/0xd0 [ppp_generic]
kernel: [47293.649446]  [<f8098057>] ppp_write+0xd7/0xe0 [ppp_generic]
kernel: [47293.649446]  [<c10fafae>] vfs_write+0xbe/0x1c0
kernel: [47293.649446]  [<f8097f80>] ? ppp_write+0x0/0xe0 [ppp_generic]
kernel: [47293.649446]  [<c10fba38>] sys_write+0x58/0xa0
kernel: [47293.649446]  [<c100339b>] sysenter_do_call+0x12/0x32


	Sergey

Comment 21 Sergey Senozhatsky 2009-07-07 16:26:50 UTC

Hello.

kernel: [ 1048.295158] BUG: scheduling while atomic: ksoftirqd/0/4/0x00000506
kernel: [ 1048.295169] INFO: lockdep is turned off.
kernel: [ 1048.295176] Modules linked in: ppp_deflate zlib_deflate ppp_async crc_ccitt ppp_generic slhc ipv6 fuse loop snd_hda_codec_si3054 snd_hda_codec_realtek snd_hda_intel
snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device snd psmouse i2c_i801 pcspkr serio_raw rng_core soundcore snd_page_alloc
asus_laptop evdev usbhid hid sg sr_mod cdrom sd_mod ata_generic pata_acpi uhci_hcd ricoh_mmc sdhci_pci sdhci ata_piix ide_pci_generic mmc_core led_class ehci_hcd usbcore r8169 mii
kernel: [ 1048.295334] Pid: 4, comm: ksoftirqd/0 Not tainted 2.6.31-rc2-pnv-git1-00254-ga4e2f0a-dirty #2
kernel: [ 1048.295342] Call Trace:
kernel: [ 1048.295361]  [<c1035f55>] __schedule_bug+0x95/0xa0
kernel: [ 1048.295375]  [<c14394f2>] schedule+0x962/0xb60
kernel: [ 1048.295388]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1048.295403]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1048.295416]  [<c143a3df>] __mutex_lock_common+0x17f/0x450
kernel: [ 1048.295428]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1048.295438]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1048.295451]  [<c143a754>] mutex_lock_nested+0x24/0x40
kernel: [ 1048.295461]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1048.295472]  [<c1300355>] tty_throttle+0x25/0x70
kernel: [ 1048.295483]  [<c12fecd7>] n_tty_receive_buf+0x3a7/0x11c0
kernel: [ 1048.295495]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1048.295507]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1048.295519]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1048.295531]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1048.295550]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1048.295561]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1048.295573]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1048.295585]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
kernel: [ 1048.295596]  [<c13012d2>] ? tty_ldisc_try+0x42/0x60
kernel: [ 1048.295608]  [<c13038dd>] pty_write+0x7d/0x90
kernel: [ 1048.295624]  [<f80b3c8a>] ppp_async_push+0x3da/0x4d0 [ppp_async]
kernel: [ 1048.295639]  [<f80b3dc8>] ppp_async_send+0x48/0x70 [ppp_async]
kernel: [ 1048.295655]  [<f8097086>] ppp_push+0x176/0x5f0 [ppp_generic]
kernel: [ 1048.295670]  [<f8097816>] ? ppp_xmit_process+0x26/0x570 [ppp_generic]
kernel: [ 1048.295685]  [<f8097816>] ? ppp_xmit_process+0x26/0x570 [ppp_generic]
kernel: [ 1048.295702]  [<f809784a>] ppp_xmit_process+0x5a/0x570 [ppp_generic]
kernel: [ 1048.295717]  [<f8097fa5>] ? ppp_channel_push+0x95/0xd0 [ppp_generic]
kernel: [ 1048.295733]  [<f8097fb4>] ppp_channel_push+0xa4/0xd0 [ppp_generic]
kernel: [ 1048.295748]  [<f8097ffe>] ppp_output_wakeup+0x1e/0x40 [ppp_generic]
kernel: [ 1048.295760]  [<f80b43f0>] ppp_async_process+0x90/0xa0 [ppp_async]
kernel: [ 1048.295773]  [<c104ee15>] tasklet_action+0xf5/0x110
kernel: [ 1048.295784]  [<c104f9ea>] __do_softirq+0xba/0x240
kernel: [ 1048.295795]  [<c104fbe5>] do_softirq+0x75/0x90
kernel: [ 1048.295807]  [<c10502e2>] ksoftirqd+0x82/0x1a0
kernel: [ 1048.295817]  [<c1050260>] ? ksoftirqd+0x0/0x1a0
kernel: [ 1048.295831]  [<c10646e4>] kthread+0x84/0x90
kernel: [ 1048.295843]  [<c1064660>] ? kthread+0x0/0x90
kernel: [ 1048.295855]  [<c1004217>] kernel_thread_helper+0x7/0x70
kernel: [ 1048.296425] BUG: scheduling while atomic: ksoftirqd/0/4/0x00000406
kernel: [ 1048.296431] INFO: lockdep is turned off.
kernel: [ 1048.296437] Modules linked in: ppp_deflate zlib_deflate ppp_async crc_ccitt ppp_generic slhc ipv6 fuse loop snd_hda_codec_si3054 snd_hda_codec_realtek snd_hda_intel
snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device snd psmouse i2c_i801 pcspkr serio_raw rng_core soundcore snd_page_alloc
asus_laptop evdev usbhid hid sg sr_mod cdrom sd_mod ata_generic pata_acpi uhci_hcd ricoh_mmc sdhci_pci sdhci ata_piix ide_pci_generic mmc_core led_class ehci_hcd usbcore r8169 mii
kernel: [ 1048.296648] Pid: 4, comm: ksoftirqd/0 Not tainted 2.6.31-rc2-pnv-git1-00254-ga4e2f0a-dirty #2
kernel: [ 1048.296661] Call Trace:
kernel: [ 1048.296685]  [<c1035f55>] __schedule_bug+0x95/0xa0
kernel: [ 1048.296706]  [<c14394f2>] schedule+0x962/0xb60
kernel: [ 1048.296723]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1048.296741]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1048.296759]  [<c143a3df>] __mutex_lock_common+0x17f/0x450
kernel: [ 1048.296779]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1048.296794]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1048.296811]  [<c143a754>] mutex_lock_nested+0x24/0x40
kernel: [ 1048.296828]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1048.296842]  [<c1300355>] tty_throttle+0x25/0x70
kernel: [ 1048.296858]  [<c12fecd7>] n_tty_receive_buf+0x3a7/0x11c0
kernel: [ 1048.296872]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
kernel: [ 1048.296885]  [<c1033273>] ? __wake_up+0x53/0x70
kernel: [ 1048.296899]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1048.296916]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1048.296937]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1048.296955]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1048.296973]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1048.296989]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1048.297007]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1048.297023]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
kernel: [ 1048.297041]  [<c13012d2>] ? tty_ldisc_try+0x42/0x60
kernel: [ 1048.297057]  [<c13038dd>] pty_write+0x7d/0x90
kernel: [ 1048.297074]  [<f80b3c8a>] ppp_async_push+0x3da/0x4d0 [ppp_async]
kernel: [ 1048.297096]  [<f80b3dc8>] ppp_async_send+0x48/0x70 [ppp_async]
kernel: [ 1048.297113]  [<f8097086>] ppp_push+0x176/0x5f0 [ppp_generic]
kernel: [ 1048.297135]  [<c1397436>] ? skb_dequeue+0x26/0x80
kernel: [ 1048.297150]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1048.297168]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
kernel: [ 1048.297185]  [<c1397463>] ? skb_dequeue+0x53/0x80
kernel: [ 1048.297204]  [<f8097bcd>] ppp_xmit_process+0x3dd/0x570 [ppp_generic]
kernel: [ 1048.297224]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1048.297238]  [<c1397311>] ? skb_queue_tail+0x41/0x60
kernel: [ 1048.297259]  [<f8097ec7>] ppp_start_xmit+0x167/0x1b0 [ppp_generic]
kernel: [ 1048.297274]  [<c13b8522>] ? __qdisc_run+0x192/0x260
kernel: [ 1048.297290]  [<c13a466c>] dev_hard_start_xmit+0x21c/0x3c0
kernel: [ 1048.297305]  [<c143bdae>] ? _spin_lock+0x6e/0x90
kernel: [ 1048.297321]  [<c13b853c>] __qdisc_run+0x1ac/0x260
kernel: [ 1048.297342]  [<c13a38e7>] net_tx_action+0xf7/0x170
kernel: [ 1048.297356]  [<c104f9ea>] __do_softirq+0xba/0x240
kernel: [ 1048.297375]  [<c104fbe5>] do_softirq+0x75/0x90
kernel: [ 1048.297389]  [<c10502e2>] ksoftirqd+0x82/0x1a0
kernel: [ 1048.297401]  [<c1050260>] ? ksoftirqd+0x0/0x1a0
kernel: [ 1048.297413]  [<c10646e4>] kthread+0x84/0x90
kernel: [ 1048.297433]  [<c1064660>] ? kthread+0x0/0x90
kernel: [ 1048.297453]  [<c1004217>] kernel_thread_helper+0x7/0x70
kernel: [ 1048.298100] BUG: scheduling while atomic: ksoftirqd/0/4/0x00000406
kernel: [ 1048.298115] INFO: lockdep is turned off.
kernel: [ 1048.298127] Modules linked in: ppp_deflate zlib_deflate ppp_async crc_ccitt ppp_generic slhc ipv6 fuse loop snd_hda_codec_si3054 snd_hda_codec_realtek snd_hda_intel
snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device snd psmouse i2c_i801 pcspkr serio_raw rng_core soundcore snd_page_alloc
asus_laptop evdev usbhid hid sg sr_mod cdrom sd_mod ata_generic pata_acpi uhci_hcd ricoh_mmc sdhci_pci sdhci ata_piix ide_pci_generic mmc_core led_class ehci_hcd usbcore r8169 mii
kernel: [ 1048.298482] Pid: 4, comm: ksoftirqd/0 Not tainted 2.6.31-rc2-pnv-git1-00254-ga4e2f0a-dirty #2
kernel: [ 1048.298490] Call Trace:
kernel: [ 1048.298507]  [<c1035f55>] __schedule_bug+0x95/0xa0
kernel: [ 1048.298527]  [<c14394f2>] schedule+0x962/0xb60
kernel: [ 1048.298544]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1048.298562]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1048.298580]  [<c143a3df>] __mutex_lock_common+0x17f/0x450
kernel: [ 1048.298594]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1048.298608]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1048.298626]  [<c143a754>] mutex_lock_nested+0x24/0x40
kernel: [ 1048.298641]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1048.298662]  [<c1300355>] tty_throttle+0x25/0x70
kernel: [ 1048.298683]  [<c12fecd7>] n_tty_receive_buf+0x3a7/0x11c0
kernel: [ 1048.298698]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
kernel: [ 1048.298716]  [<c1033273>] ? __wake_up+0x53/0x70
kernel: [ 1048.298732]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1048.298750]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1048.298764]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1048.298780]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1048.298799]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1048.298817]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1048.298834]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1048.298849]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
kernel: [ 1048.298866]  [<c13012d2>] ? tty_ldisc_try+0x42/0x60
kernel: [ 1048.298882]  [<c13038dd>] pty_write+0x7d/0x90
kernel: [ 1048.298897]  [<f80b3c8a>] ppp_async_push+0x3da/0x4d0 [ppp_async]
kernel: [ 1048.298916]  [<f80b3dc8>] ppp_async_send+0x48/0x70 [ppp_async]
kernel: [ 1048.298944]  [<f8097086>] ppp_push+0x176/0x5f0 [ppp_generic]
kernel: [ 1048.298963]  [<c1397436>] ? skb_dequeue+0x26/0x80
kernel: [ 1048.298980]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1048.299000]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
kernel: [ 1048.299013]  [<c1397463>] ? skb_dequeue+0x53/0x80
kernel: [ 1048.299038]  [<f8097bcd>] ppp_xmit_process+0x3dd/0x570 [ppp_generic]
kernel: [ 1048.299054]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1048.299074]  [<c1397311>] ? skb_queue_tail+0x41/0x60
kernel: [ 1048.299092]  [<f8097ec7>] ppp_start_xmit+0x167/0x1b0 [ppp_generic]
kernel: [ 1048.299110]  [<c13b8522>] ? __qdisc_run+0x192/0x260
kernel: [ 1048.299130]  [<c13a466c>] dev_hard_start_xmit+0x21c/0x3c0
kernel: [ 1048.299145]  [<c143bdae>] ? _spin_lock+0x6e/0x90
kernel: [ 1048.299168]  [<c13b853c>] __qdisc_run+0x1ac/0x260
kernel: [ 1048.299186]  [<c13a38e7>] net_tx_action+0xf7/0x170
kernel: [ 1048.299208]  [<c104f9ea>] __do_softirq+0xba/0x240
kernel: [ 1048.299226]  [<c104fbe5>] do_softirq+0x75/0x90
kernel: [ 1048.299240]  [<c10502e2>] ksoftirqd+0x82/0x1a0
kernel: [ 1048.299253]  [<c1050260>] ? ksoftirqd+0x0/0x1a0
kernel: [ 1048.299268]  [<c10646e4>] kthread+0x84/0x90
kernel: [ 1048.299288]  [<c1064660>] ? kthread+0x0/0x90
kernel: [ 1048.299301]  [<c1004217>] kernel_thread_helper+0x7/0x70
kernel: [ 1067.299845] BUG: scheduling while atomic: ksoftirqd/0/4/0x00000506
kernel: [ 1067.299854] INFO: lockdep is turned off.
kernel: [ 1067.299860] Modules linked in: ppp_deflate zlib_deflate ppp_async crc_ccitt ppp_generic slhc ipv6 fuse loop snd_hda_codec_si3054 snd_hda_codec_realtek snd_hda_intel
snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device snd psmouse i2c_i801 pcspkr serio_raw rng_core soundcore snd_page_alloc
asus_laptop evdev usbhid hid sg sr_mod cdrom sd_mod ata_generic pata_acpi uhci_hcd ricoh_mmc sdhci_pci sdhci ata_piix ide_pci_generic mmc_core led_class ehci_hcd usbcore r8169 mii
kernel: [ 1067.300216] Pid: 4, comm: ksoftirqd/0 Not tainted 2.6.31-rc2-pnv-git1-00254-ga4e2f0a-dirty #2
kernel: [ 1067.300229] Call Trace:
kernel: [ 1067.300249]  [<c1035f55>] __schedule_bug+0x95/0xa0
kernel: [ 1067.300270]  [<c14394f2>] schedule+0x962/0xb60
kernel: [ 1067.300289]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1067.300306]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1067.300324]  [<c143a3df>] __mutex_lock_common+0x17f/0x450
kernel: [ 1067.300347]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1067.300365]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1067.300384]  [<c143a754>] mutex_lock_nested+0x24/0x40
kernel: [ 1067.300400]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1067.300418]  [<c1300355>] tty_throttle+0x25/0x70
kernel: [ 1067.300438]  [<c12fecd7>] n_tty_receive_buf+0x3a7/0x11c0
kernel: [ 1067.300452]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1067.300468]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1067.300485]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1067.300502]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1067.300518]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1067.300535]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1067.300554]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1067.300569]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
kernel: [ 1067.300585]  [<c13012d2>] ? tty_ldisc_try+0x42/0x60
kernel: [ 1067.300600]  [<c13038dd>] pty_write+0x7d/0x90
kernel: [ 1067.300621]  [<f80b3c8a>] ppp_async_push+0x3da/0x4d0 [ppp_async]
kernel: [ 1067.300640]  [<f80b3dc8>] ppp_async_send+0x48/0x70 [ppp_async]
kernel: [ 1067.300665]  [<f8097086>] ppp_push+0x176/0x5f0 [ppp_generic]
kernel: [ 1067.300687]  [<f8097816>] ? ppp_xmit_process+0x26/0x570 [ppp_generic]
kernel: [ 1067.300709]  [<f8097816>] ? ppp_xmit_process+0x26/0x570 [ppp_generic]
kernel: [ 1067.300731]  [<f809784a>] ppp_xmit_process+0x5a/0x570 [ppp_generic]
kernel: [ 1067.300746]  [<f8097fa5>] ? ppp_channel_push+0x95/0xd0 [ppp_generic]
kernel: [ 1067.300763]  [<f8097fb4>] ppp_channel_push+0xa4/0xd0 [ppp_generic]
kernel: [ 1067.300781]  [<f8097ffe>] ppp_output_wakeup+0x1e/0x40 [ppp_generic]
kernel: [ 1067.300804]  [<f80b43f0>] ppp_async_process+0x90/0xa0 [ppp_async]
kernel: [ 1067.300822]  [<c104ee15>] tasklet_action+0xf5/0x110
kernel: [ 1067.300841]  [<c104f9ea>] __do_softirq+0xba/0x240
kernel: [ 1067.300855]  [<c104fbe5>] do_softirq+0x75/0x90
kernel: [ 1067.300873]  [<c10502e2>] ksoftirqd+0x82/0x1a0
kernel: [ 1067.300886]  [<c1050260>] ? ksoftirqd+0x0/0x1a0
kernel: [ 1067.300901]  [<c10646e4>] kthread+0x84/0x90
kernel: [ 1067.300918]  [<c1064660>] ? kthread+0x0/0x90
kernel: [ 1067.300937]  [<c1004217>] kernel_thread_helper+0x7/0x70
kernel: [ 1067.301595] BUG: scheduling while atomic: ksoftirqd/0/4/0x00000406
kernel: [ 1067.301608] INFO: lockdep is turned off.
kernel: [ 1067.301617] Modules linked in: ppp_deflate zlib_deflate ppp_async crc_ccitt ppp_generic slhc ipv6 fuse loop snd_hda_codec_si3054 snd_hda_codec_realtek snd_hda_intel
snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device snd psmouse i2c_i801 pcspkr serio_raw rng_core soundcore snd_page_alloc
asus_laptop evdev usbhid hid sg sr_mod cdrom sd_mod ata_generic pata_acpi uhci_hcd ricoh_mmc sdhci_pci sdhci ata_piix ide_pci_generic mmc_core led_class ehci_hcd usbcore r8169 mii
kernel: [ 1067.301969] Pid: 4, comm: ksoftirqd/0 Not tainted 2.6.31-rc2-pnv-git1-00254-ga4e2f0a-dirty #2
kernel: [ 1067.301981] Call Trace:
kernel: [ 1067.301998]  [<c1035f55>] __schedule_bug+0x95/0xa0
kernel: [ 1067.302018]  [<c14394f2>] schedule+0x962/0xb60
kernel: [ 1067.302031]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1067.302052]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1067.302072]  [<c143a3df>] __mutex_lock_common+0x17f/0x450
kernel: [ 1067.302092]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1067.302110]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1067.302126]  [<c143a754>] mutex_lock_nested+0x24/0x40
kernel: [ 1067.302139]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1067.302152]  [<c1300355>] tty_throttle+0x25/0x70
kernel: [ 1067.302167]  [<c12fecd7>] n_tty_receive_buf+0x3a7/0x11c0
kernel: [ 1067.302188]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
kernel: [ 1067.302203]  [<c1033273>] ? __wake_up+0x53/0x70
kernel: [ 1067.302219]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1067.302234]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1067.302251]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1067.302268]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1067.302284]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1067.302301]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1067.302319]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1067.302339]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
kernel: [ 1067.302352]  [<c13012d2>] ? tty_ldisc_try+0x42/0x60
kernel: [ 1067.302370]  [<c13038dd>] pty_write+0x7d/0x90
kernel: [ 1067.302389]  [<f80b3c8a>] ppp_async_push+0x3da/0x4d0 [ppp_async]
kernel: [ 1067.302412]  [<f80b3dc8>] ppp_async_send+0x48/0x70 [ppp_async]
kernel: [ 1067.302437]  [<f8097086>] ppp_push+0x176/0x5f0 [ppp_generic]
kernel: [ 1067.302454]  [<c1397436>] ? skb_dequeue+0x26/0x80
kernel: [ 1067.302473]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1067.302490]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
kernel: [ 1067.302503]  [<c1397463>] ? skb_dequeue+0x53/0x80
kernel: [ 1067.302526]  [<f8097bcd>] ppp_xmit_process+0x3dd/0x570 [ppp_generic]
kernel: [ 1067.302541]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1067.302558]  [<c1397311>] ? skb_queue_tail+0x41/0x60
kernel: [ 1067.302577]  [<f8097ec7>] ppp_start_xmit+0x167/0x1b0 [ppp_generic]
kernel: [ 1067.302601]  [<c13b8522>] ? __qdisc_run+0x192/0x260
kernel: [ 1067.302622]  [<c13a466c>] dev_hard_start_xmit+0x21c/0x3c0
kernel: [ 1067.302640]  [<c143bdae>] ? _spin_lock+0x6e/0x90
kernel: [ 1067.302654]  [<c13b853c>] __qdisc_run+0x1ac/0x260
kernel: [ 1067.302675]  [<c13a38e7>] net_tx_action+0xf7/0x170
kernel: [ 1067.302698]  [<c104f9ea>] __do_softirq+0xba/0x240
kernel: [ 1067.302713]  [<c104fbe5>] do_softirq+0x75/0x90
kernel: [ 1067.302728]  [<c10502e2>] ksoftirqd+0x82/0x1a0
kernel: [ 1067.302745]  [<c1050260>] ? ksoftirqd+0x0/0x1a0
kernel: [ 1067.302761]  [<c10646e4>] kthread+0x84/0x90
kernel: [ 1067.302783]  [<c1064660>] ? kthread+0x0/0x90
kernel: [ 1067.302798]  [<c1004217>] kernel_thread_helper+0x7/0x70
kernel: [ 1076.490281] BUG: scheduling while atomic: ksoftirqd/0/4/0x00000406
kernel: [ 1076.490290] INFO: lockdep is turned off.
kernel: [ 1076.490296] Modules linked in: ppp_deflate zlib_deflate ppp_async crc_ccitt ppp_generic slhc ipv6 fuse loop snd_hda_codec_si3054 snd_hda_codec_realtek snd_hda_intel
snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device snd psmouse i2c_i801 pcspkr serio_raw rng_core soundcore snd_page_alloc
asus_laptop evdev usbhid hid sg sr_mod cdrom sd_mod ata_generic pata_acpi uhci_hcd ricoh_mmc sdhci_pci sdhci ata_piix ide_pci_generic mmc_core led_class ehci_hcd usbcore r8169 mii
kernel: [ 1076.490452] Pid: 4, comm: ksoftirqd/0 Not tainted 2.6.31-rc2-pnv-git1-00254-ga4e2f0a-dirty #2
kernel: [ 1076.490460] Call Trace:
kernel: [ 1076.490478]  [<c1035f55>] __schedule_bug+0x95/0xa0
kernel: [ 1076.490492]  [<c14394f2>] schedule+0x962/0xb60
kernel: [ 1076.490506]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1076.490521]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1076.490534]  [<c143a3df>] __mutex_lock_common+0x17f/0x450
kernel: [ 1076.490545]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1076.490556]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1076.490568]  [<c143a754>] mutex_lock_nested+0x24/0x40
kernel: [ 1076.490579]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1076.490590]  [<c1300355>] tty_throttle+0x25/0x70
kernel: [ 1076.490601]  [<c12fecd7>] n_tty_receive_buf+0x3a7/0x11c0
kernel: [ 1076.490613]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
kernel: [ 1076.490625]  [<c1033273>] ? __wake_up+0x53/0x70
kernel: [ 1076.490636]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1076.490648]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1076.490659]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1076.490672]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1076.490682]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1076.490693]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1076.490705]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1076.490717]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
kernel: [ 1076.490728]  [<c13012d2>] ? tty_ldisc_try+0x42/0x60
kernel: [ 1076.490741]  [<c13038dd>] pty_write+0x7d/0x90
kernel: [ 1076.490757]  [<f80b3c8a>] ppp_async_push+0x3da/0x4d0 [ppp_async]
kernel: [ 1076.490774]  [<f80b3dc8>] ppp_async_send+0x48/0x70 [ppp_async]
kernel: [ 1076.490790]  [<f8097086>] ppp_push+0x176/0x5f0 [ppp_generic]
kernel: [ 1076.490802]  [<c1397436>] ? skb_dequeue+0x26/0x80
kernel: [ 1076.490815]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1076.490835]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
kernel: [ 1076.490846]  [<c1397463>] ? skb_dequeue+0x53/0x80
kernel: [ 1076.490861]  [<f8097bcd>] ppp_xmit_process+0x3dd/0x570 [ppp_generic]
kernel: [ 1076.490874]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1076.490885]  [<c1397311>] ? skb_queue_tail+0x41/0x60
kernel: [ 1076.490900]  [<f8097ec7>] ppp_start_xmit+0x167/0x1b0 [ppp_generic]
kernel: [ 1076.490912]  [<c13b8522>] ? __qdisc_run+0x192/0x260
kernel: [ 1076.490925]  [<c13a466c>] dev_hard_start_xmit+0x21c/0x3c0
kernel: [ 1076.490937]  [<c143bdae>] ? _spin_lock+0x6e/0x90
kernel: [ 1076.490948]  [<c13b853c>] __qdisc_run+0x1ac/0x260
kernel: [ 1076.490960]  [<c13a38e7>] net_tx_action+0xf7/0x170
kernel: [ 1076.490972]  [<c104f9ea>] __do_softirq+0xba/0x240
kernel: [ 1076.490984]  [<c104fbe5>] do_softirq+0x75/0x90
kernel: [ 1076.490995]  [<c10502e2>] ksoftirqd+0x82/0x1a0
kernel: [ 1076.491005]  [<c1050260>] ? ksoftirqd+0x0/0x1a0
kernel: [ 1076.491017]  [<c10646e4>] kthread+0x84/0x90
kernel: [ 1076.491029]  [<c1064660>] ? kthread+0x0/0x90
kernel: [ 1076.491041]  [<c1004217>] kernel_thread_helper+0x7/0x70
kernel: [ 1076.491622] BUG: scheduling while atomic: ksoftirqd/0/4/0x00000406
kernel: [ 1076.491629] INFO: lockdep is turned off.
kernel: [ 1076.491635] Modules linked in: ppp_deflate zlib_deflate ppp_async crc_ccitt ppp_generic slhc ipv6 fuse loop snd_hda_codec_si3054 snd_hda_codec_realtek snd_hda_intel
snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device snd psmouse i2c_i801 pcspkr serio_raw rng_core soundcore snd_page_alloc
asus_laptop evdev usbhid hid sg sr_mod cdrom sd_mod ata_generic pata_acpi uhci_hcd ricoh_mmc sdhci_pci sdhci ata_piix ide_pci_generic mmc_core led_class ehci_hcd usbcore r8169 mii
kernel: [ 1076.491786] Pid: 4, comm: ksoftirqd/0 Not tainted 2.6.31-rc2-pnv-git1-00254-ga4e2f0a-dirty #2
kernel: [ 1076.491794] Call Trace:
kernel: [ 1076.491807]  [<c1035f55>] __schedule_bug+0x95/0xa0
kernel: [ 1076.491818]  [<c14394f2>] schedule+0x962/0xb60
kernel: [ 1076.491829]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1076.491842]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1076.491856]  [<c143a3df>] __mutex_lock_common+0x17f/0x450
kernel: [ 1076.491867]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1076.491877]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1076.491889]  [<c143a754>] mutex_lock_nested+0x24/0x40
kernel: [ 1076.491900]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1076.491910]  [<c1300355>] tty_throttle+0x25/0x70
kernel: [ 1076.491921]  [<c12fecd7>] n_tty_receive_buf+0x3a7/0x11c0
kernel: [ 1076.491933]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
kernel: [ 1076.491945]  [<c1033273>] ? __wake_up+0x53/0x70
kernel: [ 1076.491955]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1076.491967]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1076.491979]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1076.491991]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1076.492001]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1076.492012]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1076.492024]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1076.492035]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
kernel: [ 1076.492047]  [<c13012d2>] ? tty_ldisc_try+0x42/0x60
kernel: [ 1076.492058]  [<c13038dd>] pty_write+0x7d/0x90
kernel: [ 1076.492072]  [<f80b3c8a>] ppp_async_push+0x3da/0x4d0 [ppp_async]
kernel: [ 1076.492088]  [<f80b3dc8>] ppp_async_send+0x48/0x70 [ppp_async]
kernel: [ 1076.492103]  [<f8097086>] ppp_push+0x176/0x5f0 [ppp_generic]
kernel: [ 1076.492114]  [<c1397436>] ? skb_dequeue+0x26/0x80
kernel: [ 1076.492126]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1076.492138]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
kernel: [ 1076.492149]  [<c1397463>] ? skb_dequeue+0x53/0x80
kernel: [ 1076.492164]  [<f8097bcd>] ppp_xmit_process+0x3dd/0x570 [ppp_generic]
kernel: [ 1076.492176]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1076.492187]  [<c1397311>] ? skb_queue_tail+0x41/0x60
kernel: [ 1076.492202]  [<f8097ec7>] ppp_start_xmit+0x167/0x1b0 [ppp_generic]
kernel: [ 1076.492213]  [<c13b8522>] ? __qdisc_run+0x192/0x260
kernel: [ 1076.492225]  [<c13a466c>] dev_hard_start_xmit+0x21c/0x3c0
kernel: [ 1076.492236]  [<c143bdae>] ? _spin_lock+0x6e/0x90
kernel: [ 1076.492248]  [<c13b853c>] __qdisc_run+0x1ac/0x260
kernel: [ 1076.492259]  [<c13a38e7>] net_tx_action+0xf7/0x170
kernel: [ 1076.492270]  [<c104f9ea>] __do_softirq+0xba/0x240
kernel: [ 1076.492282]  [<c104fbe5>] do_softirq+0x75/0x90
kernel: [ 1076.492292]  [<c10502e2>] ksoftirqd+0x82/0x1a0
kernel: [ 1076.492302]  [<c1050260>] ? ksoftirqd+0x0/0x1a0
kernel: [ 1076.492313]  [<c10646e4>] kthread+0x84/0x90
kernel: [ 1076.492325]  [<c1064660>] ? kthread+0x0/0x90
kernel: [ 1076.492336]  [<c1004217>] kernel_thread_helper+0x7/0x70
kernel: [ 1117.962740] BUG: scheduling while atomic: ksoftirqd/0/4/0x00000406
kernel: [ 1117.962750] INFO: lockdep is turned off.
kernel: [ 1117.962756] Modules linked in: ppp_deflate zlib_deflate ppp_async crc_ccitt ppp_generic slhc ipv6 fuse loop snd_hda_codec_si3054 snd_hda_codec_realtek snd_hda_intel
snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device snd psmouse i2c_i801 pcspkr serio_raw rng_core soundcore snd_page_alloc
asus_laptop evdev usbhid hid sg sr_mod cdrom sd_mod ata_generic pata_acpi uhci_hcd ricoh_mmc sdhci_pci sdhci ata_piix ide_pci_generic mmc_core led_class ehci_hcd usbcore r8169 mii
kernel: [ 1117.962910] Pid: 4, comm: ksoftirqd/0 Not tainted 2.6.31-rc2-pnv-git1-00254-ga4e2f0a-dirty #2
kernel: [ 1117.962918] Call Trace:
kernel: [ 1117.962937]  [<c1035f55>] __schedule_bug+0x95/0xa0
kernel: [ 1117.962950]  [<c14394f2>] schedule+0x962/0xb60
kernel: [ 1117.962963]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1117.962978]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1117.962991]  [<c143a3df>] __mutex_lock_common+0x17f/0x450
kernel: [ 1117.963002]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1117.963013]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1117.963025]  [<c143a754>] mutex_lock_nested+0x24/0x40
kernel: [ 1117.963035]  [<c1300355>] ? tty_throttle+0x25/0x70
kernel: [ 1117.963046]  [<c1300355>] tty_throttle+0x25/0x70
kernel: [ 1117.963057]  [<c12fecd7>] n_tty_receive_buf+0x3a7/0x11c0
kernel: [ 1117.963069]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
kernel: [ 1117.963081]  [<c1033273>] ? __wake_up+0x53/0x70
kernel: [ 1117.963092]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1117.963103]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1117.963115]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
kernel: [ 1117.963127]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1117.963138]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1117.963149]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
kernel: [ 1117.963161]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1117.963172]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
kernel: [ 1117.963184]  [<c13012d2>] ? tty_ldisc_try+0x42/0x60
kernel: [ 1117.963195]  [<c13038dd>] pty_write+0x7d/0x90
kernel: [ 1117.963210]  [<f80b3c8a>] ppp_async_push+0x3da/0x4d0 [ppp_async]
kernel: [ 1117.963263]  [<f80b3dc8>] ppp_async_send+0x48/0x70 [ppp_async]
kernel: [ 1117.963285]  [<f8097086>] ppp_push+0x176/0x5f0 [ppp_generic]
kernel: [ 1117.963304]  [<c1397436>] ? skb_dequeue+0x26/0x80
kernel: [ 1117.963319]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1117.963335]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
kernel: [ 1117.963350]  [<c1397463>] ? skb_dequeue+0x53/0x80
kernel: [ 1117.963369]  [<f8097bcd>] ppp_xmit_process+0x3dd/0x570 [ppp_generic]
kernel: [ 1117.963385]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
kernel: [ 1117.963403]  [<c1397311>] ? skb_queue_tail+0x41/0x60
kernel: [ 1117.963420]  [<f8097ec7>] ppp_start_xmit+0x167/0x1b0 [ppp_generic]
kernel: [ 1117.963440]  [<c13b8522>] ? __qdisc_run+0x192/0x260
kernel: [ 1117.963459]  [<c13a466c>] dev_hard_start_xmit+0x21c/0x3c0
kernel: [ 1117.963473]  [<c143bdae>] ? _spin_lock+0x6e/0x90
kernel: [ 1117.963489]  [<c13b853c>] __qdisc_run+0x1ac/0x260
kernel: [ 1117.963502]  [<c13a38e7>] net_tx_action+0xf7/0x170
kernel: [ 1117.963521]  [<c104f9ea>] __do_softirq+0xba/0x240
kernel: [ 1117.963536]  [<c1272008>] ? trace_hardirqs_on_thunk+0xc/0x10
kernel: [ 1117.963552]  [<c104fbe5>] do_softirq+0x75/0x90
kernel: [ 1117.963569]  [<c10502e2>] ksoftirqd+0x82/0x1a0
kernel: [ 1117.963585]  [<c1050260>] ? ksoftirqd+0x0/0x1a0
kernel: [ 1117.963602]  [<c10646e4>] kthread+0x84/0x90
kernel: [ 1117.963624]  [<c1064660>] ? kthread+0x0/0x90
kernel: [ 1117.963641]  [<c1004217>] kernel_thread_helper+0x7/0x70


	Sergey

Comment 22 Rafael J. Wysocki 2009-07-07 20:34:46 UTC

On Tuesday 07 July 2009, Sergey Senozhatsky wrote:
> On (07/07/09 02:00), Rafael J. Wysocki wrote:
> > 
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> > 
> > The following bug entry is on the current list of known regressions
> > from 2.6.30.  Please verify if it still should be listed and let me know
> > (either way).
> > 
> > 
> > Bug-Entry   : http://bugzilla.kernel.org/show_bug.cgi?id=13522
> > Subject             : BUG: scheduling while atomic
> > Submitter   : Sergey Senozhatsky <sergey.senozhatsky@mail.by>
> > Date                : 2009-06-12 19:29 (25 days old)
> > References  : http://lkml.org/lkml/2009/6/29/237
> > 
> 
> it should be listed.
> 
> kernel: [ 1048.295158] BUG: scheduling while atomic: ksoftirqd/0/4/0x00000506
> kernel: [ 1048.295169] INFO: lockdep is turned off.
> kernel: [ 1048.295176] Modules linked in: ppp_deflate zlib_deflate ppp_async
> crc_ccitt ppp_generic slhc ipv6 fuse loop snd_hda_codec_si3054
> snd_hda_codec_realtek snd_hda_intel
> snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq
> snd_timer snd_seq_device snd psmouse i2c_i801 pcspkr serio_raw rng_core
> soundcore snd_page_alloc
> asus_laptop evdev usbhid hid sg sr_mod cdrom sd_mod ata_generic pata_acpi
> uhci_hcd ricoh_mmc sdhci_pci sdhci ata_piix ide_pci_generic mmc_core
> led_class ehci_hcd usbcore r8169 mii
> kernel: [ 1048.295334] Pid: 4, comm: ksoftirqd/0 Not tainted
> 2.6.31-rc2-pnv-git1-00254-ga4e2f0a-dirty #2
> kernel: [ 1048.295342] Call Trace:
> kernel: [ 1048.295361]  [<c1035f55>] __schedule_bug+0x95/0xa0
> kernel: [ 1048.295375]  [<c14394f2>] schedule+0x962/0xb60
> kernel: [ 1048.295388]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1048.295403]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1048.295416]  [<c143a3df>] __mutex_lock_common+0x17f/0x450
> kernel: [ 1048.295428]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1048.295438]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1048.295451]  [<c143a754>] mutex_lock_nested+0x24/0x40
> kernel: [ 1048.295461]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1048.295472]  [<c1300355>] tty_throttle+0x25/0x70
> kernel: [ 1048.295483]  [<c12fecd7>] n_tty_receive_buf+0x3a7/0x11c0
> kernel: [ 1048.295495]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1048.295507]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1048.295519]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1048.295531]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1048.295550]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1048.295561]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1048.295573]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
> kernel: [ 1048.295585]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
> kernel: [ 1048.295596]  [<c13012d2>] ? tty_ldisc_try+0x42/0x60
> kernel: [ 1048.295608]  [<c13038dd>] pty_write+0x7d/0x90
> kernel: [ 1048.295624]  [<f80b3c8a>] ppp_async_push+0x3da/0x4d0 [ppp_async]
> kernel: [ 1048.295639]  [<f80b3dc8>] ppp_async_send+0x48/0x70 [ppp_async]
> kernel: [ 1048.295655]  [<f8097086>] ppp_push+0x176/0x5f0 [ppp_generic]
> kernel: [ 1048.295670]  [<f8097816>] ? ppp_xmit_process+0x26/0x570
> [ppp_generic]
> kernel: [ 1048.295685]  [<f8097816>] ? ppp_xmit_process+0x26/0x570
> [ppp_generic]
> kernel: [ 1048.295702]  [<f809784a>] ppp_xmit_process+0x5a/0x570
> [ppp_generic]
> kernel: [ 1048.295717]  [<f8097fa5>] ? ppp_channel_push+0x95/0xd0
> [ppp_generic]
> kernel: [ 1048.295733]  [<f8097fb4>] ppp_channel_push+0xa4/0xd0 [ppp_generic]
> kernel: [ 1048.295748]  [<f8097ffe>] ppp_output_wakeup+0x1e/0x40
> [ppp_generic]
> kernel: [ 1048.295760]  [<f80b43f0>] ppp_async_process+0x90/0xa0 [ppp_async]
> kernel: [ 1048.295773]  [<c104ee15>] tasklet_action+0xf5/0x110
> kernel: [ 1048.295784]  [<c104f9ea>] __do_softirq+0xba/0x240
> kernel: [ 1048.295795]  [<c104fbe5>] do_softirq+0x75/0x90
> kernel: [ 1048.295807]  [<c10502e2>] ksoftirqd+0x82/0x1a0
> kernel: [ 1048.295817]  [<c1050260>] ? ksoftirqd+0x0/0x1a0
> kernel: [ 1048.295831]  [<c10646e4>] kthread+0x84/0x90
> kernel: [ 1048.295843]  [<c1064660>] ? kthread+0x0/0x90
> kernel: [ 1048.295855]  [<c1004217>] kernel_thread_helper+0x7/0x70
> kernel: [ 1048.296425] BUG: scheduling while atomic: ksoftirqd/0/4/0x00000406
> kernel: [ 1048.296431] INFO: lockdep is turned off.
> kernel: [ 1048.296437] Modules linked in: ppp_deflate zlib_deflate ppp_async
> crc_ccitt ppp_generic slhc ipv6 fuse loop snd_hda_codec_si3054
> snd_hda_codec_realtek snd_hda_intel
> snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq
> snd_timer snd_seq_device snd psmouse i2c_i801 pcspkr serio_raw rng_core
> soundcore snd_page_alloc
> asus_laptop evdev usbhid hid sg sr_mod cdrom sd_mod ata_generic pata_acpi
> uhci_hcd ricoh_mmc sdhci_pci sdhci ata_piix ide_pci_generic mmc_core
> led_class ehci_hcd usbcore r8169 mii
> kernel: [ 1048.296648] Pid: 4, comm: ksoftirqd/0 Not tainted
> 2.6.31-rc2-pnv-git1-00254-ga4e2f0a-dirty #2
> kernel: [ 1048.296661] Call Trace:
> kernel: [ 1048.296685]  [<c1035f55>] __schedule_bug+0x95/0xa0
> kernel: [ 1048.296706]  [<c14394f2>] schedule+0x962/0xb60
> kernel: [ 1048.296723]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1048.296741]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1048.296759]  [<c143a3df>] __mutex_lock_common+0x17f/0x450
> kernel: [ 1048.296779]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1048.296794]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1048.296811]  [<c143a754>] mutex_lock_nested+0x24/0x40
> kernel: [ 1048.296828]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1048.296842]  [<c1300355>] tty_throttle+0x25/0x70
> kernel: [ 1048.296858]  [<c12fecd7>] n_tty_receive_buf+0x3a7/0x11c0
> kernel: [ 1048.296872]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
> kernel: [ 1048.296885]  [<c1033273>] ? __wake_up+0x53/0x70
> kernel: [ 1048.296899]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1048.296916]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1048.296937]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1048.296955]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1048.296973]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1048.296989]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1048.297007]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
> kernel: [ 1048.297023]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
> kernel: [ 1048.297041]  [<c13012d2>] ? tty_ldisc_try+0x42/0x60
> kernel: [ 1048.297057]  [<c13038dd>] pty_write+0x7d/0x90
> kernel: [ 1048.297074]  [<f80b3c8a>] ppp_async_push+0x3da/0x4d0 [ppp_async]
> kernel: [ 1048.297096]  [<f80b3dc8>] ppp_async_send+0x48/0x70 [ppp_async]
> kernel: [ 1048.297113]  [<f8097086>] ppp_push+0x176/0x5f0 [ppp_generic]
> kernel: [ 1048.297135]  [<c1397436>] ? skb_dequeue+0x26/0x80
> kernel: [ 1048.297150]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
> kernel: [ 1048.297168]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
> kernel: [ 1048.297185]  [<c1397463>] ? skb_dequeue+0x53/0x80
> kernel: [ 1048.297204]  [<f8097bcd>] ppp_xmit_process+0x3dd/0x570
> [ppp_generic]
> kernel: [ 1048.297224]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
> kernel: [ 1048.297238]  [<c1397311>] ? skb_queue_tail+0x41/0x60
> kernel: [ 1048.297259]  [<f8097ec7>] ppp_start_xmit+0x167/0x1b0 [ppp_generic]
> kernel: [ 1048.297274]  [<c13b8522>] ? __qdisc_run+0x192/0x260
> kernel: [ 1048.297290]  [<c13a466c>] dev_hard_start_xmit+0x21c/0x3c0
> kernel: [ 1048.297305]  [<c143bdae>] ? _spin_lock+0x6e/0x90
> kernel: [ 1048.297321]  [<c13b853c>] __qdisc_run+0x1ac/0x260
> kernel: [ 1048.297342]  [<c13a38e7>] net_tx_action+0xf7/0x170
> kernel: [ 1048.297356]  [<c104f9ea>] __do_softirq+0xba/0x240
> kernel: [ 1048.297375]  [<c104fbe5>] do_softirq+0x75/0x90
> kernel: [ 1048.297389]  [<c10502e2>] ksoftirqd+0x82/0x1a0
> kernel: [ 1048.297401]  [<c1050260>] ? ksoftirqd+0x0/0x1a0
> kernel: [ 1048.297413]  [<c10646e4>] kthread+0x84/0x90
> kernel: [ 1048.297433]  [<c1064660>] ? kthread+0x0/0x90
> kernel: [ 1048.297453]  [<c1004217>] kernel_thread_helper+0x7/0x70
> kernel: [ 1048.298100] BUG: scheduling while atomic: ksoftirqd/0/4/0x00000406
> kernel: [ 1048.298115] INFO: lockdep is turned off.
> kernel: [ 1048.298127] Modules linked in: ppp_deflate zlib_deflate ppp_async
> crc_ccitt ppp_generic slhc ipv6 fuse loop snd_hda_codec_si3054
> snd_hda_codec_realtek snd_hda_intel
> snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq
> snd_timer snd_seq_device snd psmouse i2c_i801 pcspkr serio_raw rng_core
> soundcore snd_page_alloc
> asus_laptop evdev usbhid hid sg sr_mod cdrom sd_mod ata_generic pata_acpi
> uhci_hcd ricoh_mmc sdhci_pci sdhci ata_piix ide_pci_generic mmc_core
> led_class ehci_hcd usbcore r8169 mii
> kernel: [ 1048.298482] Pid: 4, comm: ksoftirqd/0 Not tainted
> 2.6.31-rc2-pnv-git1-00254-ga4e2f0a-dirty #2
> kernel: [ 1048.298490] Call Trace:
> kernel: [ 1048.298507]  [<c1035f55>] __schedule_bug+0x95/0xa0
> kernel: [ 1048.298527]  [<c14394f2>] schedule+0x962/0xb60
> kernel: [ 1048.298544]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1048.298562]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1048.298580]  [<c143a3df>] __mutex_lock_common+0x17f/0x450
> kernel: [ 1048.298594]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1048.298608]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1048.298626]  [<c143a754>] mutex_lock_nested+0x24/0x40
> kernel: [ 1048.298641]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1048.298662]  [<c1300355>] tty_throttle+0x25/0x70
> kernel: [ 1048.298683]  [<c12fecd7>] n_tty_receive_buf+0x3a7/0x11c0
> kernel: [ 1048.298698]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
> kernel: [ 1048.298716]  [<c1033273>] ? __wake_up+0x53/0x70
> kernel: [ 1048.298732]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1048.298750]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1048.298764]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1048.298780]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1048.298799]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1048.298817]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1048.298834]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
> kernel: [ 1048.298849]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
> kernel: [ 1048.298866]  [<c13012d2>] ? tty_ldisc_try+0x42/0x60
> kernel: [ 1048.298882]  [<c13038dd>] pty_write+0x7d/0x90
> kernel: [ 1048.298897]  [<f80b3c8a>] ppp_async_push+0x3da/0x4d0 [ppp_async]
> kernel: [ 1048.298916]  [<f80b3dc8>] ppp_async_send+0x48/0x70 [ppp_async]
> kernel: [ 1048.298944]  [<f8097086>] ppp_push+0x176/0x5f0 [ppp_generic]
> kernel: [ 1048.298963]  [<c1397436>] ? skb_dequeue+0x26/0x80
> kernel: [ 1048.298980]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
> kernel: [ 1048.299000]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
> kernel: [ 1048.299013]  [<c1397463>] ? skb_dequeue+0x53/0x80
> kernel: [ 1048.299038]  [<f8097bcd>] ppp_xmit_process+0x3dd/0x570
> [ppp_generic]
> kernel: [ 1048.299054]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
> kernel: [ 1048.299074]  [<c1397311>] ? skb_queue_tail+0x41/0x60
> kernel: [ 1048.299092]  [<f8097ec7>] ppp_start_xmit+0x167/0x1b0 [ppp_generic]
> kernel: [ 1048.299110]  [<c13b8522>] ? __qdisc_run+0x192/0x260
> kernel: [ 1048.299130]  [<c13a466c>] dev_hard_start_xmit+0x21c/0x3c0
> kernel: [ 1048.299145]  [<c143bdae>] ? _spin_lock+0x6e/0x90
> kernel: [ 1048.299168]  [<c13b853c>] __qdisc_run+0x1ac/0x260
> kernel: [ 1048.299186]  [<c13a38e7>] net_tx_action+0xf7/0x170
> kernel: [ 1048.299208]  [<c104f9ea>] __do_softirq+0xba/0x240
> kernel: [ 1048.299226]  [<c104fbe5>] do_softirq+0x75/0x90
> kernel: [ 1048.299240]  [<c10502e2>] ksoftirqd+0x82/0x1a0
> kernel: [ 1048.299253]  [<c1050260>] ? ksoftirqd+0x0/0x1a0
> kernel: [ 1048.299268]  [<c10646e4>] kthread+0x84/0x90
> kernel: [ 1048.299288]  [<c1064660>] ? kthread+0x0/0x90
> kernel: [ 1048.299301]  [<c1004217>] kernel_thread_helper+0x7/0x70
> kernel: [ 1067.299845] BUG: scheduling while atomic: ksoftirqd/0/4/0x00000506
> kernel: [ 1067.299854] INFO: lockdep is turned off.
> kernel: [ 1067.299860] Modules linked in: ppp_deflate zlib_deflate ppp_async
> crc_ccitt ppp_generic slhc ipv6 fuse loop snd_hda_codec_si3054
> snd_hda_codec_realtek snd_hda_intel
> snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq
> snd_timer snd_seq_device snd psmouse i2c_i801 pcspkr serio_raw rng_core
> soundcore snd_page_alloc
> asus_laptop evdev usbhid hid sg sr_mod cdrom sd_mod ata_generic pata_acpi
> uhci_hcd ricoh_mmc sdhci_pci sdhci ata_piix ide_pci_generic mmc_core
> led_class ehci_hcd usbcore r8169 mii
> kernel: [ 1067.300216] Pid: 4, comm: ksoftirqd/0 Not tainted
> 2.6.31-rc2-pnv-git1-00254-ga4e2f0a-dirty #2
> kernel: [ 1067.300229] Call Trace:
> kernel: [ 1067.300249]  [<c1035f55>] __schedule_bug+0x95/0xa0
> kernel: [ 1067.300270]  [<c14394f2>] schedule+0x962/0xb60
> kernel: [ 1067.300289]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1067.300306]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1067.300324]  [<c143a3df>] __mutex_lock_common+0x17f/0x450
> kernel: [ 1067.300347]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1067.300365]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1067.300384]  [<c143a754>] mutex_lock_nested+0x24/0x40
> kernel: [ 1067.300400]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1067.300418]  [<c1300355>] tty_throttle+0x25/0x70
> kernel: [ 1067.300438]  [<c12fecd7>] n_tty_receive_buf+0x3a7/0x11c0
> kernel: [ 1067.300452]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1067.300468]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1067.300485]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1067.300502]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1067.300518]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1067.300535]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1067.300554]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
> kernel: [ 1067.300569]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
> kernel: [ 1067.300585]  [<c13012d2>] ? tty_ldisc_try+0x42/0x60
> kernel: [ 1067.300600]  [<c13038dd>] pty_write+0x7d/0x90
> kernel: [ 1067.300621]  [<f80b3c8a>] ppp_async_push+0x3da/0x4d0 [ppp_async]
> kernel: [ 1067.300640]  [<f80b3dc8>] ppp_async_send+0x48/0x70 [ppp_async]
> kernel: [ 1067.300665]  [<f8097086>] ppp_push+0x176/0x5f0 [ppp_generic]
> kernel: [ 1067.300687]  [<f8097816>] ? ppp_xmit_process+0x26/0x570
> [ppp_generic]
> kernel: [ 1067.300709]  [<f8097816>] ? ppp_xmit_process+0x26/0x570
> [ppp_generic]
> kernel: [ 1067.300731]  [<f809784a>] ppp_xmit_process+0x5a/0x570
> [ppp_generic]
> kernel: [ 1067.300746]  [<f8097fa5>] ? ppp_channel_push+0x95/0xd0
> [ppp_generic]
> kernel: [ 1067.300763]  [<f8097fb4>] ppp_channel_push+0xa4/0xd0 [ppp_generic]
> kernel: [ 1067.300781]  [<f8097ffe>] ppp_output_wakeup+0x1e/0x40
> [ppp_generic]
> kernel: [ 1067.300804]  [<f80b43f0>] ppp_async_process+0x90/0xa0 [ppp_async]
> kernel: [ 1067.300822]  [<c104ee15>] tasklet_action+0xf5/0x110
> kernel: [ 1067.300841]  [<c104f9ea>] __do_softirq+0xba/0x240
> kernel: [ 1067.300855]  [<c104fbe5>] do_softirq+0x75/0x90
> kernel: [ 1067.300873]  [<c10502e2>] ksoftirqd+0x82/0x1a0
> kernel: [ 1067.300886]  [<c1050260>] ? ksoftirqd+0x0/0x1a0
> kernel: [ 1067.300901]  [<c10646e4>] kthread+0x84/0x90
> kernel: [ 1067.300918]  [<c1064660>] ? kthread+0x0/0x90
> kernel: [ 1067.300937]  [<c1004217>] kernel_thread_helper+0x7/0x70
> kernel: [ 1067.301595] BUG: scheduling while atomic: ksoftirqd/0/4/0x00000406
> kernel: [ 1067.301608] INFO: lockdep is turned off.
> kernel: [ 1067.301617] Modules linked in: ppp_deflate zlib_deflate ppp_async
> crc_ccitt ppp_generic slhc ipv6 fuse loop snd_hda_codec_si3054
> snd_hda_codec_realtek snd_hda_intel
> snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq
> snd_timer snd_seq_device snd psmouse i2c_i801 pcspkr serio_raw rng_core
> soundcore snd_page_alloc
> asus_laptop evdev usbhid hid sg sr_mod cdrom sd_mod ata_generic pata_acpi
> uhci_hcd ricoh_mmc sdhci_pci sdhci ata_piix ide_pci_generic mmc_core
> led_class ehci_hcd usbcore r8169 mii
> kernel: [ 1067.301969] Pid: 4, comm: ksoftirqd/0 Not tainted
> 2.6.31-rc2-pnv-git1-00254-ga4e2f0a-dirty #2
> kernel: [ 1067.301981] Call Trace:
> kernel: [ 1067.301998]  [<c1035f55>] __schedule_bug+0x95/0xa0
> kernel: [ 1067.302018]  [<c14394f2>] schedule+0x962/0xb60
> kernel: [ 1067.302031]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1067.302052]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1067.302072]  [<c143a3df>] __mutex_lock_common+0x17f/0x450
> kernel: [ 1067.302092]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1067.302110]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1067.302126]  [<c143a754>] mutex_lock_nested+0x24/0x40
> kernel: [ 1067.302139]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1067.302152]  [<c1300355>] tty_throttle+0x25/0x70
> kernel: [ 1067.302167]  [<c12fecd7>] n_tty_receive_buf+0x3a7/0x11c0
> kernel: [ 1067.302188]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
> kernel: [ 1067.302203]  [<c1033273>] ? __wake_up+0x53/0x70
> kernel: [ 1067.302219]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1067.302234]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1067.302251]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1067.302268]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1067.302284]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1067.302301]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1067.302319]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
> kernel: [ 1067.302339]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
> kernel: [ 1067.302352]  [<c13012d2>] ? tty_ldisc_try+0x42/0x60
> kernel: [ 1067.302370]  [<c13038dd>] pty_write+0x7d/0x90
> kernel: [ 1067.302389]  [<f80b3c8a>] ppp_async_push+0x3da/0x4d0 [ppp_async]
> kernel: [ 1067.302412]  [<f80b3dc8>] ppp_async_send+0x48/0x70 [ppp_async]
> kernel: [ 1067.302437]  [<f8097086>] ppp_push+0x176/0x5f0 [ppp_generic]
> kernel: [ 1067.302454]  [<c1397436>] ? skb_dequeue+0x26/0x80
> kernel: [ 1067.302473]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
> kernel: [ 1067.302490]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
> kernel: [ 1067.302503]  [<c1397463>] ? skb_dequeue+0x53/0x80
> kernel: [ 1067.302526]  [<f8097bcd>] ppp_xmit_process+0x3dd/0x570
> [ppp_generic]
> kernel: [ 1067.302541]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
> kernel: [ 1067.302558]  [<c1397311>] ? skb_queue_tail+0x41/0x60
> kernel: [ 1067.302577]  [<f8097ec7>] ppp_start_xmit+0x167/0x1b0 [ppp_generic]
> kernel: [ 1067.302601]  [<c13b8522>] ? __qdisc_run+0x192/0x260
> kernel: [ 1067.302622]  [<c13a466c>] dev_hard_start_xmit+0x21c/0x3c0
> kernel: [ 1067.302640]  [<c143bdae>] ? _spin_lock+0x6e/0x90
> kernel: [ 1067.302654]  [<c13b853c>] __qdisc_run+0x1ac/0x260
> kernel: [ 1067.302675]  [<c13a38e7>] net_tx_action+0xf7/0x170
> kernel: [ 1067.302698]  [<c104f9ea>] __do_softirq+0xba/0x240
> kernel: [ 1067.302713]  [<c104fbe5>] do_softirq+0x75/0x90
> kernel: [ 1067.302728]  [<c10502e2>] ksoftirqd+0x82/0x1a0
> kernel: [ 1067.302745]  [<c1050260>] ? ksoftirqd+0x0/0x1a0
> kernel: [ 1067.302761]  [<c10646e4>] kthread+0x84/0x90
> kernel: [ 1067.302783]  [<c1064660>] ? kthread+0x0/0x90
> kernel: [ 1067.302798]  [<c1004217>] kernel_thread_helper+0x7/0x70
> kernel: [ 1076.490281] BUG: scheduling while atomic: ksoftirqd/0/4/0x00000406
> kernel: [ 1076.490290] INFO: lockdep is turned off.
> kernel: [ 1076.490296] Modules linked in: ppp_deflate zlib_deflate ppp_async
> crc_ccitt ppp_generic slhc ipv6 fuse loop snd_hda_codec_si3054
> snd_hda_codec_realtek snd_hda_intel
> snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq
> snd_timer snd_seq_device snd psmouse i2c_i801 pcspkr serio_raw rng_core
> soundcore snd_page_alloc
> asus_laptop evdev usbhid hid sg sr_mod cdrom sd_mod ata_generic pata_acpi
> uhci_hcd ricoh_mmc sdhci_pci sdhci ata_piix ide_pci_generic mmc_core
> led_class ehci_hcd usbcore r8169 mii
> kernel: [ 1076.490452] Pid: 4, comm: ksoftirqd/0 Not tainted
> 2.6.31-rc2-pnv-git1-00254-ga4e2f0a-dirty #2
> kernel: [ 1076.490460] Call Trace:
> kernel: [ 1076.490478]  [<c1035f55>] __schedule_bug+0x95/0xa0
> kernel: [ 1076.490492]  [<c14394f2>] schedule+0x962/0xb60
> kernel: [ 1076.490506]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1076.490521]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1076.490534]  [<c143a3df>] __mutex_lock_common+0x17f/0x450
> kernel: [ 1076.490545]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1076.490556]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1076.490568]  [<c143a754>] mutex_lock_nested+0x24/0x40
> kernel: [ 1076.490579]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1076.490590]  [<c1300355>] tty_throttle+0x25/0x70
> kernel: [ 1076.490601]  [<c12fecd7>] n_tty_receive_buf+0x3a7/0x11c0
> kernel: [ 1076.490613]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
> kernel: [ 1076.490625]  [<c1033273>] ? __wake_up+0x53/0x70
> kernel: [ 1076.490636]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1076.490648]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1076.490659]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1076.490672]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1076.490682]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1076.490693]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1076.490705]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
> kernel: [ 1076.490717]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
> kernel: [ 1076.490728]  [<c13012d2>] ? tty_ldisc_try+0x42/0x60
> kernel: [ 1076.490741]  [<c13038dd>] pty_write+0x7d/0x90
> kernel: [ 1076.490757]  [<f80b3c8a>] ppp_async_push+0x3da/0x4d0 [ppp_async]
> kernel: [ 1076.490774]  [<f80b3dc8>] ppp_async_send+0x48/0x70 [ppp_async]
> kernel: [ 1076.490790]  [<f8097086>] ppp_push+0x176/0x5f0 [ppp_generic]
> kernel: [ 1076.490802]  [<c1397436>] ? skb_dequeue+0x26/0x80
> kernel: [ 1076.490815]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
> kernel: [ 1076.490835]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
> kernel: [ 1076.490846]  [<c1397463>] ? skb_dequeue+0x53/0x80
> kernel: [ 1076.490861]  [<f8097bcd>] ppp_xmit_process+0x3dd/0x570
> [ppp_generic]
> kernel: [ 1076.490874]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
> kernel: [ 1076.490885]  [<c1397311>] ? skb_queue_tail+0x41/0x60
> kernel: [ 1076.490900]  [<f8097ec7>] ppp_start_xmit+0x167/0x1b0 [ppp_generic]
> kernel: [ 1076.490912]  [<c13b8522>] ? __qdisc_run+0x192/0x260
> kernel: [ 1076.490925]  [<c13a466c>] dev_hard_start_xmit+0x21c/0x3c0
> kernel: [ 1076.490937]  [<c143bdae>] ? _spin_lock+0x6e/0x90
> kernel: [ 1076.490948]  [<c13b853c>] __qdisc_run+0x1ac/0x260
> kernel: [ 1076.490960]  [<c13a38e7>] net_tx_action+0xf7/0x170
> kernel: [ 1076.490972]  [<c104f9ea>] __do_softirq+0xba/0x240
> kernel: [ 1076.490984]  [<c104fbe5>] do_softirq+0x75/0x90
> kernel: [ 1076.490995]  [<c10502e2>] ksoftirqd+0x82/0x1a0
> kernel: [ 1076.491005]  [<c1050260>] ? ksoftirqd+0x0/0x1a0
> kernel: [ 1076.491017]  [<c10646e4>] kthread+0x84/0x90
> kernel: [ 1076.491029]  [<c1064660>] ? kthread+0x0/0x90
> kernel: [ 1076.491041]  [<c1004217>] kernel_thread_helper+0x7/0x70
> kernel: [ 1076.491622] BUG: scheduling while atomic: ksoftirqd/0/4/0x00000406
> kernel: [ 1076.491629] INFO: lockdep is turned off.
> kernel: [ 1076.491635] Modules linked in: ppp_deflate zlib_deflate ppp_async
> crc_ccitt ppp_generic slhc ipv6 fuse loop snd_hda_codec_si3054
> snd_hda_codec_realtek snd_hda_intel
> snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq
> snd_timer snd_seq_device snd psmouse i2c_i801 pcspkr serio_raw rng_core
> soundcore snd_page_alloc
> asus_laptop evdev usbhid hid sg sr_mod cdrom sd_mod ata_generic pata_acpi
> uhci_hcd ricoh_mmc sdhci_pci sdhci ata_piix ide_pci_generic mmc_core
> led_class ehci_hcd usbcore r8169 mii
> kernel: [ 1076.491786] Pid: 4, comm: ksoftirqd/0 Not tainted
> 2.6.31-rc2-pnv-git1-00254-ga4e2f0a-dirty #2
> kernel: [ 1076.491794] Call Trace:
> kernel: [ 1076.491807]  [<c1035f55>] __schedule_bug+0x95/0xa0
> kernel: [ 1076.491818]  [<c14394f2>] schedule+0x962/0xb60
> kernel: [ 1076.491829]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1076.491842]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1076.491856]  [<c143a3df>] __mutex_lock_common+0x17f/0x450
> kernel: [ 1076.491867]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1076.491877]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1076.491889]  [<c143a754>] mutex_lock_nested+0x24/0x40
> kernel: [ 1076.491900]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1076.491910]  [<c1300355>] tty_throttle+0x25/0x70
> kernel: [ 1076.491921]  [<c12fecd7>] n_tty_receive_buf+0x3a7/0x11c0
> kernel: [ 1076.491933]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
> kernel: [ 1076.491945]  [<c1033273>] ? __wake_up+0x53/0x70
> kernel: [ 1076.491955]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1076.491967]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1076.491979]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1076.491991]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1076.492001]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1076.492012]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1076.492024]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
> kernel: [ 1076.492035]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
> kernel: [ 1076.492047]  [<c13012d2>] ? tty_ldisc_try+0x42/0x60
> kernel: [ 1076.492058]  [<c13038dd>] pty_write+0x7d/0x90
> kernel: [ 1076.492072]  [<f80b3c8a>] ppp_async_push+0x3da/0x4d0 [ppp_async]
> kernel: [ 1076.492088]  [<f80b3dc8>] ppp_async_send+0x48/0x70 [ppp_async]
> kernel: [ 1076.492103]  [<f8097086>] ppp_push+0x176/0x5f0 [ppp_generic]
> kernel: [ 1076.492114]  [<c1397436>] ? skb_dequeue+0x26/0x80
> kernel: [ 1076.492126]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
> kernel: [ 1076.492138]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
> kernel: [ 1076.492149]  [<c1397463>] ? skb_dequeue+0x53/0x80
> kernel: [ 1076.492164]  [<f8097bcd>] ppp_xmit_process+0x3dd/0x570
> [ppp_generic]
> kernel: [ 1076.492176]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
> kernel: [ 1076.492187]  [<c1397311>] ? skb_queue_tail+0x41/0x60
> kernel: [ 1076.492202]  [<f8097ec7>] ppp_start_xmit+0x167/0x1b0 [ppp_generic]
> kernel: [ 1076.492213]  [<c13b8522>] ? __qdisc_run+0x192/0x260
> kernel: [ 1076.492225]  [<c13a466c>] dev_hard_start_xmit+0x21c/0x3c0
> kernel: [ 1076.492236]  [<c143bdae>] ? _spin_lock+0x6e/0x90
> kernel: [ 1076.492248]  [<c13b853c>] __qdisc_run+0x1ac/0x260
> kernel: [ 1076.492259]  [<c13a38e7>] net_tx_action+0xf7/0x170
> kernel: [ 1076.492270]  [<c104f9ea>] __do_softirq+0xba/0x240
> kernel: [ 1076.492282]  [<c104fbe5>] do_softirq+0x75/0x90
> kernel: [ 1076.492292]  [<c10502e2>] ksoftirqd+0x82/0x1a0
> kernel: [ 1076.492302]  [<c1050260>] ? ksoftirqd+0x0/0x1a0
> kernel: [ 1076.492313]  [<c10646e4>] kthread+0x84/0x90
> kernel: [ 1076.492325]  [<c1064660>] ? kthread+0x0/0x90
> kernel: [ 1076.492336]  [<c1004217>] kernel_thread_helper+0x7/0x70
> kernel: [ 1117.962740] BUG: scheduling while atomic: ksoftirqd/0/4/0x00000406
> kernel: [ 1117.962750] INFO: lockdep is turned off.
> kernel: [ 1117.962756] Modules linked in: ppp_deflate zlib_deflate ppp_async
> crc_ccitt ppp_generic slhc ipv6 fuse loop snd_hda_codec_si3054
> snd_hda_codec_realtek snd_hda_intel
> snd_hda_codec snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq
> snd_timer snd_seq_device snd psmouse i2c_i801 pcspkr serio_raw rng_core
> soundcore snd_page_alloc
> asus_laptop evdev usbhid hid sg sr_mod cdrom sd_mod ata_generic pata_acpi
> uhci_hcd ricoh_mmc sdhci_pci sdhci ata_piix ide_pci_generic mmc_core
> led_class ehci_hcd usbcore r8169 mii
> kernel: [ 1117.962910] Pid: 4, comm: ksoftirqd/0 Not tainted
> 2.6.31-rc2-pnv-git1-00254-ga4e2f0a-dirty #2
> kernel: [ 1117.962918] Call Trace:
> kernel: [ 1117.962937]  [<c1035f55>] __schedule_bug+0x95/0xa0
> kernel: [ 1117.962950]  [<c14394f2>] schedule+0x962/0xb60
> kernel: [ 1117.962963]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1117.962978]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1117.962991]  [<c143a3df>] __mutex_lock_common+0x17f/0x450
> kernel: [ 1117.963002]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1117.963013]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1117.963025]  [<c143a754>] mutex_lock_nested+0x24/0x40
> kernel: [ 1117.963035]  [<c1300355>] ? tty_throttle+0x25/0x70
> kernel: [ 1117.963046]  [<c1300355>] tty_throttle+0x25/0x70
> kernel: [ 1117.963057]  [<c12fecd7>] n_tty_receive_buf+0x3a7/0x11c0
> kernel: [ 1117.963069]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
> kernel: [ 1117.963081]  [<c1033273>] ? __wake_up+0x53/0x70
> kernel: [ 1117.963092]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1117.963103]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1117.963115]  [<c107b015>] ? print_lock_contention_bug+0x25/0x100
> kernel: [ 1117.963127]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1117.963138]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1117.963149]  [<c13012b5>] ? tty_ldisc_try+0x25/0x60
> kernel: [ 1117.963161]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
> kernel: [ 1117.963172]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
> kernel: [ 1117.963184]  [<c13012d2>] ? tty_ldisc_try+0x42/0x60
> kernel: [ 1117.963195]  [<c13038dd>] pty_write+0x7d/0x90
> kernel: [ 1117.963210]  [<f80b3c8a>] ppp_async_push+0x3da/0x4d0 [ppp_async]
> kernel: [ 1117.963263]  [<f80b3dc8>] ppp_async_send+0x48/0x70 [ppp_async]
> kernel: [ 1117.963285]  [<f8097086>] ppp_push+0x176/0x5f0 [ppp_generic]
> kernel: [ 1117.963304]  [<c1397436>] ? skb_dequeue+0x26/0x80
> kernel: [ 1117.963319]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
> kernel: [ 1117.963335]  [<c143c5f7>] ? _spin_unlock_irqrestore+0x47/0x90
> kernel: [ 1117.963350]  [<c1397463>] ? skb_dequeue+0x53/0x80
> kernel: [ 1117.963369]  [<f8097bcd>] ppp_xmit_process+0x3dd/0x570
> [ppp_generic]
> kernel: [ 1117.963385]  [<c107c709>] ? trace_hardirqs_on+0x19/0x40
> kernel: [ 1117.963403]  [<c1397311>] ? skb_queue_tail+0x41/0x60
> kernel: [ 1117.963420]  [<f8097ec7>] ppp_start_xmit+0x167/0x1b0 [ppp_generic]
> kernel: [ 1117.963440]  [<c13b8522>] ? __qdisc_run+0x192/0x260
> kernel: [ 1117.963459]  [<c13a466c>] dev_hard_start_xmit+0x21c/0x3c0
> kernel: [ 1117.963473]  [<c143bdae>] ? _spin_lock+0x6e/0x90
> kernel: [ 1117.963489]  [<c13b853c>] __qdisc_run+0x1ac/0x260
> kernel: [ 1117.963502]  [<c13a38e7>] net_tx_action+0xf7/0x170
> kernel: [ 1117.963521]  [<c104f9ea>] __do_softirq+0xba/0x240
> kernel: [ 1117.963536]  [<c1272008>] ? trace_hardirqs_on_thunk+0xc/0x10
> kernel: [ 1117.963552]  [<c104fbe5>] do_softirq+0x75/0x90
> kernel: [ 1117.963569]  [<c10502e2>] ksoftirqd+0x82/0x1a0
> kernel: [ 1117.963585]  [<c1050260>] ? ksoftirqd+0x0/0x1a0
> kernel: [ 1117.963602]  [<c10646e4>] kthread+0x84/0x90
> kernel: [ 1117.963624]  [<c1064660>] ? kthread+0x0/0x90
> kernel: [ 1117.963641]  [<c1004217>] kernel_thread_helper+0x7/0x70