Subject : [r8169] WARNING: at net/sched/sch_generic.c Submitter : Sergey Senozhatsky <sergey.senozhatsky@gmail.com> Date : 2010-03-31 10:21 Message-ID : 20100331102142.GA3294@swordfish.minsk.epam.com References : http://marc.info/?l=linux-kernel&m=127003090406108&w=2 This entry is being used for tracking a regression from 2.6.33. Please don't close it until the problem is fixed in the mainline.
On Wednesday 14 April 2010, Sergey Senozhatsky wrote: > Hello, > Sorry for the delay, I've been away. > > On (04/07/10 23:13), Rafael J. Wysocki wrote: > > This message has been generated automatically as a part of a summary report > > of recent regressions. > > > > The following bug entry is on the current list of known regressions > > from 2.6.33. Please verify if it still should be listed and let the > tracking team > > know (either way). > > > > > > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=15704 > > Subject : [r8169] WARNING: at net/sched/sch_generic.c > > Submitter : Sergey Senozhatsky <sergey.senozhatsky@gmail.com> > > Date : 2010-03-31 10:21 (8 days old) > > Message-ID : <20100331102142.GA3294@swordfish.minsk.epam.com> > > References : http://marc.info/?l=linux-kernel&m=127003090406108&w=2 > > > > > > > Yes, bug is still here (though I'm not sure whether it's correct to test with > frame size 2048). > > Here it is: > > kernel: [ 2292.172886] pktgen 2.72: Packet Generator for packet performance > testing. > kernel: [ 2339.999788] ------------[ cut here ]------------ > kernel: [ 2339.999800] WARNING: at net/sched/sch_generic.c:256 > dev_watchdog+0xc1/0x125() > kernel: [ 2339.999803] Hardware name: F3JC > kernel: [ 2339.999806] NETDEV WATCHDOG: eth0 (r8169): transmit queue 0 timed > out > kernel: [ 2339.999809] Modules linked in: pktgen ipv6 snd_hwdep > snd_hda_codec_si3054 snd_hda_codec_realtek sdhci_pci sdhci asus_laptop > sparse_keymap snd_hda_intel mmc_core > led_class snd_hda_codec psmouse snd_pcm snd_timer snd soundcore > snd_page_alloc i2c_i801 serio_raw sg evdev rng_core r8169 mii usbhid hid > uhci_hcd ehci_hcd sr_mod cdrom sd_mod > usbcore ata_piix > kernel: [ 2339.999857] Pid: 3720, comm: kpktgend_0 Not tainted > 2.6.34-rc3-dbg-r8169-git6 #81 > kernel: [ 2339.999861] Call Trace: > kernel: [ 2339.999868] [<c102e7f7>] warn_slowpath_common+0x65/0x7c > kernel: [ 2339.999874] [<c126e4f8>] ? dev_watchdog+0xc1/0x125 > kernel: [ 2339.999879] [<c102e842>] warn_slowpath_fmt+0x24/0x27 > kernel: [ 2339.999884] [<c126e4f8>] dev_watchdog+0xc1/0x125 > kernel: [ 2339.999891] [<c1040039>] ? abort_exclusive_wait+0x26/0x63 > kernel: [ 2339.999897] [<c103712b>] ? run_timer_softirq+0x120/0x1eb > kernel: [ 2339.999902] [<c1037181>] run_timer_softirq+0x176/0x1eb > kernel: [ 2339.999907] [<c103712b>] ? run_timer_softirq+0x120/0x1eb > kernel: [ 2339.999913] [<c126e437>] ? dev_watchdog+0x0/0x125 > kernel: [ 2339.999919] [<c10332a1>] __do_softirq+0x8d/0x117 > kernel: [ 2339.999925] [<c1033356>] do_softirq+0x2b/0x43 > kernel: [ 2339.999929] [<c103347b>] irq_exit+0x38/0x75 > kernel: [ 2339.999936] [<c1015200>] smp_apic_timer_interrupt+0x6d/0x7b > kernel: [ 2339.999943] [<c12cbcda>] apic_timer_interrupt+0x36/0x3c > kernel: [ 2339.999948] [<c104007b>] ? finish_wait+0x5/0x49 > kernel: [ 2339.999953] [<c10400d8>] ? prepare_to_wait_exclusive+0x19/0x5b > kernel: [ 2339.999959] [<c105106d>] ? lock_release+0x19a/0x1a1 > kernel: [ 2339.999965] [<c12cb259>] _raw_spin_unlock+0x16/0x3f > kernel: [ 2339.999972] [<f80ea460>] pktgen_thread_worker+0xe0/0x631 [pktgen] > kernel: [ 2339.999978] [<c103ffad>] ? autoremove_wake_function+0x0/0x2f > kernel: [ 2339.999983] [<c103ffad>] ? autoremove_wake_function+0x0/0x2f > kernel: [ 2339.999989] [<f80ea380>] ? pktgen_thread_worker+0x0/0x631 > [pktgen] > kernel: [ 2339.999994] [<c103fc8a>] kthread+0x6a/0x6f > kernel: [ 2339.999999] [<c103fc20>] ? kthread+0x0/0x6f > kernel: [ 2340.000005] [<c1002e82>] kernel_thread_helper+0x6/0x1a > kernel: [ 2340.000008] ---[ end trace 25113681b662d02f ]---
On Monday 26 April 2010, Sergey Senozhatsky wrote: > Hello, > > On (04/20/10 05:19), Rafael J. Wysocki wrote: > > This message has been generated automatically as a part of a summary report > > of recent regressions. > > > > The following bug entry is on the current list of known regressions > > from 2.6.33. Please verify if it still should be listed and let the > tracking team > > know (either way). > > > > > > .34-rc5-git7 > > kernel: [12887.906682] pktgen 2.72: Packet Generator for packet performance > testing. > kernel: [12938.998730] ------------[ cut here ]------------ > kernel: [12938.998741] WARNING: at net/sched/sch_generic.c:256 > dev_watchdog+0xc1/0x129() > kernel: [12938.998745] Hardware name: F3JC > kernel: [12938.998748] NETDEV WATCHDOG: eth0 (r8169): transmit queue 0 timed > out > kernel: [12938.998751] Modules linked in: pktgen usb_storage ipv6 snd_hwdep > snd_hda_codec_si3054 snd_hda_codec_realtek asus_laptop sparse_keymap > sdhci_pci sdhci snd_hda_intel mmc_core led_class psmouse snd_hda_codec > snd_pcm snd_timer snd > soundcore snd_page_alloc serio_raw rng_core sg i2c_i801 evdev r8169 mii > usbhid hid uhci_hcd ehci_hcd sr_mod cdrom sd_mod usbcore ata_piix > kernel: [12938.998808] Pid: 4617, comm: kpktgend_0 Not tainted > 2.6.34-rc5-dbg-r8169-git7 #97 > kernel: [12938.998811] Call Trace: > kernel: [12938.998819] [<c102e1b2>] warn_slowpath_common+0x65/0x7c > kernel: [12938.998824] [<c1268085>] ? dev_watchdog+0xc1/0x129 > kernel: [12938.998829] [<c102e1fd>] warn_slowpath_fmt+0x24/0x27 > kernel: [12938.998834] [<c1268085>] dev_watchdog+0xc1/0x129 > kernel: [12938.998841] [<c1040039>] ? __kfifo_from_user_generic+0x30/0x5c > kernel: [12938.998848] [<c1036ae3>] ? run_timer_softirq+0x136/0x203 > kernel: [12938.998853] [<c1036b3c>] run_timer_softirq+0x18f/0x203 > kernel: [12938.998858] [<c1036ae3>] ? run_timer_softirq+0x136/0x203 > kernel: [12938.998864] [<c1267fc4>] ? dev_watchdog+0x0/0x129 > kernel: [12938.998870] [<c1032a72>] __do_softirq+0x88/0x10c > kernel: [12938.998875] [<c1032b25>] do_softirq+0x2f/0x47 > kernel: [12938.998883] [<f8095488>] ? pktgen_xmit+0xd3e/0xe0b [pktgen] > kernel: [12938.998888] [<c1032d08>] _local_bh_enable_ip+0x8b/0xb3 > kernel: [12938.998894] [<c1032d38>] local_bh_enable_ip+0x8/0xa > kernel: [12938.998900] [<c12c3818>] _raw_spin_unlock_bh+0x2f/0x32 > kernel: [12938.998906] [<f8095488>] pktgen_xmit+0xd3e/0xe0b [pktgen] > kernel: [12938.998913] [<c104463c>] ? sched_clock_cpu+0x121/0x131 > kernel: [12938.998922] [<fd1cf5e4>] ? rtl8169_start_xmit+0x0/0x2f9 [r8169] > kernel: [12938.998928] [<f8095793>] ? pktgen_thread_worker+0xd9/0x5e8 > [pktgen] > kernel: [12938.998934] [<c1044679>] ? cpu_clock+0x2d/0x4e > kernel: [12938.998941] [<c104d1f7>] ? put_lock_stats+0xd/0x21 > kernel: [12938.998946] [<c104d2c0>] ? lock_release_holdtime+0xb5/0xba > kernel: [12938.998952] [<f8095793>] ? pktgen_thread_worker+0xd9/0x5e8 > [pktgen] > kernel: [12938.998959] [<f809582a>] pktgen_thread_worker+0x170/0x5e8 > [pktgen] > kernel: [12938.998965] [<c103fa46>] ? autoremove_wake_function+0x0/0x2f > kernel: [12938.998971] [<c103fa46>] ? autoremove_wake_function+0x0/0x2f > kernel: [12938.998977] [<f80956ba>] ? pktgen_thread_worker+0x0/0x5e8 > [pktgen] > kernel: [12938.998982] [<c103f76a>] kthread+0x6a/0x6f > kernel: [12938.998988] [<c103f700>] ? kthread+0x0/0x6f > kernel: [12938.998994] [<c1002dc2>] kernel_thread_helper+0x6/0x10 > kernel: [12938.998998] ---[ end trace 030ae30a64f8cf05 ]--- > kernel: [12939.012657] r8169 0000:02:00.0: eth0: link up > kernel: [12951.012273] r8169 0000:02:00.0: eth0: link up > kernel: [12951.475547] pktgen: interface: eth0 is already stopped
On Thursday 06 May 2010, Sergey Senozhatsky wrote: > Hello, > > On (05/04/10 23:21), Rafael J. Wysocki wrote: > > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=15704 > > Subject : [r8169] WARNING: at net/sched/sch_generic.c > > Submitter : Sergey Senozhatsky <sergey.senozhatsky@gmail.com> > > Date : 2010-03-31 10:21 (35 days old) > > Message-ID : <<20100331102142.GA3294@swordfish.minsk.epam.com>> > > References : http://marc.info/?l=linux-kernel&m=127003090406108&w=2 > > > > > > .34-rc6-git5 > > kernel: [ 1176.647885] pktgen 2.72: Packet Generator for packet performance > testing. > kernel: [ 1210.999903] ------------[ cut here ]------------ > kernel: [ 1210.999914] WARNING: at net/sched/sch_generic.c:256 > dev_watchdog+0xc1/0x129() > kernel: [ 1210.999918] Hardware name: F3JC > kernel: [ 1210.999921] NETDEV WATCHDOG: eth0 (r8169): transmit queue 0 timed > out > kernel: [ 1210.999924] Modules linked in: pktgen ppp_async crc_ccitt ipv6 > ppp_generic slhc asus_laptop sparse_keymap snd_hwdep sdhci_pci sdhci > snd_hda_codec_si3054 mmc_core led_class snd_hda_codec_realtek snd_hda_intel > snd_hda_codec > snd_pcm rng_core snd_timer snd_page_alloc psmouse snd evdev soundcore sg > r8169 serio_raw mii i2c_i801 usbhid hid uhci_hcd sr_mod ehci_hcd cdrom > usbcore sd_mod ata_piix > kernel: [ 1210.999983] Pid: 3819, comm: kpktgend_0 Not tainted > 2.6.34-rc6-dbg-git5 #109 > kernel: [ 1210.999987] Call Trace: > kernel: [ 1210.999994] [<c102e1c6>] warn_slowpath_common+0x65/0x7c > kernel: [ 1210.999999] [<c12690a5>] ? dev_watchdog+0xc1/0x129 > kernel: [ 1211.000005] [<c102e211>] warn_slowpath_fmt+0x24/0x27 > kernel: [ 1211.000010] [<c12690a5>] dev_watchdog+0xc1/0x129 > kernel: [ 1211.000017] [<c1040039>] ? __kfifo_from_user_generic+0x18/0x5c > kernel: [ 1211.000024] [<c1036af7>] ? run_timer_softirq+0x136/0x203 > kernel: [ 1211.000029] [<c1036b50>] run_timer_softirq+0x18f/0x203 > kernel: [ 1211.000034] [<c1036af7>] ? run_timer_softirq+0x136/0x203 > kernel: [ 1211.000039] [<c1268fe4>] ? dev_watchdog+0x0/0x129 > kernel: [ 1211.000046] [<c1032a86>] __do_softirq+0x88/0x10c > kernel: [ 1211.000051] [<c1032b39>] do_softirq+0x2f/0x47 > kernel: [ 1211.000059] [<f80a3488>] ? pktgen_xmit+0xd3e/0xe0b [pktgen] > kernel: [ 1211.000064] [<c1032d1c>] _local_bh_enable_ip+0x8b/0xb3 > kernel: [ 1211.000070] [<c1032d4c>] local_bh_enable_ip+0x8/0xa > kernel: [ 1211.000076] [<c12c4858>] _raw_spin_unlock_bh+0x2f/0x32 > kernel: [ 1211.000082] [<f80a3488>] pktgen_xmit+0xd3e/0xe0b [pktgen] > kernel: [ 1211.000088] [<c1044654>] ? sched_clock_cpu+0x121/0x131 > kernel: [ 1211.000098] [<fd1e75e9>] ? rtl8169_start_xmit+0x0/0x2f9 [r8169] > kernel: [ 1211.000104] [<c1044691>] ? cpu_clock+0x2d/0x4e > kernel: [ 1211.000111] [<c104d20f>] ? put_lock_stats+0xd/0x21 > kernel: [ 1211.000116] [<c104d2d8>] ? lock_release_holdtime+0xb5/0xba > kernel: [ 1211.000122] [<f80a3793>] ? pktgen_thread_worker+0xd9/0x5e8 > [pktgen] > kernel: [ 1211.000129] [<f80a382a>] pktgen_thread_worker+0x170/0x5e8 > [pktgen] > kernel: [ 1211.000135] [<c103fa5e>] ? autoremove_wake_function+0x0/0x2f > kernel: [ 1211.000141] [<c103fa5e>] ? autoremove_wake_function+0x0/0x2f > kernel: [ 1211.000147] [<f80a36ba>] ? pktgen_thread_worker+0x0/0x5e8 > [pktgen] > kernel: [ 1211.000152] [<c103f782>] kthread+0x6a/0x6f > kernel: [ 1211.000158] [<c103f718>] ? kthread+0x0/0x6f > kernel: [ 1211.000163] [<c1002dc2>] kernel_thread_helper+0x6/0x10 > kernel: [ 1211.000167] ---[ end trace adf7c4da01cca262 ]--- > kernel: [ 1211.013760] r8169 0000:02:00.0: eth0: link up
Running a stock 2.6.34 kernel and by disabling Power Management support from the kernel, this issue is NOT present. It is present the moment that I turn on Power Management on the same kernel. The issue does not seem to be an IRQ issue which I first thought, because the interrupts are exactly the same so noapic and acpi=noirq does not make a difference. The BIOS has Plug-n-Play but turning this on or off in the BIOS does not make a difference. So my guess is that there is something in the configuration of the r8169 when it is part of the chipset and APCI is configuring the board. I am running an Athlon II X2 240 on a 880 chipset. The r8169 is on the motherboard and not in a PCI slot. So for me, turning off Power Management made this problem disappear.
On Monday, June 14, 2010, Sergey Senozhatsky wrote: > On (06/13/10 16:48), Rafael J. Wysocki wrote: > > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=15704 > > Subject : [r8169] WARNING: at net/sched/sch_generic.c > > Submitter : Sergey Senozhatsky <sergey.senozhatsky@gmail.com> > > Date : 2010-03-31 10:21 (75 days old) Message-ID : 20100331102142.GA3294@swordfish.minsk.epam.com > > References : http://marc.info/?l=linux-kernel&m=127003090406108&w=2 > > > > > > > Hello, > > .35-rc3 > > kernel: [ 109.024175] pktgen 2.73: Packet Generator for packet performance > testing. > kernel: [ 144.360001] ------------[ cut here ]------------ > kernel: [ 144.360012] WARNING: at net/sched/sch_generic.c:258 > dev_watchdog+0xc1/0x129() > kernel: [ 144.360016] Hardware name: F3JC > kernel: [ 144.360019] NETDEV WATCHDOG: eth0 (r8169): transmit queue 0 timed > out > kernel: [ 144.360022] Modules linked in: pktgen snd_hwdep > snd_hda_codec_si3054 snd_hda_codec_realtek snd_hda_intel asus_laptop > sparse_keymap sdhci_pci sdhci mmc_core led_class snd_hda_codec i2c_i801 > rng_core snd_pcm snd_timer > snd_page_alloc snd soundcore psmouse sg serio_raw evdev r8169 mii usbhid hid > uhci_hcd ehci_hcd sr_mod usbcore cdrom sd_mod ata_piix > kernel: [ 144.360073] Pid: 3424, comm: kpktgend_0 Not tainted > 2.6.35-rc3-dbg-00106-ga75e02b #2 > kernel: [ 144.360076] Call Trace: > kernel: [ 144.360085] [<c102dc0a>] warn_slowpath_common+0x65/0x7a > kernel: [ 144.360090] [<c1266683>] ? dev_watchdog+0xc1/0x129 > kernel: [ 144.360095] [<c102dc83>] warn_slowpath_fmt+0x26/0x2a > kernel: [ 144.360100] [<c1266683>] dev_watchdog+0xc1/0x129 > kernel: [ 144.360107] [<c10365af>] ? run_timer_softirq+0x136/0x22b > kernel: [ 144.360113] [<c1036608>] run_timer_softirq+0x18f/0x22b > kernel: [ 144.360118] [<c10365af>] ? run_timer_softirq+0x136/0x22b > kernel: [ 144.360123] [<c12665c2>] ? dev_watchdog+0x0/0x129 > kernel: [ 144.360129] [<c10322d2>] __do_softirq+0x79/0xec > kernel: [ 144.360135] [<c103236f>] do_softirq+0x2a/0x2f > kernel: [ 144.360140] [<c1032519>] _local_bh_enable_ip+0x65/0x86 > kernel: [ 144.360145] [<c1032542>] local_bh_enable_ip+0x8/0xa > kernel: [ 144.360151] [<c12c354b>] _raw_spin_unlock_bh+0x2f/0x32 > kernel: [ 144.360158] [<f808e5c7>] pktgen_xmit+0xda2/0xe6f [pktgen] > kernel: [ 144.360168] [<fd1d57b8>] ? rtl8169_start_xmit+0x0/0x307 [r8169] > kernel: [ 144.360174] [<c1002d70>] ? common_interrupt+0x30/0x38 > kernel: [ 144.360180] [<c104c65d>] ? __lock_acquire+0x1d6/0x3ca > kernel: [ 144.360187] [<f808e88f>] ? pktgen_thread_worker+0x96/0x5c3 > [pktgen] > kernel: [ 144.360193] [<c1028a2f>] ? get_parent_ip+0xb/0x31 > kernel: [ 144.360198] [<c1028ad1>] ? sub_preempt_count+0x7c/0x89 > kernel: [ 144.360205] [<f808e969>] pktgen_thread_worker+0x170/0x5c3 > [pktgen] > kernel: [ 144.360209] [<c12c1e13>] ? schedule+0x535/0x545 > kernel: [ 144.360216] [<c103f1d2>] ? autoremove_wake_function+0x0/0x2f > kernel: [ 144.360221] [<c103f1d2>] ? autoremove_wake_function+0x0/0x2f > kernel: [ 144.360227] [<f808e7f9>] ? pktgen_thread_worker+0x0/0x5c3 > [pktgen] > kernel: [ 144.360232] [<c103eee2>] kthread+0x6a/0x6f > kernel: [ 144.360238] [<c103ee78>] ? kthread+0x0/0x6f > kernel: [ 144.360243] [<c1002d7e>] kernel_thread_helper+0x6/0x10 > kernel: [ 144.360247] ---[ end trace a692664bf7dac635 ]---
On Saturday, July 10, 2010, Sergey Senozhatsky wrote: > On (07/10/10 02:33), Rafael J. Wysocki wrote: > > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=15704 > > Subject : [r8169] WARNING: at net/sched/sch_generic.c > > Submitter : Sergey Senozhatsky <sergey.senozhatsky@gmail.com> > > Date : 2010-03-31 10:21 (101 days old) > > > > > > Hello, > > [ 3564.850086] pktgen 2.73: Packet Generator for packet performance testing. > [ 3580.993027] ------------[ cut here ]------------ > [ 3580.993039] WARNING: at net/sched/sch_generic.c:258 > dev_watchdog+0xc1/0x129() > [ 3580.993043] Hardware name: F3JC > [ 3580.993046] NETDEV WATCHDOG: eth0 (r8169): transmit queue 0 timed out > [ 3580.993049] Modules linked in: pktgen ppp_async crc_ccitt ipv6 ppp_generic > slhc usb_storage snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq > snd_seq_device snd_hwdep snd_hda_codec_si3054 snd_pcm_oss snd_mixer_oss > snd_hda_codec_realtek sdhci_pci sdhci mmc_core asus_laptop sparse_keymap > led_class snd_hda_intel snd_hda_codec snd_pcm psmouse rng_core i2c_i801 > snd_timer snd_page_alloc sg snd soundcore serio_raw evdev r8169 mii usbhid > hid uhci_hcd > ehci_hcd sr_mod usbcore cdrom sd_mod ata_piix > [ 3580.993120] Pid: 0, comm: swapper Not tainted > 2.6.35-rc4-dbg-git4-00247-g5bbf43d-dirty #72 > [ 3580.993124] Call Trace: > [ 3580.993132] [<c102e3a6>] warn_slowpath_common+0x65/0x7a > [ 3580.993137] [<c12694af>] ? dev_watchdog+0xc1/0x129 > [ 3580.993143] [<c102e41f>] warn_slowpath_fmt+0x26/0x2a > [ 3580.993148] [<c12694af>] dev_watchdog+0xc1/0x129 > [ 3580.993155] [<c10370ab>] ? run_timer_softirq+0x136/0x22b > [ 3580.993160] [<c1037104>] run_timer_softirq+0x18f/0x22b > [ 3580.993165] [<c10370ab>] ? run_timer_softirq+0x136/0x22b > [ 3580.993171] [<c12693ee>] ? dev_watchdog+0x0/0x129 > [ 3580.993177] [<c1032c2d>] __do_softirq+0x88/0x10c > [ 3580.993183] [<c1032ce0>] do_softirq+0x2f/0x47 > [ 3580.993188] [<c1032f9d>] irq_exit+0x38/0x75 > [ 3580.993193] [<c10159b8>] smp_apic_timer_interrupt+0x5f/0x6d > [ 3580.993200] [<c12c732a>] apic_timer_interrupt+0x36/0x3c > [ 3580.993207] [<c104007b>] ? __kfifo_peek_n+0x1b/0x2b > [ 3580.993213] [<c11dd5f6>] ? acpi_idle_enter_simple+0x117/0x14b > [ 3580.993220] [<c12468e9>] cpuidle_idle_call+0x6a/0xa0 > [ 3580.993225] [<c100170d>] cpu_idle+0x89/0xbe > [ 3580.993232] [<c12c1e6d>] start_secondary+0x1f4/0x1fb > [ 3580.993237] ---[ end trace 0e0b0610391f3050 ]--- > [ 3581.009527] r8169 0000:02:00.0: eth0: link up
Created attachment 27901 [details] Tx queue race
Patched it over my two servers with this chipset. I made some tests with pktgen with the script, Sergey had used. So far without errors, but as I turned vnstat on the receiving host on, it had not noticed heavy incoming traffic. Maybe I'm doing something wrong with pktgen? After that I changed the packet size to 2048. Same result, nothing happened on the receiver machine (vnstat had not recognized heavy traffic). I will test it under real conditions later. Regards, Alex Here the script I used to send packets to my victim machine: #! /bin/sh #modprobe pktgen function pgset() { local result echo $1 > $PGDEV result=`cat $PGDEV | fgrep "Result: OK:"` if [ "$result" = "" ]; then cat $PGDEV | fgrep Result: fi } function pg() { echo inject > $PGDEV cat $PGDEV } # Config Start Here ----------------------------------------------------------- # thread config # Each CPU has own thread. Two CPU exammple. We add eth1, eth2 respectivly. PGDEV=/proc/net/pktgen/kpktgend_0 echo "Removing all devices" pgset "rem_device_all" echo "Adding eth0" pgset "add_device eth0" echo "Setting max_before_softirq 10000" pgset "max_before_softirq 10000" # device config # delay 0 means maximum speed. CLONE_SKB="clone_skb 1000000" # NIC adds 4 bytes CRC PKT_SIZE="pkt_size 42" CLONE_SKB="clone_skb 4000000" # COUNT 0 means forever #COUNT="count 0" COUNT="count 100000000" DELAY="delay 0" PGDEV=/proc/net/pktgen/eth0 echo "Configuring $PGDEV" pgset "$COUNT" pgset "$CLONE_SKB" pgset "$PKT_SIZE" pgset "$DELAY" pgset "dst xx.xx.xxx.xxx" pgset "dst_mac xx:xx:xx:xx:xx:xx" # Time to run PGDEV=/proc/net/pktgen/pgctrl echo "Running... ctrl^C to stop" pgset "start" echo "Done" # Result can be vieved in /proc/net/pktgen/eth1
(In reply to comment #8) > I will test it under real conditions later. > > Regards, > Alex Did you have a chance to test it?
(In reply to comment #9) > (In reply to comment #8) > > I will test it under real conditions later. > > > > Regards, > > Alex > > Did you have a chance to test it? Yes I had a chance to test it. Under real conditions the patch does NOT help at all. My machines crashed approx every two weeks. The only solution is to use the newer driver. All kernel (including the newest one) are using the same outdated one-for-all driver. You have to implement the newest realtek driver to your kernel (sourcecode can be found here: http://www.realtek.com.tw/downloads/downloadsView.aspx?Langid=1&PNid=5&PFid=5&Level=5&Conn=4&DownTypeID=3&GetDown=false#RTL8111B/RTL8168B/RTL8111/RTL8168<br>RTL8111C/RTL8111CP/RTL8111D(L)<br>RTL8168C/RTL8111DP ). If the link isnt working, just search on the realtek website for "8168". So as a conclusion, only the newest driver from realtek solved this problem. Hope this was helpful.
Uhm..yes. In a way. Thx for the update.
I have the same kind of messages with 2.6.38: [ 5721.610018] WARNING: at /build/buildd-linux-2.6_2.6.38-1-amd64-t80ZCo/linux-2.6-2.6.38/debian/build/source_amd64_none/net/sched/sch_generic.c:256 dev_watchdog+0x104/0x1ad() [ 5721.610023] Hardware name: Precision M4500 [ 5721.610027] NETDEV WATCHDOG: hso0 (hso): transmit queue 0 timed out [ 5721.610029] Modules linked in: tun usb_storage uas hso acpi_cpufreq mperf cpufreq_userspace cpufreq_stats cpufreq_conservative cpufreq_powersave binfmt_misc fuse nls_utf8 nls_cp437 vfat fat loop firewire_sbp2 snd_hda_codec_hdmi snd_hda_codec_idt arc4 ecb snd_hda_intel snd_hda_codec snd_hwdep pcmcia iwlagn dell_wmi snd_seq snd_pcm snd_seq_device i7core_edac iwlcore snd_timer btusb nvidia(P) bluetooth sparse_keymap parport_pc edac_core mac80211 parport snd processor tpm_tis yenta_socket soundcore i2c_i801 battery wmi button ac video evdev pcmcia_rsrc dell_laptop power_supply tpm i2c_core tpm_bios snd_page_alloc psmouse cfg80211 pcmcia_core dcdbas pcspkr serio_raw rfkill thermal_sys ext3 jbd mbcache sg sd_mod sr_mod cdrom crc_t10dif ahci libahci libata sdhci_pci firewire_ohci sdhci mmc_core scsi_mod ehci_hcd firewire_core usbcore crc_itu_t e1000e nls_base [last unloaded: scsi_wait_scan] [ 5721.610120] Pid: 0, comm: kworker/0:0 Tainted: P O 2.6.38-1-amd64 #1 [ 5721.610124] Call Trace: [ 5721.610126] <IRQ> [<ffffffff81046cf8>] ? warn_slowpath_common+0x78/0x8c [ 5721.610138] [<ffffffff81046dab>] ? warn_slowpath_fmt+0x45/0x4a [ 5721.610144] [<ffffffff81284acc>] ? netif_tx_lock+0x43/0x74 [ 5721.610149] [<ffffffff81284c01>] ? dev_watchdog+0x104/0x1ad [ 5721.610155] [<ffffffff8105b648>] ? __queue_work+0x30c/0x334 [ 5721.610162] [<ffffffff8119d39d>] ? timerqueue_add+0x80/0x9b [ 5721.610170] [<ffffffff8100f502>] ? read_tsc+0x5/0x16 [ 5721.610175] [<ffffffff81067758>] ? ktime_get+0x5f/0xb8 [ 5721.610182] [<ffffffff81052d40>] ? run_timer_softirq+0x1c2/0x28e [ 5721.610187] [<ffffffff81284afd>] ? dev_watchdog+0x0/0x1ad [ 5721.610193] [<ffffffff8104c6d7>] ? __do_softirq+0xc3/0x1a0 [ 5721.610200] [<ffffffff8109096f>] ? handle_IRQ_event+0x4c/0x10a [ 5721.610205] [<ffffffff8100a85c>] ? call_softirq+0x1c/0x30 [ 5721.610210] [<ffffffff8100be03>] ? do_softirq+0x3f/0x79 [ 5721.610215] [<ffffffff8104c579>] ? irq_exit+0x36/0x7b [ 5721.610219] [<ffffffff8100b55d>] ? do_IRQ+0x94/0xaa [ 5721.610225] [<ffffffff81322b53>] ? ret_from_intr+0x0/0x15 [ 5721.610228] <EOI> [<ffffffffa01c3fc7>] ? acpi_idle_enter_bm+0x259/0x291 [processor] [ 5721.610245] [<ffffffffa01c3fc0>] ? acpi_idle_enter_bm+0x252/0x291 [processor] [ 5721.610253] [<ffffffff81255b17>] ? cpuidle_idle_call+0x11f/0x1cc [ 5721.610259] [<ffffffff81008d93>] ? cpu_idle+0xab/0xe1 [ 5721.610265] [<ffffffff8131bcbf>] ? start_secondary+0x1bf/0x1c5
*** Bug 43342 has been marked as a duplicate of this bug. ***
When I do a transfer 2 GB or bigger files to a linux samba share on a gigabit connection, my ethernet network keeps failing within a minute. I seem to have ran into the same problem with getting a message transmit queue 0 timed out when my IOMMU is off in BIOS, or getting a message IO_PAGE_FAULT with IOMMU turned on in the BIOS. My realtek ethernet stops working when the message shows, until I roboot or do a rmmod r8169, modprobe r8169. I looked at kernel driver r8169.c source code and compare it to realtek r8168 source code, found that by changing the numbers to 1024 for r8169.c linux kernel driver fixes my problems. define NUM_TX_DESC 1024 define NUM_RX_DESC 1024 Now the problem might come with other hardware needing a different number. The realtek source code of r8169 (gigabit PCI) have it both set to 256, and r8168 (gigabit PCI-Express) have it both set to 1024. I have RTL8168 hardware. uname -a Linux debian986 3.10.1 #2 SMP Sun Jul 21 21:51:20 CDT 2013 x86_64 GNU/Linux lspci -nn 02:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 09) dmesg | grep RTL r8169 0000:02:00.0 eth4: RTL8168f/8111f at 0xffffc90000c1c000, 60:a4:4c:xx:xx:xx, XID 08000880 IRQ 74
I just experienced this in RHEL7.2 on a Lenovo ThinkPad W520 laptop with an Intel NIC: [88877.066010] WARNING: at net/sched/sch_generic.c:259 dev_watchdog+0x270/0x280() [36/1968] [88877.066018] NETDEV WATCHDOG: enp0s25 (e1000e): transmit queue 0 timed out [88877.066023] Modules linked in: cdc_acm hidp rfcomm fuse cmac rmd160 crypto_null ip_vti ip_tunnel af_key ah6 ah4 esp6 esp4 xfrm4_mode_beet xfrm4_tunnel tunnel4 xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_transport xfrm6_mode_ro xfr m6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm6_tunnel tunnel6 xfrm_ipcomp sha1_ssse3 xt_set salsa20_x86_64 xt_multiport ip_set_hash_ip ip_set nfnetlink camellia_generic camellia_aesni_avx_x86_64 camellia_x86_64 xt_CHECKSUM cast6_avx_x 86_64 cast6_generic cast5_avx_x86_64 cast5_generic cast_common deflate zlib_deflate cts tun gcm serpent_avx_x86_64 serpent_sse2_x86_64 serpent_generic blowfish_generic blowfish_x86_64 blowfish_common twofish_generic twofish_avx_x86_64 twof ish_x86_64_3way xts twofish_x86_64 twofish_common xcbc sha256_ssse3 sha512_ssse3 sha512_generic [88877.066096] des_generic tpm_rng timeriomem_rng ccm ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_ftp ip6t_rpfilter ip6t_REJECT ipt_REJECT xt_conntrack ebtable_nat ebtable_broute bridg e stp llc ebtable_filter ebtables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip6table_filter ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntr ack iptable_mangle iptable_security iptable_raw iptable_filter bnep dm_crypt drbg ansi_cprng snd_hda_codec_hdmi ext4 mbcache jbd2 intel_powerclamp coretemp iTCO_wdt iTCO_vendor_support intel_rapl kvm_intel kvm arc4 crc32_pclmul crc32c_inte l ghash_clmulni_intel iwldvm aesni_intel lrw gf128mul glue_helper ablk_helper cryptd mac80211 [88877.066126] snd_hda_codec_conexant snd_hda_codec_generic btusb uvcvideo videobuf2_vmalloc snd_hda_intel videobuf2_memops videobuf2_core snd_hda_controller bluetooth videodev snd_hda_codec pcspkr i2c_i801 sg sdhci_pci snd_hwdep iwlwifi snd_seq sdhci r8169 snd_seq_device cfg80211 mmc_core lpc_ich snd_pcm mfd_core mii mei_me mei thinkpad_acpi snd_timer shpchp snd soundcore rfkill sch_fq_codel binfmt_misc virtio_blk nfsd nfs_acl lockd auth_rpcgss sunrpc ip_tables xfs libcrc 32c sd_mod crc_t10dif crct10dif_generic nvidia(POF) ahci libahci crct10dif_pclmul libata crct10dif_common serio_raw firewire_ohci drm e1000e firewire_core crc_itu_t wmi video dm_mirror dm_region_hash dm_log dm_mod virtio_pci virtio_net vir tio_rng virtio_ring virtio igb dca ptp pps_core i2c_algo_bit i2c_core [88877.066165] CPU: 0 PID: 0 Comm: swapper/0 Tainted: PF O-------------- 3.10.0-229.14.1.el7.x86_64 #1 [88877.066166] Hardware name: LENOVO 42762QU/42762QU, BIOS 8BET62WW (1.42 ) 07/26/2013 [88877.066168] ffff88043dc03d88 ec97c01d2b64c953 ffff88043dc03d40 ffffffff81604516 [88877.066171] ffff88043dc03d78 ffffffff8106e29b 0000000000000000 ffff8800b7728000 [88877.066174] ffff88041e84c480 0000000000000001 0000000000000000 ffff88043dc03de0 [88877.066176] Call Trace: [88877.066178] <IRQ> [<ffffffff81604516>] dump_stack+0x19/0x1b [88877.066187] [<ffffffff8106e29b>] warn_slowpath_common+0x6b/0xb0 [88877.066189] [<ffffffff8106e33c>] warn_slowpath_fmt+0x5c/0x80 [88877.066196] [<ffffffff8151d790>] dev_watchdog+0x270/0x280 [88877.066198] [<ffffffff8151d520>] ? dev_graft_qdisc+0x80/0x80 [88877.066202] [<ffffffff8107df66>] call_timer_fn+0x36/0x110 [88877.066204] [<ffffffff8151d520>] ? dev_graft_qdisc+0x80/0x80 [88877.066207] [<ffffffff8107fddf>] run_timer_softirq+0x21f/0x320 [88877.066209] [<ffffffff81077b3f>] __do_softirq+0xef/0x280 [88877.066212] [<ffffffff81615c9c>] call_softirq+0x1c/0x30 [88877.066216] [<ffffffff81015d95>] do_softirq+0x65/0xa0 [88877.066218] [<ffffffff81077ed5>] irq_exit+0x115/0x120 [88877.066221] [<ffffffff81616915>] smp_apic_timer_interrupt+0x45/0x60 [88877.066224] [<ffffffff81614fdd>] apic_timer_interrupt+0x6d/0x80 [88877.066233] <EOI> [<ffffffff814aa1bf>] ? cpuidle_enter_state+0x4f/0xc0 [88877.066239] [<ffffffff814aa2f5>] cpuidle_idle_call+0xc5/0x200 [88877.066242] [<ffffffff8101d14e>] arch_cpu_idle+0xe/0x30 [88877.066245] [<ffffffff810c6921>] cpu_startup_entry+0xf1/0x290 [88877.066249] [<ffffffff815f2ac7>] rest_init+0x77/0x80 [88877.066252] [<ffffffff81a45057>] start_kernel+0x429/0x44a [88877.066256] [<ffffffff81a44a37>] ? repair_env_string+0x5c/0x5c [88877.066259] [<ffffffff81a44120>] ? early_idt_handlers+0x120/0x120 [88877.066261] [<ffffffff81a445ee>] x86_64_start_reservations+0x2a/0x2c [88877.066264] [<ffffffff81a44742>] x86_64_start_kernel+0x152/0x175 [88877.066266] ---[ end trace cf565c69e9b62b60 ]---