Bug 9305

Summary: BUG: unable to handle kernel paging request at virtual address 26121228/kswapd0[231] exited with preempt_count 1
Product: Memory Management Reporter: Krzysztof Oledzki (ole)
Component: OtherAssignee: Andrew Morton (akpm)
Status: CLOSED UNREPRODUCIBLE    
Severity: high CC: clameter, ole, randy.dunlap
Priority: P1    
Hardware: All   
OS: Linux   
Kernel Version: 2.6.22.6 Subsystem:
Regression: --- Bisected commit-id:

Description Krzysztof Oledzki 2007-11-04 06:21:34 UTC
Most recent kernel where this bug did not occur: unknown
Distribution: slackware
Hardware Environment:
# lspci -vn
00:00.0 0600: 8086:3590 (rev 09)
        Subsystem: 1028:019a
        Flags: bus master, fast devsel, latency 0
        Capabilities: [40] Vendor Specific Information

00:02.0 0604: 8086:3595 (rev 09)
        Flags: bus master, fast devsel, latency 0
        Bus: primary=00, secondary=01, subordinate=03, sec-latency=0
        I/O behind bridge: 0000e000-0000efff
        Memory behind bridge: fe700000-feafffff
        Capabilities: [50] Power Management version 2
        Capabilities: [58] Message Signalled Interrupts: 64bit- Queue=0/1 Enable-
        Capabilities: [64] Express Root Port (Slot-) IRQ 0
        Capabilities: [100] Advanced Error Reporting

00:1d.0 0c03: 8086:24d2 (rev 02)
        Subsystem: 1028:019a
        Flags: bus master, medium devsel, latency 0, IRQ 16
        I/O ports at cce0 [size=32]

00:1d.1 0c03: 8086:24d4 (rev 02)
        Subsystem: 1028:019a
        Flags: bus master, medium devsel, latency 0, IRQ 22
        I/O ports at ccc0 [size=32]

00:1d.7 0c03: 8086:24dd (rev 02) (prog-if 20)
        Subsystem: 1028:019a
        Flags: bus master, medium devsel, latency 0, IRQ 21
        Memory at feb00000 (32-bit, non-prefetchable) [size=1K]
        Capabilities: [50] Power Management version 2
        Capabilities: [58] Debug port

00:1e.0 0604: 8086:244e (rev c2)
        Flags: bus master, fast devsel, latency 0
        Bus: primary=00, secondary=04, subordinate=04, sec-latency=32
        I/O behind bridge: 0000d000-0000dfff
        Memory behind bridge: fe500000-fe6fffff
        Prefetchable memory behind bridge: f0000000-f7ffffff

00:1f.0 0601: 8086:24d0 (rev 02)
        Flags: bus master, medium devsel, latency 0

00:1f.1 0101: 8086:24db (rev 02) (prog-if 8a)
        Subsystem: 1028:019a
        Flags: bus master, medium devsel, latency 0, IRQ 20
        I/O ports at 01f0 [size=8]
        I/O ports at 03f4 [size=1]
        I/O ports at 0170 [size=8]
        I/O ports at 0374 [size=1]
        I/O ports at fc00 [size=16]
        Memory at 80000000 (32-bit, non-prefetchable) [size=1K]

00:1f.2 0101: 8086:24d1 (rev 02) (prog-if 8f)
        Subsystem: 1028:019a
        Flags: bus master, 66MHz, medium devsel, latency 0, IRQ 20
        I/O ports at ccb8 [size=8]
        I/O ports at ccb0 [size=4]
        I/O ports at cca0 [size=8]
        I/O ports at cc98 [size=4]
        I/O ports at cc80 [size=16]

01:00.0 0604: 8086:0329 (rev 09)
        Flags: bus master, fast devsel, latency 0
        Bus: primary=01, secondary=02, subordinate=02, sec-latency=32
        I/O behind bridge: 0000e000-0000efff
        Memory behind bridge: fe900000-feafffff
        Capabilities: [44] Express PCI/PCI-X Bridge IRQ 0
        Capabilities: [5c] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-
        Capabilities: [6c] Power Management version 2
        Capabilities: [d8] PCI-X bridge device
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [300] Power Budgeting

01:00.2 0604: 8086:032a (rev 09)
        Flags: bus master, fast devsel, latency 0
        Bus: primary=01, secondary=03, subordinate=03, sec-latency=64
        Capabilities: [44] Express PCI/PCI-X Bridge IRQ 0
        Capabilities: [5c] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-
        Capabilities: [6c] Power Management version 2
        Capabilities: [d8] PCI-X bridge device
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [300] Power Budgeting

02:04.0 0200: 8086:1076 (rev 05)
        Subsystem: 1028:019a
        Flags: bus master, 66MHz, medium devsel, latency 32, IRQ 18
        Memory at fe9e0000 (32-bit, non-prefetchable) [size=128K]
        I/O ports at ecc0 [size=64]
        Capabilities: [dc] Power Management version 2
        Capabilities: [e4] PCI-X non-bridge device

04:03.0 0200: 8086:1076 (rev 05)
        Subsystem: 1028:019a
        Flags: bus master, 66MHz, medium devsel, latency 32, IRQ 19
        Memory at fe5e0000 (32-bit, non-prefetchable) [size=128K]
        I/O ports at dcc0 [size=64]
        Capabilities: [dc] Power Management version 2
        Capabilities: [e4] PCI-X non-bridge device

04:0d.0 0300: 1002:5159
        Subsystem: 1028:019a
        Flags: bus master, VGA palette snoop, stepping, medium devsel, latency 32, IRQ 17
        Memory at f0000000 (32-bit, prefetchable) [size=128M]
        I/O ports at d800 [size=256]
        Memory at fe5d0000 (32-bit, non-prefetchable) [size=64K]
        [virtual] Expansion ROM at fe500000 [disabled] [size=128K]
        Capabilities: [50] Power Management version 2

Linux version 2.6.22.6 (root@fw1) (gcc version 3.4.6) #1 SMP PREEMPT Sat Sep 15 02:45:50 CEST 2007
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 00000000000a0000 (usable)
 BIOS-e820: 0000000000100000 - 000000007ffc0000 (usable)
 BIOS-e820: 000000007ffc0000 - 000000007ffcfc00 (ACPI data)
 BIOS-e820: 000000007ffcfc00 - 000000007ffff000 (reserved)
 BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
 BIOS-e820: 00000000fec00000 - 00000000fec90000 (reserved)
 BIOS-e820: 00000000fed00000 - 00000000fed00400 (reserved)
 BIOS-e820: 00000000fee00000 - 00000000fee10000 (reserved)
 BIOS-e820: 00000000ffb00000 - 0000000100000000 (reserved)
2047MB LOWMEM available.
found SMP MP-table at 000fe710
Entering add_active_range(0, 0, 524224) 0 entries of 256 used
Zone PFN ranges:
  DMA             0 ->     4096
  Normal       4096 ->   524224
early_node_map[1] active PFN ranges
    0:        0 ->   524224
On node 0 totalpages: 524224
  DMA zone: 32 pages used for memmap
  DMA zone: 0 pages reserved
  DMA zone: 4064 pages, LIFO batch:0
  Normal zone: 4063 pages used for memmap
  Normal zone: 516065 pages, LIFO batch:31
DMI 2.3 present.
ACPI: RSDP 000FD650, 0014 (r0 DELL  )
ACPI: RSDT 000FD664, 0038 (r1 DELL   PESC1425        1 MSFT  100000A)
ACPI: FACP 000FD6B0, 0074 (r1 DELL   PESC1425        1 MSFT  100000A)
ACPI: DSDT 7FFC0000, 174C (r1 DELL   PESC1425        1 MSFT  100000E)
ACPI: FACS 7FFCFC00, 0040
ACPI: APIC 000FD724, 009C (r1 DELL   PESC1425        1 MSFT  100000A)
ACPI: SPCR 000FD7C0, 0050 (r1 DELL   PESC1425        1 MSFT  100000A)
ACPI: HPET 000FD810, 0038 (r1 DELL   PESC1425        1 MSFT  100000A)
ACPI: MCFG 000FD848, 003C (r1 DELL   PESC1425        1 MSFT  100000A)
ACPI: PM-Timer IO Port: 0x808
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
Processor #0 15:4 APIC version 20
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x06] enabled)
Processor #6 15:4 APIC version 20
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x01] enabled)
Processor #1 15:4 APIC version 20
ACPI: LAPIC (acpi_id[0x04] lapic_id[0x07] enabled)
Processor #7 15:4 APIC version 20
ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])
ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
ACPI: IOAPIC (id[0x09] address[0xfec80000] gsi_base[32])
IOAPIC[1]: apic_id 9, version 32, address 0xfec80000, GSI 32-55
ACPI: IOAPIC (id[0x0a] address[0xfec80800] gsi_base[64])
IOAPIC[2]: apic_id 10, version 32, address 0xfec80800, GSI 64-87
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Enabling APIC mode:  Flat.  Using 3 I/O APICs
ACPI: HPET id: 0xffffffff base: 0xfed00000
Using ACPI (MADT) for SMP configuration information
Allocating PCI resources starting at 80000000 (gap: 7ffff000:60001000)
Built 1 zonelists.  Total pages: 520129
Kernel command line: auto BOOT_IMAGE=Linux-2.6.22.6 ro root=900 rootflags=data=journal nf_conntrack.hashsize=262144
mapped APIC to ffffd000 (fee00000)
mapped IOAPIC to ffffc000 (fec00000)
mapped IOAPIC to ffffb000 (fec80000)
mapped IOAPIC to ffffa000 (fec80800)
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 16384 bytes)
Detected 3200.525 MHz processor.
Console: colour VGA+ 80x30
Dentry cache hash table entries: 262144 (order: 8, 1048576 bytes)
Inode-cache hash table entries: 131072 (order: 7, 524288 bytes)
Memory: 2072892k/2096896k available (3525k kernel code, 23552k reserved, 1253k data, 232k init, 0k highmem)
virtual kernel memory layout:
    fixmap  : 0xfffb7000 - 0xfffff000   ( 288 kB)
    vmalloc : 0xf8800000 - 0xfffb5000   ( 119 MB)
    lowmem  : 0x78000000 - 0xf7fc0000   (2047 MB)
      .init : 0x785b3000 - 0x785ed000   ( 232 kB)
      .data : 0x784716c6 - 0x785aae00   (1253 kB)
      .text : 0x78100000 - 0x784716c6   (3525 kB)
Checking if this processor honours the WP bit even in supervisor mode... Ok.
hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
hpet0: 3 64-bit timers, 14318180 Hz
Calibrating delay using timer specific routine.. 6403.64 BogoMIPS (lpj=3201820)
Mount-cache hash table entries: 512
CPU: After generic identify, caps: bfebfbff 20100000 00000000 00000000 0000641d 00000000 00000000
monitor/mwait feature present.
using mwait in idle threads.
CPU: Trace cache: 12K uops, L1 D cache: 16K
CPU: L2 cache: 2048K
CPU: Physical Processor ID: 0
CPU: After all inits, caps: bfebfbff 20100000 00000000 0000b180 0000641d 00000000 00000000
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#0.
CPU0: Intel P4/Xeon Extended MCE MSRs (24) available
CPU0: Thermal monitoring enabled
Compat vDSO mapped to ffffe000.
Checking 'hlt' instruction... OK.
SMP alternatives: switching to UP code
ACPI: Core revision 20070126
CPU0: Intel(R) Xeon(TM) CPU 3.20GHz stepping 03
SMP alternatives: switching to SMP code
Booting processor 1/1 eip 3000
Initializing CPU#1
Calibrating delay using timer specific routine.. 6400.27 BogoMIPS (lpj=3200138)
CPU: After generic identify, caps: bfebfbff 20100000 00000000 00000000 0000641d 00000000 00000000
monitor/mwait feature present.
CPU: Trace cache: 12K uops, L1 D cache: 16K
CPU: L2 cache: 2048K
CPU: Physical Processor ID: 0
CPU: After all inits, caps: bfebfbff 20100000 00000000 0000b180 0000641d 00000000 00000000
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#1.
CPU1: Intel P4/Xeon Extended MCE MSRs (24) available
CPU1: Thermal monitoring enabled
CPU1: Intel(R) Xeon(TM) CPU 3.20GHz stepping 03
SMP alternatives: switching to SMP code
Booting processor 2/6 eip 3000
Initializing CPU#2
Calibrating delay using timer specific routine.. 6400.30 BogoMIPS (lpj=3200151)
CPU: After generic identify, caps: bfebfbff 20100000 00000000 00000000 0000641d 00000000 00000000
monitor/mwait feature present.
CPU: Trace cache: 12K uops, L1 D cache: 16K
CPU: L2 cache: 2048K
CPU: Physical Processor ID: 3
CPU: After all inits, caps: bfebfbff 20100000 00000000 0000b180 0000641d 00000000 00000000
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#2.
CPU2: Intel P4/Xeon Extended MCE MSRs (24) available
CPU2: Thermal monitoring enabled
CPU2: Intel(R) Xeon(TM) CPU 3.20GHz stepping 03
SMP alternatives: switching to SMP code
Booting processor 3/7 eip 3000
Initializing CPU#3
Calibrating delay using timer specific routine.. 6400.28 BogoMIPS (lpj=3200144)
CPU: After generic identify, caps: bfebfbff 20100000 00000000 00000000 0000641d 00000000 00000000
monitor/mwait feature present.
CPU: Trace cache: 12K uops, L1 D cache: 16K
CPU: L2 cache: 2048K
CPU: Physical Processor ID: 3
CPU: After all inits, caps: bfebfbff 20100000 00000000 0000b180 0000641d 00000000 00000000
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#3.
CPU3: Intel P4/Xeon Extended MCE MSRs (24) available
CPU3: Thermal monitoring enabled
CPU3: Intel(R) Xeon(TM) CPU 3.20GHz stepping 03
Total of 4 processors activated (25604.50 BogoMIPS).
ENABLING IO-APIC IRQs
..TIMER: vector=0x31 apic1=0 pin1=2 apic2=-1 pin2=-1
checking TSC synchronization [CPU#0 -> CPU#1]: passed.
checking TSC synchronization [CPU#0 -> CPU#2]: passed.
checking TSC synchronization [CPU#0 -> CPU#3]: passed.
Brought up 4 CPUs
migration_cost=33,1419
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: Found Intel Corporation E7520 Memory Controller Hub with MMCONFIG support.
PCI: Using MMCONFIG
Setting up standard PCI resources
ACPI: Interpreter enabled
ACPI: (supports S0 S4 S5)
ACPI: Using IOAPIC for interrupt routing
ACPI: PCI Root Bridge [PCI0] (0000:00)
PCI: Probing PCI hardware (bus 00)
PCI quirk: region 0800-087f claimed by ICH4 ACPI/GPIO/TCO
PCI quirk: region 0880-08bf claimed by ICH4 GPIO
PCI: PXH quirk detected, disabling MSI for SHPC device
PCI: PXH quirk detected, disabling MSI for SHPC device
PCI: Transparent bridge - 0000:00:1e.0
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PALO._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PALO.PXHB._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PALO.PXHA._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PICH._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 10 *11 12)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 10 *11 12)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 *5 6 7 10 11 12)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 7 *10 11 12)
ACPI: PCI Interrupt Link [LNKE] (IRQs *3 4 5 6 7 10 11 12)
ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 10 11 12) *0, disabled.
ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 10 11 12) *0, disabled.
ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 *6 7 10 11 12)
Linux Plug and Play Support v0.97 (c) Adam Belay
pnp: PnP ACPI init
ACPI: bus type pnp registered
pnp: PnP ACPI: found 12 devices
ACPI: ACPI bus type pnp unregistered
SCSI subsystem initialized
libata version 2.21 loaded.
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
PCI: Using ACPI for IRQ routing
PCI: If a device doesn't work, try "pci=routeirq".  If it helps, post a report
pnp: 00:08: ioport range 0x800-0x87f has been reserved
pnp: 00:08: ioport range 0x880-0x8bf has been reserved
pnp: 00:08: ioport range 0x8c0-0x8df has been reserved
Time: tsc clocksource has been installed.
Switched to high resolution mode on CPU 0
pnp: 00:08: ioport range 0x8e0-0x8e3 has been reserved
pnp: 00:08: ioport range 0xc00-0xc0f has been reserved
pnp: 00:08: ioport range 0xc10-0xc1f has been reserved
Switched to high resolution mode on CPU 1
pnp: 00:08: ioport range 0xca0-0xca7 has been reserved
pnp: 00:08: ioport range 0xca9-0xcab has been reserved
pnp: 00:0a: iomem range 0xe0000000-0xefffffff could not be reserved
Switched to high resolution mode on CPU 2
Switched to high resolution mode on CPU 3
PCI: Bridge: 0000:01:00.0
  IO window: e000-efff
  MEM window: fe900000-feafffff
  PREFETCH window: disabled.
PCI: Bridge: 0000:01:00.2
  IO window: disabled.
  MEM window: disabled.
  PREFETCH window: disabled.
PCI: Bridge: 0000:00:02.0
  IO window: e000-efff
  MEM window: fe700000-feafffff
  PREFETCH window: disabled.
PCI: Bridge: 0000:00:1e.0
  IO window: d000-dfff
  MEM window: fe500000-fe6fffff
  PREFETCH window: f0000000-f7ffffff
ACPI: PCI Interrupt 0000:00:02.0[A] -> GSI 16 (level, low) -> IRQ 16
PCI: Setting latency timer of device 0000:00:02.0 to 64
PCI: Setting latency timer of device 0000:01:00.0 to 64
PCI: Setting latency timer of device 0000:01:00.2 to 64
PCI: Setting latency timer of device 0000:00:1e.0 to 64
NET: Registered protocol family 2
IP route cache hash table entries: 65536 (order: 6, 262144 bytes)
TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 7, 786432 bytes)
TCP: Hash tables configured (established 262144 bind 65536)
TCP reno registered
Machine check exception polling timer started.
IA-32 Microcode Update Driver: v1.14a <tigran@aivazian.fsnet.co.uk>
audit: initializing netlink socket (disabled)
audit(1194177736.870:1): initialized
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered (default)
io scheduler cfq registered
Boot video device is 0000:04:0d.0
PCI: Setting latency timer of device 0000:00:02.0 to 64
Allocate Port Service[0000:00:02.0:pcie00]
Allocate Port Service[0000:00:02.0:pcie01]
Evaluate _OSC Set fails. Status = 0x0005
Evaluate _OSC Set fails. Status = 0x0005
aer_init: AER service init fails - Run ACPI _OSC fails
aer: probe of 0000:00:02.0:pcie01 failed with error 2
ACPI: PCI Interrupt 0000:04:0d.0[A] -> GSI 17 (level, low) -> IRQ 17
radeonfb: Found Intel x86 BIOS ROM Image
radeonfb: Retrieved PLL infos from BIOS
radeonfb: Reference=27.00 MHz (RefDiv=60) Memory=143.00 Mhz, System=143.00 MHz
radeonfb: PLL min 12000 max 35000
i2c-adapter i2c-1: unable to read EDID block.
i2c-adapter i2c-1: unable to read EDID block.
i2c-adapter i2c-1: unable to read EDID block.
i2c-adapter i2c-3: unable to read EDID block.
i2c-adapter i2c-3: unable to read EDID block.
i2c-adapter i2c-3: unable to read EDID block.
radeonfb: Monitor 1 type CRT found
radeonfb: EDID probed
radeonfb: Monitor 2 type no found
Console: switching to colour frame buffer device 128x48
radeonfb (0000:04:0d.0): ATI Radeon QY
input: Power Button (FF) as /class/input/input0
ACPI: Power Button (FF) [PWRF]
ACPI Exception (video-1476): AE_NOT_FOUND, Evaluating _DOD [20070126]
ACPI: Video Device [EVGA] (multi-head: no  rom: yes  post: no)
Real Time Clock Driver v1.12ac
hpet_resources: 0xfed00000 is busy
ipmi message handler version 39.1
ipmi device interface
IPMI System Interface driver.
ipmi_si: Trying SMBIOS-specified kcs state machine at i/o address 0xca8, slave address 0x20, irq 0
ipmi: Found new BMC (man_id: 0x0002a2,  prod_id: 0x0000, dev_id: 0x20)
 IPMI kcs interface initialized
IPMI Watchdog: driver initialized
Copyright (C) 2004 MontaVista Software - IPMI Powerdown via sys_reboot.
IPMI poweroff: ATCA Detect mfg 0x2A2 prod 0x0
IPMI poweroff: Found a chassis style poweroff function
Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing disabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
00:07: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
loop: module loaded
Intel(R) PRO/1000 Network Driver - version 7.3.20-k2-NAPI
Copyright (c) 1999-2006 Intel Corporation.
ACPI: PCI Interrupt 0000:02:04.0[A] -> GSI 32 (level, low) -> IRQ 18
e1000: 0000:02:04.0: e1000_probe: (PCI:66MHz:32-bit) 00:14:22:b0:c9:f9
e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
ACPI: PCI Interrupt 0000:04:03.0[A] -> GSI 20 (level, low) -> IRQ 19
e1000: 0000:04:03.0: e1000_probe: (PCI:33MHz:32-bit) 00:14:22:b0:c9:fa
e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connection
Ethernet Channel Bonding Driver: v3.1.3 (June 13, 2007)
bonding: Warning: either miimon or arp_interval and arp_ip_target module parameters must be specified, otherwise bonding will not detect link failures! see bonding.txt for details.
tun: Universal TUN/TAP device driver, 1.6
tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
ata_piix 0000:00:1f.1: version 2.11
PCI: Enabling device 0000:00:1f.1 (0005 -> 0007)
ACPI: PCI Interrupt 0000:00:1f.1[A] -> GSI 18 (level, low) -> IRQ 20
PCI: Setting latency timer of device 0000:00:1f.1 to 64
scsi0 : ata_piix
scsi1 : ata_piix
ata1: PATA max UDMA/133 cmd 0x000101f0 ctl 0x000103f6 bmdma 0x0001fc00 irq 14
ata2: PATA max UDMA/133 cmd 0x00010170 ctl 0x00010376 bmdma 0x0001fc08 irq 15
ata1.00: ATAPI: HL-DT-ST DVD-ROM GDR-8084N, 1.01, max UDMA/33
ata1.00: configured for UDMA/33
ata2: port disabled. ignoring.
scsi 0:0:0:0: CD-ROM            HL-DT-ST DVD-ROM GDR8084N 1.01 PQ: 0 ANSI: 5
sr0: scsi3-mmc drive: 24x/24x cd/rw xa/form2 cdda tray
Uniform CD-ROM driver Revision: 3.20
sr 0:0:0:0: Attached scsi CD-ROM sr0
sr 0:0:0:0: Attached scsi generic sg0 type 5
ata_piix 0000:00:1f.2: MAP [ P0 -- P1 -- ]
ACPI: PCI Interrupt 0000:00:1f.2[A] -> GSI 18 (level, low) -> IRQ 20
PCI: Setting latency timer of device 0000:00:1f.2 to 64
scsi2 : ata_piix
scsi3 : ata_piix
ata3: SATA max UDMA/133 cmd 0x0001ccb8 ctl 0x0001ccb2 bmdma 0x0001cc80 irq 20
ata4: SATA max UDMA/133 cmd 0x0001cca0 ctl 0x0001cc9a bmdma 0x0001cc88 irq 20
ata3.00: ATA-7: WDC WD800JD-75MSA1, 10.01E01, max UDMA/133
ata3.00: 156250000 sectors, multi 8: LBA48 NCQ (depth 0/32)
ata3.00: configured for UDMA/133
ata4.00: ATA-7: WDC WD800JD-75MSA1, 10.01E01, max UDMA/133
ata4.00: 156250000 sectors, multi 8: LBA48 NCQ (depth 0/32)
ata4.00: configured for UDMA/133
scsi 2:0:0:0: Direct-Access     ATA      WDC WD800JD-75MS 10.0 PQ: 0 ANSI: 5
sd 2:0:0:0: [sda] 156250000 512-byte hardware sectors (80000 MB)
sd 2:0:0:0: [sda] Write Protect is off
sd 2:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sd 2:0:0:0: [sda] 156250000 512-byte hardware sectors (80000 MB)
sd 2:0:0:0: [sda] Write Protect is off
sd 2:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sda: sda1 sda2 sda3 sda4 < sda5 sda6 sda7 sda8 sda9 >
sd 2:0:0:0: [sda] Attached SCSI disk
sd 2:0:0:0: Attached scsi generic sg1 type 0
scsi 3:0:0:0: Direct-Access     ATA      WDC WD800JD-75MS 10.0 PQ: 0 ANSI: 5
sd 3:0:0:0: [sdb] 156250000 512-byte hardware sectors (80000 MB)
sd 3:0:0:0: [sdb] Write Protect is off
sd 3:0:0:0: [sdb] Mode Sense: 00 3a 00 00
sd 3:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sd 3:0:0:0: [sdb] 156250000 512-byte hardware sectors (80000 MB)
sd 3:0:0:0: [sdb] Write Protect is off
sd 3:0:0:0: [sdb] Mode Sense: 00 3a 00 00
sd 3:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sdb: sdb1 sdb2 sdb3 sdb4 < sdb5 sdb6 sdb7 sdb8 sdb9 >
sd 3:0:0:0: [sdb] Attached SCSI disk
sd 3:0:0:0: Attached scsi generic sg2 type 0
ACPI: PCI Interrupt 0000:00:1d.7[D] -> GSI 23 (level, low) -> IRQ 21
PCI: Setting latency timer of device 0000:00:1d.7 to 64
ehci_hcd 0000:00:1d.7: EHCI Host Controller
ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 1
ehci_hcd 0000:00:1d.7: debug port 1
PCI: cache line size of 128 is not supported by device 0000:00:1d.7
ehci_hcd 0000:00:1d.7: irq 21, io mem 0xfeb00000
ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004
usb usb1: configuration #1 chosen from 1 choice
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 4 ports detected
USB Universal Host Controller Interface driver v3.0
ACPI: PCI Interrupt 0000:00:1d.0[A] -> GSI 16 (level, low) -> IRQ 16
PCI: Setting latency timer of device 0000:00:1d.0 to 64
uhci_hcd 0000:00:1d.0: UHCI Host Controller
uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2
uhci_hcd 0000:00:1d.0: irq 16, io base 0x0000cce0
usb usb2: configuration #1 chosen from 1 choice
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 2 ports detected
ACPI: PCI Interrupt 0000:00:1d.1[B] -> GSI 19 (level, low) -> IRQ 22
PCI: Setting latency timer of device 0000:00:1d.1 to 64
uhci_hcd 0000:00:1d.1: UHCI Host Controller
uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3
uhci_hcd 0000:00:1d.1: irq 22, io base 0x0000ccc0
usb usb3: configuration #1 chosen from 1 choice
hub 3-0:1.0: USB hub found
hub 3-0:1.0: 2 ports detected
usbcore: registered new interface driver usblp
drivers/usb/class/usblp.c: v0.13: USB Printer Device Class driver
PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
mice: PS/2 mouse device common for all mice
input: AT Translated Set 2 keyboard as /class/input/input1
input: PC Speaker as /class/input/input2
input: ImExPS/2 Generic Explorer Mouse as /class/input/input3
md: raid1 personality registered for level 1
raid6: int32x1    835 MB/s
raid6: int32x2    839 MB/s
raid6: int32x4    671 MB/s
raid6: int32x8    605 MB/s
raid6: mmxx1     1957 MB/s
raid6: mmxx2     1960 MB/s
raid6: sse1x1    1003 MB/s
raid6: sse1x2    1289 MB/s
raid6: sse2x1    2269 MB/s
raid6: sse2x2    2144 MB/s
raid6: using algorithm sse2x1 (2269 MB/s)
md: raid6 personality registered for level 6
md: raid5 personality registered for level 5
md: raid4 personality registered for level 4
raid5: automatically using best checksumming function: pIII_sse
   pIII_sse  :  3536.000 MB/sec
raid5: using function: pIII_sse (3536.000 MB/sec)
device-mapper: ioctl: 4.11.0-ioctl (2006-10-12) initialised: dm-devel@redhat.com
EDAC MC: Ver: 2.0.1 Sep 14 2007
Contact your BIOS vendor to see if the E752x error registers can be safely un-hidden
e752x_edac: probe of 0000:00:00.0 failed with error -12
dcdbas dcdbas: Dell Systems Management Base Driver (version 5.6.0-3.2)
usbcore: registered new interface driver usbhid
drivers/hid/usbhid/hid-core.c: v2.6:USB HID core driver
pktgen v2.68: Packet Generator for packet performance testing.
GACT probability on
Mirror/redirect action on
Simple TC action Loaded
netem: version 1.2
u32 classifier
    Performance counters on
    input device check on
    Actions configured
Netfilter messages via NETLINK v0.30.
nf_conntrack version 0.5.0 (262144 buckets, 2097152 max)
ctnetlink v0.93: registering with nfnetlink.
nf_conntrack_rtsp v0.6.21 loading
IPv4 over IPv4 tunneling driver
GRE over IPv4 tunneling driver
nf_nat_rtsp v0.6.21 loading
ip_tables: (C) 2000-2006 Netfilter Core Team
IP_TPROXY: Transparent proxy support initialized, version 4.0.0
IP_TPROXY: Copyright (c) 2002-2007 BalaBit IT Ltd.
ipt_time loading
IPP2P v0.8.1_rc1 loading
ClusterIP Version 0.8 loaded successfully
arp_tables: (C) 2002 David S. Miller
IPVS: Registered protocols (TCP, UDP)
IPVS: Connection hash table configured (size=65536, memory=512Kbytes)
IPVS: Each connection entry needs 124 bytes at least
IPVS: ipvs loaded.
IPVS: [rr] scheduler registered.
IPVS: [wrr] scheduler registered.
IPVS: [lc] scheduler registered.
IPVS: [wlc] scheduler registered.
IPVS: [lblc] scheduler registered.
IPVS: [lblcr] scheduler registered.
IPVS: [dh] scheduler registered.
IPVS: [sh] scheduler registered.
IPVS: [sed] scheduler registered.
IPVS: [nq] scheduler registered.
TCP bic registered
TCP cubic registered
TCP westwood registered
TCP highspeed registered
TCP hybla registered
TCP htcp registered
TCP vegas registered
TCP veno registered
TCP scalable registered
TCP lp registered
TCP yeah registered
TCP illinois registered
Initializing XFRM netlink socket
NET: Registered protocol family 1
NET: Registered protocol family 10
lo: Disabled Privacy Extensions
tunl0: Disabled Privacy Extensions
Mobile IPv6
ip6_tables: (C) 2000-2006 Netfilter Core Team
registering ipv6 ROUTE target
IPv6 over IPv4 tunneling driver
sit0: Disabled Privacy Extensions
NET: Registered protocol family 17
NET: Registered protocol family 15
802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
All bugs added by David S. Miller <davem@redhat.com>
Using IPI No-Shortcut mode
BIOS EDD facility v0.16 2004-Jun-25, 2 devices found



Software Environment:
Linux fw1 2.6.22.6 #1 SMP PREEMPT Sat Sep 15 02:45:50 CEST 2007 i686 pentium4 i386 GNU/Linux

Gnu C                  3.4.6
Gnu make               3.81
binutils               2.15.92.0.2
util-linux             2.12r
mount                  2.12r
module-init-tools      3.2.2
e2fsprogs              1.39
quota-tools            3.12.
PPP                    2.4.4b1
Linux C Library        2.3.6
Dynamic linker (ldd)   2.3.6
Linux C++ Library      6.0.3
Procps                 3.2.7
Net-tools              1.60
Kbd                    1.12
oprofile               0.9.1
Sh-utils               5.97

CONFIG_VMSPLIT_2G_OPT=y

Problem Description:
note: kswapd0[231] exited with preempt_count 1
BUG: unable to handle kernel paging request at virtual address 26121228
 printing eip:
781657ba
*pde = 00000000
Oops: 0002 [#1]
PREEMPT SMP
CPU:    0
EIP:    0060:[<781657ba>]    Not tainted VLI
EFLAGS: 00010082   (2.6.22.6 #1)
EIP is at free_block+0x6d/0xe4
eax: 26121224   ebx: 82dd9000   ecx: 82dd9038   edx: ca470000
esi: f7e7ed80   edi: 7a24a340   ebp: 00000017   esp: f7f47e40
ds: 007b   es: 007b   fs: 00d8  gs: 0000 ss: 0068
Process kswapd0 (pid: 231, ti=f7f46000 task=7a27f030 task.ti=f7f46000)
Stack: 00000000 0000001b 7a293598 7a252800 7a24a340 7a24a340 0000001b 781658a6
       00000000 f7e7ed80 7a293580 7a293580 00000246 7a24a340 a6a65038 78165b3b
       a6a650e4 a6a650e4 f7f47eb8 00000042 78177b93 a6a650ec 78177e88 93bdadbc
Call Trace:
 [<781658a6>] cache_flusharray+0x75/0xa5
 [<78165b3b>] kmem_cache_free+0x7a/0x8a
 [<78177b93>] destroy_inode+0x24/0x33
 [<78177e88>] dispose_list+0xa3/0xc6
 [<78178131>] prune_icache+0x172/0x184
 [<78178157>] shrink_icache_memory+0x14/0x2b
 [<78153c90>] shrink_slab+0x139/0x1b2
 [<78154f17>] balance_pgdat+0x253/0x380
 [<78155133>] kswapd+0xef/0xf1
 [<781365d0>] autoremove_wake_function+0x0/0x33
 [<781365d0>] autoremove_wake_function+0x0/0x33
 [<78155044>] kswapd+0x0/0xf1
 [<781361db>] kthread+0x34/0x55
 [<781361a7>] kthread+0x0/0x55
 [<78104993>] kernel_thread_helper+0x7/0x10
 =======================
Code: 89 da 25 00 40 02 00 3d 00 40 02 00 75 03 8b 53 0c 8b 02 a8 80 75 04 0f 0b eb fe 8b 5a 1c 8b 44 24 20 8b 74 87 58 8b 53 04 8b
EIP: [<781657ba>] free_block+0x6d/0xe4 SS:ESP 0068:f7f47e40

Steps to reproduce:
Unknown. This problem happen only once, after 50d uptime.
Comment 1 Christoph Lameter 2007-11-14 14:03:33 UTC
This is a typical trace if slab memory objects were corrupted. Is this occurring
on a single machine or can it be reproduced on other machines?
Comment 2 Krzysztof Oledzki 2007-11-15 14:30:42 UTC

> This is a typical trace if slab memory objects were corrupted. Is this
> occurring
> on a single machine or can it be reproduced on other machines?

This occurred only once on a single machine. However, with no reports from 
ECC, EDAC MC and MCE it is IMHO very unlikely for this crash to be caused 
by a hardware malfunction.
Comment 3 Christoph Lameter 2007-11-15 18:26:30 UTC
> This occurred only once on a single machine. However, with no reports from 
> ECC, EDAC MC and MCE it is IMHO very unlikely for this crash to be caused 
> by a hardware malfunction.

Memory can also be corrupted by faulty system software components 
overwriting memory. Recompile the kernel with CONFIG_SLAB_DEBUG on to 
have a better chance to detect these things.
Comment 4 Krzysztof Oledzki 2007-11-18 13:12:46 UTC

> Memory can also be corrupted by faulty system software components
> overwriting memory. Recompile the kernel with CONFIG_SLAB_DEBUG on to
> have a better chance to detect these things.

If I only could do this... CONFIG_SLAB_DEBUG unfortunately makes my system 
so slow that it is not able to handle his normal tasks. Previously it took 
~2 months for this error to come out, it is not possible to leave it with 
this option enabled for so long. :(

Best regards,

 				Krzysztof Ol