Bug 35352 - WARNING: at arch/x86/mm/highmem_32.c:81 __kunmap_atomic, when doing swapoff
Summary: WARNING: at arch/x86/mm/highmem_32.c:81 __kunmap_atomic, when doing swapoff
Status: CLOSED CODE_FIX
Alias: None
Product: Memory Management
Classification: Unclassified
Component: Other (show other bugs)
Hardware: All Linux
: P1 normal
Assignee: Hugh Dickins
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-05-19 01:24 UTC by Witold Baryluk
Modified: 2011-05-30 08:42 UTC (History)
3 users (show)

See Also:
Kernel Version: 2.6.39-rc7-00897-ga2b9c1f
Subsystem:
Regression: Yes
Bisected commit-id:


Attachments
kernel config (85.90 KB, application/octet-stream)
2011-05-19 01:49 UTC, Witold Baryluk
Details
swapoff BUG (172.90 KB, text/plain)
2011-05-19 02:02 UTC, Witold Baryluk
Details

Description Witold Baryluk 2011-05-19 01:24:05 UTC
# modprobe zram
# echo $((256*1024*1024)) > /sys/block/zram0/disksize
# mkswap /dev/zram0
# swapon -p 10 /dev/zram0

[ 2386.525073] Adding 262140k swap on /dev/zram0.  Priority:10 extents:1 across:262140k SS


Ok, so now I have empty /dev/zram0, but my other swap device have some pages. With still plenty of free RAM, and lots of cached memory, it should be safe to disable this other device. So


# swapoff /dev/dm-4

[ 2397.696435] ------------[ cut here ]------------
[ 2397.700674] WARNING: at arch/x86/mm/highmem_32.c:81 __kunmap_atomic+0xdc/0xf0()
[ 2397.704769] Hardware name: 2669UYD
[ 2397.708876] Modules linked in: vboxnetadp vboxnetflt ebtable_nat ebtables acpi_cpufreq mperf uinput blowfish xcbc rmd160 nfsd hdaps tp_smapi thinkpad_ec radeonfb fb_ddc radeon ttm drm_kms_helper drm ipw2200 intel_agp intel_gtt i2c_algo_bit libipw i2c_i801 rng_core cfbfillrect agpgart cfbcopyarea cfbimgblt raid10 raid1 raid0 linear md_mod vboxdrv sr_mod cdrom [last unloaded: scsi_wait_scan]
[ 2397.722548] Pid: 14922, comm: swapoff Tainted: G        W   2.6.39-rc7-thinkpad-t43-00897-ga2b9c1f-dirty #60
[ 2397.727109] Call Trace:
[ 2397.731719]  [<c1771dcf>] ? printk+0x1d/0x1f
[ 2397.736327]  [<c1056822>] warn_slowpath_common+0x72/0xa0
[ 2397.740851]  [<c103633c>] ? __kunmap_atomic+0xdc/0xf0
[ 2397.745196]  [<c103633c>] ? __kunmap_atomic+0xdc/0xf0
[ 2397.749458]  [<c1056870>] warn_slowpath_null+0x20/0x30
[ 2397.753775]  [<c103633c>] __kunmap_atomic+0xdc/0xf0
[ 2397.758054]  [<c110fb3e>] shmem_unuse_inode+0x29e/0x380
[ 2397.762355]  [<c110fceb>] shmem_unuse+0xcb/0x120
[ 2397.766641]  [<c112c638>] try_to_unuse+0x238/0x4a0
[ 2397.770929]  [<c108dcad>] ? put_lock_stats.isra.20+0xd/0x30
[ 2397.775231]  [<c108e23b>] ? lock_release_holdtime.part.21+0x6b/0xd0
[ 2397.779596]  [<c178e10d>] ? sub_preempt_count.part.170+0x4d/0x90
[ 2397.784086]  [<c178e174>] ? sub_preempt_count+0x24/0x50
[ 2397.788557]  [<c112d55c>] sys_swapoff+0x16c/0x450
[ 2397.793076]  [<c1791a98>] sysenter_do_call+0x12/0x38
[ 2397.797608] ---[ end trace fde09f67fd5225da ]---
[ 2397.802280] BUG: unable to handle kernel paging request at fffb9000
[ 2397.807008] IP: [<c110c191>] shmem_swp_set+0x11/0x30
[ 2397.811785] *pdpt = 0000000001c8c001 *pde = 0000000001c91067 *pte = 0000000000000000 
[ 2397.812221] Oops: 0002 [#1] PREEMPT SMP 
[ 2397.812221] last sysfs file: /sys/devices/virtual/block/dm-4/dm/name
[ 2397.812221] Modules linked in: vboxnetadp vboxnetflt ebtable_nat ebtables acpi_cpufreq mperf uinput blowfish xcbc rmd160 nfsd hdaps tp_smapi thinkpad_ec radeonfb fb_ddc radeon ttm drm_kms_helper drm ipw2200 intel_agp intel_gtt i2c_algo_bit libipw i2c_i801 rng_core cfbfillrect agpgart cfbcopyarea cfbimgblt raid10 raid1 raid0 linear md_mod vboxdrv sr_mod cdrom [last unloaded: scsi_wait_scan]
[ 2397.812221] 
[ 2397.812221] Pid: 14922, comm: swapoff Tainted: G        W   2.6.39-rc7-thinkpad-t43-00897-ga2b9c1f-dirty #60 IBM 2669UYD/2669UYD
[ 2397.812221] EIP: 0060:[<c110c191>] EFLAGS: 00010296 CPU: 0
[ 2397.812221] EIP is at shmem_swp_set+0x11/0x30
[ 2397.812221] EAX: f72d1480 EBX: ffffffff ECX: 00000000 EDX: fffb9000
[ 2397.812221] ESI: fffb9000 EDI: 00000001 EBP: f7257eb4 ESP: f7257eb0
[ 2397.812221]  DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
[ 2397.812221] Process swapoff (pid: 14922, ti=f7256000 task=f066c420 task.ti=f7256000)
[ 2397.812221] Stack:
[ 2397.812221]  00000020 f7257ee8 c110fb1d 00000000 00000000 f6eb9300 00000020 fffba000
[ 2397.812221]  f72d1670 f72d1480 00000010 f6eb9300 f241653c f6eb9300 f7257f08 c110fceb
[ 2397.812221]  00000000 00000000 00000020 00000020 f8fb4020 f73b6cc0 f7257f7c c112c638
[ 2397.812221] Call Trace:
[ 2397.812221]  [<c110fb1d>] shmem_unuse_inode+0x27d/0x380
[ 2397.812221]  [<c110fceb>] shmem_unuse+0xcb/0x120
[ 2397.812221]  [<c112c638>] try_to_unuse+0x238/0x4a0
[ 2397.812221]  [<c108dcad>] ? put_lock_stats.isra.20+0xd/0x30
[ 2397.812221]  [<c108e23b>] ? lock_release_holdtime.part.21+0x6b/0xd0
[ 2397.812221]  [<c178e10d>] ? sub_preempt_count.part.170+0x4d/0x90
[ 2397.812221]  [<c178e174>] ? sub_preempt_count+0x24/0x50
[ 2397.812221]  [<c112d55c>] sys_swapoff+0x16c/0x450
[ 2397.812221]  [<c1791a98>] sysenter_do_call+0x12/0x38
[ 2397.812221] Code: fc 89 ec 5d c3 90 8d 74 26 00 3b b0 e4 00 00 00 76 ce eb b4 8d b6 00 00 00 00 55 89 e5 53 3e 8d 74 26 00 83 f9 01 19 db 83 cb 01 
[ 2397.812221]  0a 89 d1 01 58 30 83 c0 3c 29 c1 83 f9 3f 76 0a 89 d0 e8 27 
[ 2397.812221] EIP: [<c110c191>] shmem_swp_set+0x11/0x30 SS:ESP 0068:f7257eb0
[ 2397.812221] CR2: 00000000fffb9000
[ 2397.812221] ---[ end trace fde09f67fd5225db ]---
[ 2397.812221] note: swapoff[14922] exited with preempt_count 2
[ 2398.021379] BUG: sleeping function called from invalid context at kernel/rwsem.c:21
[ 2398.027799] in_atomic(): 1, irqs_disabled(): 0, pid: 14922, name: swapoff
[ 2398.034281] INFO: lockdep is turned off.
[ 2398.040764] Pid: 14922, comm: swapoff Tainted: G      D W   2.6.39-rc7-thinkpad-t43-00897-ga2b9c1f-dirty #60
[ 2398.047421] Call Trace:
[ 2398.054019]  [<c103ea52>] __might_sleep.part.159+0xb2/0xd0
[ 2398.060657]  [<c103ead2>] __might_sleep+0x62/0x90
[ 2398.067145]  [<c17895e0>] down_read+0x20/0x85
[ 2398.073483]  [<c1773de5>] ? audit_free_context+0xa8/0xcd
[ 2398.079635]  [<c105ae80>] exit_mm+0x30/0x120
[ 2398.085613]  [<c105b09b>] do_exit+0x12b/0x440
[ 2398.091483]  [<c178c034>] oops_end+0x94/0xd0
[ 2398.097239]  [<c1771236>] no_context+0xc3/0xcb
[ 2398.103022]  [<c1771366>] __bad_area_nosemaphore+0x128/0x130
[ 2398.108822]  [<c10803df>] ? sched_clock_local+0xcf/0x1d0
[ 2398.114646]  [<c1770c97>] ? pte_offset_kernel+0x1e/0x2d
[ 2398.120500]  [<c178da57>] ? spurious_fault+0x87/0xe0
[ 2398.126317]  [<c178dabe>] ? vmalloc_fault+0xe/0x190
[ 2398.132172]  [<c1771385>] bad_area_nosemaphore+0x17/0x19
[ 2398.138005]  [<c178defc>] do_page_fault+0x2bc/0x480
[ 2398.143882]  [<c1080725>] ? local_clock+0x65/0x70
[ 2398.149709]  [<c178e174>] ? sub_preempt_count+0x24/0x50
[ 2398.155566]  [<c108dcc2>] ? put_lock_stats.isra.20+0x22/0x30
[ 2398.161423]  [<c178b53f>] ? error_code+0x5b/0x64
[ 2398.167246]  [<c178dc40>] ? vmalloc_fault+0x190/0x190
[ 2398.173104]  [<c178dc40>] ? vmalloc_fault+0x190/0x190
[ 2398.178894]  [<c178b543>] error_code+0x5f/0x64
[ 2398.184681]  [<c11200d8>] ? vm_stat_account+0x28/0x50
[ 2398.190473]  [<c178dc40>] ? vmalloc_fault+0x190/0x190
[ 2398.196195]  [<c110c191>] ? shmem_swp_set+0x11/0x30
[ 2398.201952]  [<c110fb1d>] shmem_unuse_inode+0x27d/0x380
[ 2398.207611]  [<c110fceb>] shmem_unuse+0xcb/0x120
[ 2398.213224]  [<c112c638>] try_to_unuse+0x238/0x4a0
[ 2398.218758]  [<c108dcad>] ? put_lock_stats.isra.20+0xd/0x30
[ 2398.224189]  [<c108e23b>] ? lock_release_holdtime.part.21+0x6b/0xd0
[ 2398.229484]  [<c178e10d>] ? sub_preempt_count.part.170+0x4d/0x90
[ 2398.234604]  [<c178e174>] ? sub_preempt_count+0x24/0x50
[ 2398.239623]  [<c112d55c>] sys_swapoff+0x16c/0x450
[ 2398.244590]  [<c1791a98>] sysenter_do_call+0x12/0x38
[ 2398.249456] BUG: scheduling while atomic: swapoff/14922/0x10000003
[ 2398.254382] INFO: lockdep is turned off.
[ 2398.259257] Modules linked in: vboxnetadp vboxnetflt ebtable_nat ebtables acpi_cpufreq mperf uinput blowfish xcbc rmd160 nfsd hdaps tp_smapi thinkpad_ec radeonfb fb_ddc radeon ttm drm_kms_helper drm ipw2200 intel_agp intel_gtt i2c_algo_bit libipw i2c_i801 rng_core cfbfillrect agpgart cfbcopyarea cfbimgblt raid10 raid1 raid0 linear md_mod vboxdrv sr_mod cdrom [last unloaded: scsi_wait_scan]
[ 2398.275647] Pid: 14922, comm: swapoff Tainted: G      D W   2.6.39-rc7-thinkpad-t43-00897-ga2b9c1f-dirty #60
[ 2398.281114] Call Trace:
[ 2398.286491]  [<c1771657>] __schedule_bug+0x74/0x7b
[ 2398.291953]  [<c1787fa0>] schedule+0x6f0/0x730
[ 2398.297349]  [<c111a775>] ? zap_pte_range+0xf5/0x3b0
[ 2398.302720]  [<c111abf6>] ? unmap_page_range+0x1c6/0x230
[ 2398.308036]  [<c178e0c9>] ? sub_preempt_count.part.170+0x9/0x90
[ 2398.313465]  [<c104f987>] __cond_resched+0x17/0x30
[ 2398.318883]  [<c178829d>] _cond_resched+0x2d/0x40
[ 2398.324300]  [<c111b20c>] unmap_vmas+0x18c/0x310
[ 2398.329642]  [<c1121997>] exit_mmap+0xb7/0x190
[ 2398.335000]  [<c10541d9>] mmput.part.24+0x39/0x100
[ 2398.340394]  [<c10542ce>] mmput+0x2e/0x40
[ 2398.345713]  [<c105af45>] exit_mm+0xf5/0x120
[ 2398.351140]  [<c105b09b>] do_exit+0x12b/0x440
[ 2398.356355]  [<c178c034>] oops_end+0x94/0xd0
[ 2398.361462]  [<c1771236>] no_context+0xc3/0xcb
[ 2398.366485]  [<c1771366>] __bad_area_nosemaphore+0x128/0x130
[ 2398.371560]  [<c10803df>] ? sched_clock_local+0xcf/0x1d0
[ 2398.376575]  [<c1770c97>] ? pte_offset_kernel+0x1e/0x2d
[ 2398.381594]  [<c178da57>] ? spurious_fault+0x87/0xe0
[ 2398.386523]  [<c178dabe>] ? vmalloc_fault+0xe/0x190
[ 2398.391436]  [<c1771385>] bad_area_nosemaphore+0x17/0x19
[ 2398.396312]  [<c178defc>] do_page_fault+0x2bc/0x480
[ 2398.401124]  [<c1080725>] ? local_clock+0x65/0x70
[ 2398.405843]  [<c178e174>] ? sub_preempt_count+0x24/0x50
[ 2398.410546]  [<c108dcc2>] ? put_lock_stats.isra.20+0x22/0x30
[ 2398.415297]  [<c178b53f>] ? error_code+0x5b/0x64
[ 2398.420037]  [<c178dc40>] ? vmalloc_fault+0x190/0x190
[ 2398.424740]  [<c178dc40>] ? vmalloc_fault+0x190/0x190
[ 2398.429412]  [<c178b543>] error_code+0x5f/0x64
[ 2398.434123]  [<c11200d8>] ? vm_stat_account+0x28/0x50
[ 2398.438809]  [<c178dc40>] ? vmalloc_fault+0x190/0x190
[ 2398.443544]  [<c110c191>] ? shmem_swp_set+0x11/0x30
[ 2398.448201]  [<c110fb1d>] shmem_unuse_inode+0x27d/0x380
[ 2398.452909]  [<c110fceb>] shmem_unuse+0xcb/0x120
[ 2398.457573]  [<c112c638>] try_to_unuse+0x238/0x4a0
[ 2398.462235]  [<c108dcad>] ? put_lock_stats.isra.20+0xd/0x30
[ 2398.466893]  [<c108e23b>] ? lock_release_holdtime.part.21+0x6b/0xd0
[ 2398.471601]  [<c178e10d>] ? sub_preempt_count.part.170+0x4d/0x90
[ 2398.476352]  [<c178e174>] ? sub_preempt_count+0x24/0x50
[ 2398.481059]  [<c112d55c>] sys_swapoff+0x16c/0x450
[ 2398.485774]  [<c1791a98>] sysenter_do_call+0x12/0x38


# swapoff /dev/dm-4
Ivalid argument
# swapon /dev/dm-4
Device busy


I generally can reproduce this bug in number of ways (always with swapoff). The dmesg I present above is one of the most bad looking. The bug is not because system is lacking memory.

But in most cases, I just got [ 2397.700674] WARNING: at arch/x86/mm/highmem_32.c:81 __kunmap_atomic+0xdc/0xf0(), but after waiting 30 second, /dev/dm-4 swap is disabled properly, and everything looks to be working OK.

(until i start messing again with swap, like swapon /dev/dm-4, to make it back, and then swapoff /dev/zram0, it will quickly BUG somewhere, and after that freeze system).
Comment 1 Nitin Gupta 2011-05-19 01:37:53 UTC
I don't see any zram related function in any of the bracktraces, so cannot see how this might be related to zram. Do you see these warnings even when zram is not used -- i.e. you swapoff/swapon disk swaps only?

>  it will quickly BUG somewhere, and after that
freeze system).

Can you post backtrace from the BUG message?
Comment 2 Witold Baryluk 2011-05-19 01:48:20 UTC
In fact swapoff is not needed probably.

I disabled almost everything on my system, this given me lots of memory, added swapon -p 10 /dev/zram0 , then loggin into gnome, started opera webbrowser, and make -j2 on kernel. After a while, i got 

[ 3307.247895] ------------[ cut here ]------------
[ 3307.247913] WARNING: at arch/x86/mm/highmem_32.c:81 __kunmap_atomic+0xdc/0xf0()
[ 3307.247917] Hardware name: 2669UYD
[ 3307.247919] Modules linked in: lib80211_crypt_wep vboxnetadp vboxnetflt ebtable_nat ebtables acpi_cpufreq mperf uinput blowfish xcbc rmd160 nfsd hdaps tp_smapi thinkpad_ec radeonfb fb_ddc radeon ttm drm_kms_helper drm ipw2200 libipw intel_agp i2c_algo_bit i2c_i801 intel_gtt rng_core cfbfillrect cfbcopyarea agpgart cfbimgblt raid10 raid1 raid0 linear md_mod vboxdrv sr_mod cdrom [last unloaded: scsi_wait_scan]
[ 3307.247991] Pid: 3236, comm: zfs-fuse Tainted: G        W   2.6.39-rc7-thinkpad-t43-00897-ga2b9c1f-dirty #60
[ 3307.247994] Call Trace:
[ 3307.248003]  [<c1771dcf>] ? printk+0x1d/0x1f
[ 3307.248010]  [<c1056822>] warn_slowpath_common+0x72/0xa0
[ 3307.248015]  [<c103633c>] ? __kunmap_atomic+0xdc/0xf0
[ 3307.248020]  [<c103633c>] ? __kunmap_atomic+0xdc/0xf0
[ 3307.248025]  [<c1056870>] warn_slowpath_null+0x20/0x30
[ 3307.248030]  [<c103633c>] __kunmap_atomic+0xdc/0xf0
[ 3307.248038]  [<c14f61f1>] zram_read+0x1c1/0x250
[ 3307.248044]  [<c14f6c38>] zram_make_request+0x88/0xd0
[ 3307.248050]  [<c130be96>] generic_make_request.part.49+0x206/0x7b0
[ 3307.248058]  [<c1090bd1>] ? mark_held_locks+0x51/0x70
[ 3307.248064]  [<c1136c0d>] ? kmem_cache_alloc+0xbd/0x1f0
[ 3307.248070]  [<c108d94f>] ? trace_hardirqs_off_caller+0x1f/0x130
[ 3307.248076]  [<c130c47d>] generic_make_request+0x3d/0x50
[ 3307.248080]  [<c130c4f4>] submit_bio+0x64/0xf0
[ 3307.248087]  [<c117b020>] ? bio_alloc_bioset+0x40/0xc0
[ 3307.248094]  [<c1129ef0>] ? end_swap_bio_write+0x90/0x90
[ 3307.248099]  [<c112d3e0>] ? map_swap_page+0x10/0x20
[ 3307.248104]  [<c1129e12>] ? get_swap_bio+0x32/0x80
[ 3307.248109]  [<c112a058>] swap_readpage+0x38/0x50
[ 3307.248114]  [<c112a534>] read_swap_cache_async+0xb4/0x1b0
[ 3307.248119]  [<c112a695>] swapin_readahead+0x65/0x90
[ 3307.248126]  [<c111a1ae>] do_swap_page+0x11e/0x5f0
[ 3307.248132]  [<c111beb6>] handle_pte_fault+0x1a6/0x240
[ 3307.248138]  [<c111c118>] handle_mm_fault+0x138/0x260
[ 3307.248144]  [<c178dd53>] do_page_fault+0x113/0x480
[ 3307.248150]  [<c11184e5>] ? might_fault+0x95/0xa0
[ 3307.248155]  [<c178b53f>] ? error_code+0x5b/0x64
[ 3307.248159]  [<c178dc40>] ? vmalloc_fault+0x190/0x190
[ 3307.248165]  [<c178dc40>] ? vmalloc_fault+0x190/0x190
[ 3307.248169]  [<c178b543>] error_code+0x5f/0x64
[ 3307.248176]  [<c178007b>] ? acpi_processor_get_info+0x1d0/0x242
[ 3307.248181]  [<c178dc40>] ? vmalloc_fault+0x190/0x190
[ 3307.248185] ---[ end trace 339a5accf47eb713 ]---


Of course this WARNING, is different, but is probably releated. 

System is still working and i have

Every 2,0s: free;cat /proc/swaps                                 Thu May 19 03:47:20 2011

# free
             total       used       free     shared    buffers     cached
Mem:       2049232    1895852     153380          0      28472     263340
-/+ buffers/cache:    1604040     445192
Swap:      6598648      46852    6551796
#  cat /proc/swaps
Filename                                Type            Size    Used    Priority
/dev/dm-4                               partition       6074364 0       -1
/dev/zram0                              partition       524284  46852   10
#

I'm attaching my kernel config to bugzilla.
Comment 3 Witold Baryluk 2011-05-19 01:49:00 UTC
Created attachment 58392 [details]
kernel config
Comment 4 Witold Baryluk 2011-05-19 01:50:46 UTC
(In reply to comment #1)
> I don't see any zram related function in any of the bracktraces, so cannot
> see
> how this might be related to zram. Do you see these warnings even when zram
> is
> not used -- i.e. you swapoff/swapon disk swaps only?

No, when zram is not used I have no problem. In fact I was using compcache some time ago, but today I migrated my boot scripts to use zram, and hit this problem. So it is zram releated for sure.

> 
> >  it will quickly BUG somewhere, and after that
> freeze system).
> 
> Can you post backtrace from the BUG message?

Will try.
Comment 5 Witold Baryluk 2011-05-19 02:02:20 UTC
Created attachment 58402 [details]
swapoff BUG
Comment 6 Witold Baryluk 2011-05-19 02:03:47 UTC
(In reply to comment #1)
> >  it will quickly BUG somewhere, and after that
> freeze system).
> 
> Can you post backtrace from the BUG message?

Attached as https://bugzilla.kernel.org/attachment.cgi?id=58402
Comment 7 Nitin Gupta 2011-05-19 02:40:17 UTC
Can you try this patch:


diff --git a/drivers/staging/zram/zram_drv.c b/drivers/staging/zram/zram_drv.c
index aab4ec4..69414ac 100644
--- a/drivers/staging/zram/zram_drv.c
+++ b/drivers/staging/zram/zram_drv.c
@@ -198,8 +198,8 @@ static void handle_uncompressed_page(struct zram *zram,
 			zram->table[index].offset;
 
 	memcpy(user_mem, cmem, PAGE_SIZE);
-	kunmap_atomic(user_mem, KM_USER0);
 	kunmap_atomic(cmem, KM_USER1);
+	kunmap_atomic(user_mem, KM_USER0);
 
 	flush_dcache_page(page);
 }
@@ -256,8 +256,8 @@ static void zram_read(struct zram *zram, struct bio *bio)
 			xv_get_object_size(cmem) - sizeof(*zheader),
 			user_mem, &clen);
 
-		kunmap_atomic(user_mem, KM_USER0);
 		kunmap_atomic(cmem, KM_USER1);
+		kunmap_atomic(user_mem, KM_USER0);
 
 		/* Should NEVER happen. Return bio error if it does. */
 		if (unlikely(ret != LZO_E_OK)) {
Comment 8 Witold Baryluk 2011-05-19 03:59:04 UTC
I was able to reproduce BUG without using zram (but zram is still loaded in memory, hoever never referenced or used since boot).

Started with no swap.

# swapon /dev/mapper/sredniczarny-swap_1

[  386.764012] Adding 5025788k swap on /dev/mapper/sredniczarny-swap_1.  Priority:-1 extents:1 across:5025788k 

# run python with simple script to consume whole memory + 200MB of swap. suspend it, to stabilize memory consumption.

# swapon -p 10 /dev/mapper/sredniczarny-swap_2


[  557.585833] Adding 819196k swap on /dev/mapper/sredniczarny-swap_2.  Priority:10 extents:1 across:819196k 

# no problem, new swap is empty, after a moment it have maybe few kilobytes.

# swapoff /dev/mapper/sredniczarny-swap_1

# immiedietly I see

[  567.788448] ------------[ cut here ]------------
[  567.793230] WARNING: at arch/x86/mm/highmem_32.c:81 __kunmap_atomic+0xdc/0xf0()
[  567.797995] Hardware name: 2669UYD
[  567.802736] Modules linked in: vboxnetadp vboxnetflt ebtable_nat ebtables acpi_cpufreq mperf uinput blowfish xcbc rmd160 nfsd hdaps tp_smapi thinkpad_ec radeonfb fb_ddc radeon ttm ipw2200 drm_kms_helper drm libipw i2c_i801 inte
l_agp intel_gtt rng_core i2c_algo_bit cfbfillrect agpgart cfbcopyarea cfbimgblt raid10 raid1 raid0 linear md_mod vboxdrv sr_mod cdrom [last unloaded: scsi_wait_scan]
[  567.818118] Pid: 7162, comm: swapoff Tainted: G        W   2.6.39-rc7-thinkpad-t43-00897-ga2b9c1f-dirty #60
[  567.823448] Call Trace:
[  567.828609]  [<c1771dcf>] ? printk+0x1d/0x1f
[  567.833849]  [<c1056822>] warn_slowpath_common+0x72/0xa0
[  567.839052]  [<c103633c>] ? __kunmap_atomic+0xdc/0xf0
[  567.844227]  [<c103633c>] ? __kunmap_atomic+0xdc/0xf0
[  567.849290]  [<c1056870>] warn_slowpath_null+0x20/0x30
[  567.854435]  [<c103633c>] __kunmap_atomic+0xdc/0xf0
[  567.859581]  [<c110fb3e>] shmem_unuse_inode+0x29e/0x380
[  567.864755]  [<c110fceb>] shmem_unuse+0xcb/0x120
[  567.869878]  [<c112c638>] try_to_unuse+0x238/0x4a0
[  567.875004]  [<c178e10d>] ? sub_preempt_count.part.170+0x4d/0x90
[  567.880155]  [<c178e174>] ? sub_preempt_count+0x24/0x50
[  567.885232]  [<c112d55c>] sys_swapoff+0x16c/0x450
[  567.890317]  [<c1791a98>] sysenter_do_call+0x12/0x38
[  567.895299] ---[ end trace 497f82cdd9c98e59 ]---
[  567.900335] BUG: unable to handle kernel paging request at fffb9000
[  567.905411] IP: [<c110c191>] shmem_swp_set+0x11/0x30
[  567.910310] *pdpt = 0000000001c8c001 *pde = 0000000001c91067 *pte = 0000000000000000 
[  567.910310] Oops: 0002 [#1] PREEMPT SMP 
[  567.910310] last sysfs file: /sys/devices/virtual/block/dm-4/dm/name
[  567.910310] Modules linked in: vboxnetadp vboxnetflt ebtable_nat ebtables acpi_cpufreq mperf uinput blowfish xcbc rmd160 nfsd hdaps tp_smapi thinkpad_ec radeonfb fb_ddc radeon ttm ipw2200 drm_kms_helper drm libipw i2c_i801 inte
l_agp intel_gtt rng_core i2c_algo_bit cfbfillrect agpgart cfbcopyarea cfbimgblt raid10 raid1 raid0 linear md_mod vboxdrv sr_mod cdrom [last unloaded: scsi_wait_scan]
[  567.910310] 
[  567.910310] Pid: 7162, comm: swapoff Tainted: G        W   2.6.39-rc7-thinkpad-t43-00897-ga2b9c1f-dirty #60 IBM 2669UYD/2669UYD
[  567.910310] EIP: 0060:[<c110c191>] EFLAGS: 00010296 CPU: 0
[  567.910310] EIP is at shmem_swp_set+0x11/0x30
[  567.910310] EAX: f7683480 EBX: ffffffff ECX: 00000000 EDX: fffb9000
[  567.910310] ESI: fffb9000 EDI: 00000001 EBP: f2917eb4 ESP: f2917eb0
[  567.910310]  DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
[  567.910310] Process swapoff (pid: 7162, ti=f2916000 task=f3f36a20 task.ti=f2916000)
[  567.910310] Stack:
[  567.910310]  00000056 f2917ee8 c110fb1d 00000000 00000000 f6ca0620 00000056 fffba000
[  567.910310]  f7683670 f7683480 00000010 f6ca0620 ee911c7c f6ca0620 f2917f08 c110fceb
[  567.910310]  000000d0 00000000 00000056 00000056 f8c5d056 f72cf1c0 f2917f7c c112c638
[  567.910310] Call Trace:
[  567.910310]  [<c110fb1d>] shmem_unuse_inode+0x27d/0x380
[  567.910310]  [<c110fceb>] shmem_unuse+0xcb/0x120
[  567.910310]  [<c112c638>] try_to_unuse+0x238/0x4a0
[  567.910310]  [<c178e10d>] ? sub_preempt_count.part.170+0x4d/0x90
[  567.910310]  [<c178e174>] ? sub_preempt_count+0x24/0x50
[  567.910310]  [<c112d55c>] sys_swapoff+0x16c/0x450
[  567.910310]  [<c1791a98>] sysenter_do_call+0x12/0x38
[  567.910310] Code: fc 89 ec 5d c3 90 8d 74 26 00 3b b0 e4 00 00 00 76 ce eb b4 8d b6 00 00 00 00 55 89 e5 53 3e 8d 74 26 00 83 f9 01 19 db 83 cb 01 
[  567.910310]  0a 89 d1 01 58 30 83 c0 3c 29 c1 83 f9 3f 76 0a 89 d0 e8 27 
[  567.910310] EIP: [<c110c191>] shmem_swp_set+0x11/0x30 SS:ESP 0068:f2917eb0
[  567.910310] CR2: 00000000fffb9000
[  567.910310] ---[ end trace 497f82cdd9c98e5a ]---
[  567.910310] note: swapoff[7162] exited with preempt_count 2
[  568.085856] BUG: sleeping function called from invalid context at kernel/rwsem.c:21
[  568.091544] in_atomic(): 1, irqs_disabled(): 0, pid: 7162, name: swapoff
[  568.097207] INFO: lockdep is turned off.
[  568.102906] Pid: 7162, comm: swapoff Tainted: G      D W   2.6.39-rc7-thinkpad-t43-00897-ga2b9c1f-dirty #60
[  568.108775] Call Trace:
[  568.114560]  [<c103ea52>] __might_sleep.part.159+0xb2/0xd0
[  568.120449]  [<c103ead2>] __might_sleep+0x62/0x90

[  568.126291]  [<c17895e0>] down_read+0x20/0x85
[  568.132056]  [<c1773de5>] ? audit_free_context+0xa8/0xcd
[  568.137889]  [<c105ae80>] exit_mm+0x30/0x120
[  568.143686]  [<c105b09b>] do_exit+0x12b/0x440
[  568.149411]  [<c178c034>] oops_end+0x94/0xd0
[  568.154956]  [<c1771236>] no_context+0xc3/0xcb
[  568.160319]  [<c1771366>] __bad_area_nosemaphore+0x128/0x130
[  568.165481]  [<c10803df>] ? sched_clock_local+0xcf/0x1d0
[  568.170609]  [<c1770c97>] ? pte_offset_kernel+0x1e/0x2d
[  568.175629]  [<c178da57>] ? spurious_fault+0x87/0xe0
[  568.180713]  [<c178dabe>] ? vmalloc_fault+0xe/0x190
[  568.185727]  [<c1771385>] bad_area_nosemaphore+0x17/0x19
[  568.190724]  [<c178defc>] do_page_fault+0x2bc/0x480
[  568.195796]  [<c1080725>] ? local_clock+0x65/0x70
[  568.200828]  [<c178e174>] ? sub_preempt_count+0x24/0x50
[  568.205928]  [<c108dcc2>] ? put_lock_stats.isra.20+0x22/0x30
[  568.210993]  [<c178b53f>] ? error_code+0x5b/0x64
[  568.216050]  [<c178dc40>] ? vmalloc_fault+0x190/0x190
[  568.221132]  [<c178dc40>] ? vmalloc_fault+0x190/0x190
[  568.226125]  [<c178b543>] error_code+0x5f/0x64
[  568.231128]  [<c11200d8>] ? vm_stat_account+0x28/0x50
[  568.236108]  [<c178dc40>] ? vmalloc_fault+0x190/0x190
[  568.241124]  [<c110c191>] ? shmem_swp_set+0x11/0x30
[  568.246106]  [<c110fb1d>] shmem_unuse_inode+0x27d/0x380
[  568.251081]  [<c110fceb>] shmem_unuse+0xcb/0x120
[  568.256010]  [<c112c638>] try_to_unuse+0x238/0x4a0
[  568.260882]  [<c178e10d>] ? sub_preempt_count.part.170+0x4d/0x90
[  568.265734]  [<c178e174>] ? sub_preempt_count+0x24/0x50
[  568.270534]  [<c112d55c>] sys_swapoff+0x16c/0x450
[  568.275187]  [<c1791a98>] sysenter_do_call+0x12/0x38
[  568.279672] BUG: scheduling while atomic: swapoff/7162/0x10000003
[  568.284026] INFO: lockdep is turned off.
[  568.288323] Modules linked in: vboxnetadp vboxnetflt ebtable_nat ebtables acpi_cpufreq mperf uinput blowfish xcbc rmd160 nfsd hdaps tp_smapi thinkpad_ec radeonfb fb_ddc radeon ttm ipw2200 drm_kms_helper drm libipw i2c_i801 intel_agp intel_gtt rng_core i2c_algo_bit cfbfillrect agpgart cfbcopyarea cfbimgblt raid10 raid1 raid0 linear md_mod vboxdrv sr_mod cdrom [last unloaded: scsi_wait_scan]
[  568.302477] Pid: 7162, comm: swapoff Tainted: G      D W   2.6.39-rc7-thinkpad-t43-00897-ga2b9c1f-dirty #60
[  568.307311] Call Trace:
[  568.312122]  [<c1771657>] __schedule_bug+0x74/0x7b
[  568.316895]  [<c1787fa0>] schedule+0x6f0/0x730
[  568.321595]  [<c111a775>] ? zap_pte_range+0xf5/0x3b0
[  568.326257]  [<c111abf6>] ? unmap_page_range+0x1c6/0x230
[  568.330895]  [<c178e0c9>] ? sub_preempt_count.part.170+0x9/0x90
[  568.335555]  [<c104f987>] __cond_resched+0x17/0x30
[  568.340158]  [<c178829d>] _cond_resched+0x2d/0x40
[  568.344720]  [<c111b20c>] unmap_vmas+0x18c/0x310
[  568.349252]  [<c1121997>] exit_mmap+0xb7/0x190
[  568.353852]  [<c10541d9>] mmput.part.24+0x39/0x100
[  568.358479]  [<c10542ce>] mmput+0x2e/0x40
[  568.363076]  [<c105af45>] exit_mm+0xf5/0x120
[  568.367625]  [<c105b09b>] do_exit+0x12b/0x440
[  568.372181]  [<c178c034>] oops_end+0x94/0xd0
[  568.376731]  [<c1771236>] no_context+0xc3/0xcb
[  568.381141]  [<c1771366>] __bad_area_nosemaphore+0x128/0x130
[  568.385481]  [<c10803df>] ? sched_clock_local+0xcf/0x1d0
[  568.389784]  [<c1770c97>] ? pte_offset_kernel+0x1e/0x2d
[  568.394049]  [<c178da57>] ? spurious_fault+0x87/0xe0
[  568.398350]  [<c178dabe>] ? vmalloc_fault+0xe/0x190
[  568.402608]  [<c1771385>] bad_area_nosemaphore+0x17/0x19
[  568.406818]  [<c178defc>] do_page_fault+0x2bc/0x480
[  568.410979]  [<c1080725>] ? local_clock+0x65/0x70
[  568.415183]  [<c178e174>] ? sub_preempt_count+0x24/0x50
[  568.419230]  [<c108dcc2>] ? put_lock_stats.isra.20+0x22/0x30
[  568.423251]  [<c178b53f>] ? error_code+0x5b/0x64
[  568.427210]  [<c178dc40>] ? vmalloc_fault+0x190/0x190
[  568.431231]  [<c178dc40>] ? vmalloc_fault+0x190/0x190
[  568.435172]  [<c178b543>] error_code+0x5f/0x64
[  568.439092]  [<c11200d8>] ? vm_stat_account+0x28/0x50
[  568.443043]  [<c178dc40>] ? vmalloc_fault+0x190/0x190
[  568.446981]  [<c110c191>] ? shmem_swp_set+0x11/0x30
[  568.450936]  [<c110fb1d>] shmem_unuse_inode+0x27d/0x380
[  568.454924]  [<c110fceb>] shmem_unuse+0xcb/0x120
[  568.458852]  [<c112c638>] try_to_unuse+0x238/0x4a0
[  568.462777]  [<c178e10d>] ? sub_preempt_count.part.170+0x4d/0x90
[  568.466726]  [<c178e174>] ? sub_preempt_count+0x24/0x50
[  568.470673]  [<c112d55c>] sys_swapoff+0x16c/0x450
[  568.474586]  [<c1791a98>] sysenter_do_call+0x12/0x38

manual safe reboot sequence initiated
[  578.490645] SysRq : Emergency Sync.
[  578.511835] Emergency Sync complete
...


In replay to Nitin Gupta
> Can you try this patch:
> ...
Well, I can, but because it looks to be generic problem, I do not think it will change anything. However bugtrace from Comment #2, indicates it could be zram, or different bug.
Comment 9 Nitin Gupta 2011-05-19 16:03:09 UTC
I think this problem is due to shmem_unuse_inode() not doing kmap_atomic/kunmap_atomic in strictly nested manner:

https://lkml.org/lkml/2010/8/19/282

zram has the same issue which is fixed by patch in comment #7
Comment 10 Hugh Dickins 2011-05-20 21:39:31 UTC
You're exactly right: I made a fix to shmem's swapoff in -rc7, quite forgetting this new rule of kmap nesting - so my part of it is a regression in 2.6.39 (and the bug much easier for a highmem-user to reproduce than the original bug it was in the course of fixing).

I have a fix, I'll carry on testing it just a little longer, then rush it to Linus and add it here.  No disrespect intended to Witold and Nitin: but since GregKH already has the offending patch on review for 2.6.38-stable and 2.6.33-longterm, I'd like to correct it as soon as possible.

Pray that I'm not introducing a further regression with my fix!

Many thanks for your report and analysis.
Comment 11 Hugh Dickins 2011-05-20 22:51:10 UTC
Commit 778dd893ae78 "tmpfs: fix race between umount and swapoff"
forgot the new rules for strict atomic kmap nesting, causing
WARNING: at arch/x86/mm/highmem_32.c:81 from __kunmap_atomic(), then
BUG: unable to handle kernel paging request at fffb9000 from shmem_swp_set()
when shmem_unuse_inode() is handling swapoff with highmem in use.
My disgrace again.  See https://bugzilla.kernel.org/show_bug.cgi?id=35352

Reported-by: Witold Baryluk <baryluk@smp.if.uj.edu.pl>
Signed-off-by: Hugh Dickins <hughd@google.com>
Cc: stable@kernel.org
---

 mm/shmem.c |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

--- 2.6.39/mm/shmem.c	2011-05-18 21:06:34.000000000 -0700
+++ linux/mm/shmem.c	2011-05-20 13:57:20.778870967 -0700
@@ -916,11 +916,12 @@ static int shmem_unuse_inode(struct shme
 			if (size > ENTRIES_PER_PAGE)
 				size = ENTRIES_PER_PAGE;
 			offset = shmem_find_swp(entry, ptr, ptr+size);
+			shmem_swp_unmap(ptr);
 			if (offset >= 0) {
 				shmem_dir_unmap(dir);
+				ptr = shmem_swp_map(subdir);
 				goto found;
 			}
-			shmem_swp_unmap(ptr);
 		}
 	}
 lost1:
Comment 12 Florian Mickler 2011-05-30 08:00:16 UTC
A patch referencing this bug report has been merged in v3.0-rc1:

commit e6c9366b2adb52cba64b359b3050200743c7568c
Author: Hugh Dickins <hughd@google.com>
Date:   Fri May 20 15:47:33 2011 -0700

    tmpfs: fix highmem swapoff crash regression

Note You need to log in before you can comment on or make changes to this bug.