Bug 198787
Summary: | Btrfs errors leading to readonly filesystem | ||
---|---|---|---|
Product: | File System | Reporter: | Alex Adriaanse (alex) |
Component: | btrfs | Assignee: | Josef Bacik (josef) |
Status: | RESOLVED OBSOLETE | ||
Severity: | high | CC: | dsterba |
Priority: | P1 | ||
Hardware: | x86-64 | ||
OS: | Linux | ||
Kernel Version: | 4.13.13-1, 4.14.7, 4.14.13 | Subsystem: | |
Regression: | No | Bisected commit-id: |
Description
Alex Adriaanse
2018-02-14 17:56:34 UTC
We just got hit by a fourth instance. btrfs balance was not running at the time. df -h /srv/volumes: Filesystem Size Used Avail Use% Mounted on /dev/xvdc 300G 216G 84G 73% /srv/volumes btrfs filesystem usage /srv/volumes: Overall: Device size: 300.00GiB Device allocated: 217.03GiB Device unallocated: 82.97GiB Device missing: 0.00B Used: 214.74GiB Free (estimated): 83.70GiB (min: 83.70GiB) Data ratio: 1.00 Metadata ratio: 1.00 Global reserve: 512.00MiB (used: 0.00B) Data,single: Size:199.00GiB, Used:198.27GiB /dev/xvdc 199.00GiB Metadata,single: Size:18.00GiB, Used:16.47GiB /dev/xvdc 18.00GiB System,single: Size:32.00MiB, Used:48.00KiB /dev/xvdc 32.00MiB Unallocated: /dev/xvdc 82.97GiB [456096.832054] BTRFS: Transaction aborted (error -28) [456096.832084] ------------[ cut here ]------------ [456096.835169] WARNING: CPU: 1 PID: 31862 at /build/linux-3RM5ap/linux-4.14.13/fs/btrfs/extent-tree.c:2274 __btrfs_inc_extent_ref.isra.58+0x272/0x280 [btrfs] [456096.843143] Modules linked in: dm_mod xt_nat xt_tcpudp veth ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack libcrc32c br_netfilter bridge stp llc ppdev intel_rapl sb_edac crct10dif_pclmul crc32_pclmul ghash_clmulni_intel intel_rapl_perf evdev cirrus ttm serio_raw drm_kms_helper drm parport_pc parport button ip_tables x_tables autofs4 btrfs crc32c_generic xor zstd_decompress zstd_compress xxhash raid6_pq ata_generic crc32c_intel xen_blkfront aesni_intel aes_x86_64 crypto_simd cryptd glue_helper psmouse ata_piix libata scsi_mod ena i2c_piix4 [456096.875821] CPU: 1 PID: 31862 Comm: btrfs Not tainted 4.14.0-0.bpo.3-amd64 #1 Debian 4.14.13-1~bpo9+1 [456096.880994] Hardware name: Xen HVM domU, BIOS 4.2.amazon 08/24/2006 [456096.884946] task: ffff908a79142000 task.stack: ffffb038c3d1c000 [456096.889050] RIP: 0010:__btrfs_inc_extent_ref.isra.58+0x272/0x280 [btrfs] [456096.893207] RSP: 0018:ffffb038c3d1f890 EFLAGS: 00010286 [456096.896767] RAX: 0000000000000026 RBX: 0000000000000000 RCX: 0000000000000000 [456096.901131] RDX: 0000000000000000 RSI: ffff908c7fe566f8 RDI: ffff908c7fe566f8 [456096.905557] RBP: ffff90894b8bf0f0 R08: 0000000000000001 R09: 0000000000007409 [456096.910157] R10: 0000000000000001 R11: 0000000000007409 R12: 00000000ffffffe4 [456096.914758] R13: ffff908c7b53f4d0 R14: 0000000000003078 R15: ffff908c7878e000 [456096.919046] FS: 00007fdcbfda68c0(0000) GS:ffff908c7fe40000(0000) knlGS:0000000000000000 [456096.923903] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [456096.927894] CR2: 00007f26f0e14000 CR3: 000000028cd28004 CR4: 00000000001606e0 [456096.932347] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [456096.936800] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [456096.941276] Call Trace: [456096.943637] __btrfs_run_delayed_refs+0x9e7/0x12a0 [btrfs] [456096.947343] btrfs_run_delayed_refs+0x7a/0x270 [btrfs] [456096.950944] create_pending_snapshot+0x5e3/0xf00 [btrfs] [456096.954525] ? create_pending_snapshots+0x95/0xc0 [btrfs] [456096.958337] ? btrfs_wait_pending_ordered+0x76/0x80 [btrfs] [456096.962491] create_pending_snapshots+0x95/0xc0 [btrfs] [456096.966252] btrfs_commit_transaction+0x3b1/0x950 [btrfs] [456096.970080] ? start_transaction+0x89/0x410 [btrfs] [456096.973945] btrfs_mksubvol+0x4a2/0x4f0 [btrfs] [456096.977672] ? remove_wait_queue+0x60/0x60 [456096.980913] btrfs_ioctl_snap_create_transid+0x18b/0x1a0 [btrfs] [456096.985056] btrfs_ioctl_snap_create_v2+0x11d/0x170 [btrfs] [456096.988915] btrfs_ioctl+0x64d/0x24e0 [btrfs] [456096.992077] ? mem_cgroup_try_charge+0x86/0x1a0 [456096.995332] ? __handle_mm_fault+0x640/0x10e0 [456096.998693] ? do_vfs_ioctl+0x9f/0x600 [456097.001539] do_vfs_ioctl+0x9f/0x600 [456097.004440] ? handle_mm_fault+0xc6/0x1b0 [456097.007546] ? __do_page_fault+0x289/0x500 [456097.010538] SyS_ioctl+0x74/0x80 [456097.013278] system_call_fast_compare_end+0xc/0x6f [456097.016611] RIP: 0033:0x7fdcbee2ee07 [456097.019531] RSP: 002b:00007ffdb9f65ec8 EFLAGS: 00000202 [456097.019533] Code: c4 48 83 c4 18 e9 68 ff ff ff 41 bc f4 ff ff ff e9 4d fe ff ff 0f 0b e8 3d 3b a7 e2 44 89 e6 48 c7 c7 a8 e4 4a c0 e8 29 04 ad e2 <0f> ff eb 81 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 45 85 [456097.033855] ---[ end trace cdc156ddd643cca5 ]--- [456097.037360] BTRFS: error (device xvdc) in __btrfs_inc_extent_ref:2274: errno=-28 No space left [456097.043801] BTRFS info (device xvdc): forced readonly [456097.048164] BTRFS: error (device xvdc) in btrfs_run_delayed_refs:3089: errno=-28 No space left [456097.054375] BTRFS: error (device xvdc) in create_pending_snapshot:1625: errno=-28 No space left [456097.060721] BTRFS warning (device xvdc): Skipping commit of aborted transaction. [456097.066005] BTRFS: error (device xvdc) in cleanup_transaction:1873: errno=-28 No space left [456097.311006] BTRFS error (device xvdc): pending csums is 16384 btrfs-snap output at the exact same time as the kernel error: ERROR: cannot snapshot '/srv/volumes/<subvol>': No space left on device Create a readonly snapshot of '/srv/volumes/<subvol>' in '/srv/volumes/<subvol>/.snapshot/15m_2018-02-14_23:48:57' This is a semi-automated bugzilla cleanup, report is against an old kernel version. If the problem still happens, please open a new bug. Thanks. |