Bug 15931 - Redirected filesystem using bind option suddenly thinks every file is a directory
Summary: Redirected filesystem using bind option suddenly thinks every file is a direc...
Status: RESOLVED OBSOLETE
Alias: None
Product: File System
Classification: Unclassified
Component: Other (show other bugs)
Hardware: All Linux
: P1 normal
Assignee: fs_other
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-05-07 11:55 UTC by Pepijn Schmitz
Modified: 2012-07-11 16:09 UTC (History)
1 user (show)

See Also:
Kernel Version: 2.6.31.9
Subsystem:
Regression: Yes
Bisected commit-id:


Attachments

Description Pepijn Schmitz 2010-05-07 11:55:31 UTC
I'm sorry for this lengthy report, but the problem is complex and hard to explain.

I'm having a bizarre problem on my Ubuntu Karmic 64-bit workstation: a number of directories mounted using the "bind" option has suddenly started to show every file in them with the same permissions as the directory itself, including the fact that it is a directory! In other words, Linux suddenly thinks all the files in my bound directories are directories!

This started happening after installing Ubuntu kernel 2.6.31-20, which corresponds to mainline kernel 2.6.31.9. In Ubuntu kernel 2.6.31-20 the problem was not there, which corresponds to mainline kernel 2.6.31.6. I have verified that the problem exists in the actual mainline kernel (it is not caused by an Ubuntu patch).

Here's a part of my /etc/fstab:

//diskstation/home /home/pepijn_remote cifs credentials=/home/pepijn.smbcredentials,uid=pepijn,gid=pepijn,iocharset=utf8 0 0
/home/pepijn_remote/Documents /home/pepijn/Documents none bind 0 0
/home/pepijn_remote/Downloads /home/pepijn/Downloads none bind 0 0
etc...

As you can see, I mount my remote home directory from my NAS on /home/pepijn_remote using CIFS. I then bind some of the directories from that share in my real home directory. This has worked perfectly for months now.

But then suddenly all hell broke loose. Programs started going haywire. It turned out that this was because Linux had suddenly decided that all the files in my directories were now directories! As you can imagine my IDE went berserk. To illustrate, here's (the first few lines of) the result of an "ls -al" in /home/pepijn_remote/Documents:

total 256
drwxrwxrw- 12 pepijn pepijn 0 2010-03-07 12:24 .
drwxr-xr-x 10 pepijn pepijn 0 2010-02-09 22:50 ..
-rw-r--r-- 1 pepijn pepijn 14702 2010-02-12 12:43 Afrekening Merlot.ods
-rw-r--r-- 1 pepijn pepijn 22287 2010-01-15 19:37 Afrekening New York en Atlanta.ods
-rw-r--r-- 1 pepijn pepijn 13153 2010-02-18 11:34 Amerika 2010.ods
drwxrwxrwx 3 pepijn pepijn 0 2010-01-06 13:04 Belastingdienst
etc...

Perfectly normal. But here's the result of the same command in /home/pepijn/Documents (which should be exactly the same, and has been for months):

total 12
drwxrwxrw- 12 pepijn pepijn 0 2010-03-07 12:24 .
drwxr-xr-x 139 pepijn pepijn 12288 2010-03-07 12:44 ..
drwxrwxrw- 12 pepijn pepijn 0 2010-03-07 12:24 Afrekening Merlot.ods
drwxrwxrw- 12 pepijn pepijn 0 2010-03-07 12:24 Afrekening New York en Atlanta.ods
drwxrwxrw- 12 pepijn pepijn 0 2010-03-07 12:24 Amerika 2010.ods
drwxrwxrw- 12 pepijn pepijn 0 2010-03-07 12:24 Belastingdienst
etc...

As you can see it thinks every file is a directory! In fact, the permissions of every entry are precisely the same as for ., the current directory. I have verified that the same goes for the other directories redirected using the "bind" option, they all have the same permissions as . for every file in the directory.

When it happens it plays havoc with my system, since any process which processes directories recursively (such as trackerd, the search indexer) gets stuck in an endless loop, consuming more and more resources.

It gets even more bizarre: not only does Linux think every file is a directory, but it even thinks non-existent files are directories!!! I can "cd" to a non-existent directory completely successfully as far as the shell is concerned, it even thinks I'm in the directory. Here's a snippet I copy and pasted from my terminal window:

~/Documents$ cd aasdghf
~/Documents/aasdghf$ cd fnurk
~/Documents/aasdghf/fnurk$ cd blah
~/Documents/aasdghf/fnurk/blah$

Needless to say, I don't have a directory (nor a file) called aasdghf in my Documents directory. If I do an "ls" at this point I see the contents of the Documents directory (with every file listed as a directory with the same permissions as the Documents directory itself). If I try to do the same from /home/pepijn_remote I get:

/home/pepijn_remote/Documents$ cd aasdghf
bash: cd: aasdghf: No such file or directory

The problem is recurrent, but very unpredictable. When it happens, I can unmount all bound directories (after hunting down every process with open files on them) and mount them again and it will be fine for a while. When it first happened I was working in Netbeans (which I don't think had anything to do with the problem), it was not after rebooting or installing something or anything like that.

One program which seems to trigger the bug very often is Filezilla. For some reason when I start that the problem will almost always immediately occur. But not every time, and it also sometimes happens when Filezilla is not running. I've also see it happen when trackerd starts.

I've filed a bug with Ubuntu, but they seem uninterested in helping me. I haven't received one single reply.

I don't think the problem is in CIFS, since the directories look fine if I look at them under /home/pepijn_remote, it's only the rebound versions that display the problem.

Please let me know how I can help debug this problem by providing logs and testing things!

Here's the output of ver_linux:
Linux peregrin 2.6.31-02063112-generic #02063112 SMP Tue Jan 19 11:52:21 UTC 2010 x86_64 GNU/Linux
 
Gnu C                  4.4.1
Gnu make               3.81
binutils               2.20
util-linux             2.16
mount                  support
module-init-tools      3.10
e2fsprogs              1.41.9
pcmciautils            014
Linux C Library        2.10.1
Dynamic linker (ldd)   2.10.1
Procps                 3.2.8
Net-tools              1.60
Kbd                    1.15
Sh-utils               7.4
wireless-tools         29
Modules Loaded         nls_iso8859_1 vfat fat bridge stp bnep binfmt_misc vboxnetadp vboxnetflt vboxdrv nls_cp437 nls_utf8 cifs nvidia ppdev parport_pc snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_usb_audio snd_pcm_oss snd_mixer_oss snd_usb_lib snd_pcm snd_seq_dummy snd_hwdep snd_seq_oss coretemp snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq f71882fg snd_timer iptable_filter snd_seq_device uvcvideo psmouse videodev snd snd_page_alloc ip_tables i2c_nforce2 v4l1_compat soundcore v4l2_compat_ioctl32 serio_raw lp x_tables btusb parport usbhid usb_storage ohci1394 ieee1394 forcedeth
Comment 1 Pepijn Schmitz 2010-05-07 11:58:45 UTC
One additional clue: using "ls -i" i discovered that the reason that the permissions are all the same as those of "." is that the inode number is also the same as that of "."!
Comment 2 Pepijn Schmitz 2010-05-10 23:23:03 UTC
I've been running with kernel 2.6.34 for a while now and the problem does not seem to occur in that version, so the bug may have been fixed somewhere between 2.6.31.12 and 2.6.34.

Note You need to log in before you can comment on or make changes to this bug.