Bug 15154 - directory (index) becoming huge after time
Summary: directory (index) becoming huge after time
Status: REOPENED
Alias: None
Product: File System
Classification: Unclassified
Component: JFS (show other bugs)
Hardware: All Linux
: P1 normal
Assignee: Dave Kleikamp
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-01-28 01:38 UTC by Christian Kujau
Modified: 2020-05-11 23:43 UTC (History)
2 users (show)

See Also:
Kernel Version: 3.12
Subsystem:
Regression: No
Bisected commit-id:


Attachments

Description Christian Kujau 2010-01-28 01:38:24 UTC
This is about a longstanding issue, as described in the archives ([0],[1]):

 > It [the directory] slowly grows every time a file is created, but does not
 > shrink when files are deleted.  It is reset to zero if a directory is
 > completely emptied

Steps to reproduce, on a freshly formatted JFS:

$ touch dummy
$ for j in `seq -w 1 100`; do
    for i in `seq 1 10000`; do
         touch file.$i
    done
    sync
    rm -f file.*
    printf "`expr $j \* $i`   "; du -sk .
  done | tee ../jfs.log 

Which gives us the following result:

   file     directory
 creations   size
   130000    1028 KB
   260000    2048 KB
   390000    3068 KB
   510000    4012 KB
   640000    5036 KB
   770000    6052 KB
   900000    7072 KB


[0] http://www.mail-archive.com/jfs-discussion@lists.sourceforge.net/msg00748.html

[1] http://www.mail-archive.com/jfs-discussion@lists.sourceforge.net/msg01549.html
Comment 1 Christian Kujau 2010-01-30 04:20:13 UTC
Another report, from Nov 2009:

 http://www.mail-archive.com/jfs-discussion@lists.sourceforge.net/msg01480.html
Comment 2 Christian Kujau 2013-12-14 22:06:50 UTC
Why was this marked obsolete? This is still an issue with e.g. 3.11.10-200.fc19.i686 - as long as all but one file are removed in a directory, this directory will grow over time as new files are created and deleted:

$ rm -f *
$ for j in {1..10}; do 
   seq 1 10000 | xargs -I '{}' touch file.'{}'
   echo "run: $j  `ls | wc -l` files   `du -sk` KB"; rm -f file.*
done
run: 1  10000 files   692       . KB
run: 2  10000 files   692       . KB
run: 3  10000 files   692       . KB
run: 4  10000 files   692       . KB
run: 5  10000 files   692       . KB
run: 6  10000 files   692       . KB
run: 7  10000 files   692       . KB
run: 8  10000 files   692       . KB
run: 9  10000 files   692       . KB
run: 10  10000 files   692      . KB

$ touch dummy
$ for j in {1..10}; do seq 1 10000 | xargs -I '{}' touch file.'{}'
  echo "run: $j  `ls | wc -l` files   `du -sk` KB"; rm -f file.*
done
run: 1  10001 files   772       . KB
run: 2  10001 files   848       . KB
run: 3  10001 files   928       . KB
run: 4  10001 files   1004      . KB
run: 5  10001 files   1084      . KB
run: 6  10001 files   1160      . KB
run: 7  10001 files   1240      . KB
run: 8  10001 files   1316      . KB
run: 9  10001 files   1396      . KB
run: 10  10001 files   1472     . KB

Now, maybe this one is hard to fix and may not be worth fixing, as very few people seem to be affected. And with the posts on jfs-discussion and this bug the information is out there, so I could understand a WONTFIX, but OBSOLETE?
Comment 3 Alan 2013-12-14 22:35:27 UTC
Because it as filed against an ancient now unsupported kernel. Its just part of a mass close of stale old bugs.


I suspect it is 'WONTFIX' however.
Comment 4 Christian Kujau 2020-05-11 23:43:02 UTC
FWIW, this is still present with a 5.4 kernel:

$ for j in {1..10}; do seq 1 1000 | xargs -I '{}' touch file.'{}'; echo "run: $j $(ls | wc -l) files $(du -sk .) KB"; rm -f file.*; done
run: 1 1000 files 72    . KB
run: 2 1000 files 72    . KB
run: 3 1000 files 72    . KB
run: 4 1000 files 72    . KB
run: 5 1000 files 72    . KB
run: 6 1000 files 72    . KB
run: 7 1000 files 72    . KB
run: 8 1000 files 72    . KB
run: 9 1000 files 72    . KB
run: 10 1000 files 72   . KB

$ touch dummy
$ for j in {1..10}; do seq 1 1000 | xargs -I '{}' touch file.'{}'; echo "run: $j $(ls | wc -l) files $(du -sk .) KB"; rm -f file.*; done
run: 1 1001 files 72    . KB
run: 2 1001 files 80    . KB
run: 3 1001 files 92    . KB
run: 4 1001 files 100   . KB
run: 5 1001 files 108   . KB
run: 6 1001 files 116   . KB
run: 7 1001 files 124   . KB
run: 8 1001 files 132   . KB
run: 9 1001 files 140   . KB
run: 10 1001 files 148  . KB

But yeah, I can live with WONTFIX :)

Note You need to log in before you can comment on or make changes to this bug.