Thursday, May 3, 2007

Why command df and du reports different output?

You will never notice something like this on FreeBSD or Linux Desktop home system or your personal UNIX or Linux workstation. However, sometime on a production UNIX server you will notice that both df (display free disk space) and du (display disk usage statistics) reporting different output. Usually df will output a bigger disk usage than du.

If Linux or UNIX inode is deallocated you will see this problem. If you are using clustered system (file system such as GFS) you may see this scenario commonly.

Note following examples are FreeBSD and GNU/Linux specific.

Following is normal output of df and du for /tmp filesystem:

# df -h /tmp

Output:

Filesystem Size Used Avail Capacity Mounted on
/dev/ad0s1e 496M 22M 434M 5% /tmp

Now type du command:

# du -d 0 -h /tmp/

Output:

22M /tmp/

Why is there a mismatch between df and du outputs?

However, some time it reports different output (a bigger disk usage), for example:

# df -h /tmp/

Output:

Filesystem Size Used Avail Capacity Mounted on
/dev/ad0s1e 496M 39M 417M 9% /tmp

Now type du command:

1. du -d 0 -h /tmp/

Output:

22M /tmp/

As you see, both df and du reporting different output. Many new UNIX admin get confused with output (39M vs 22M).

Open file descriptor is main causes of such wrong information. For example if file called /tmp/application.log is open by third party application OR by a user and same file is deleted, both df and du reports different output. You can use lsof command to verify this:

# lsof | grep tmp

Output:

bash 594 root cwd VDIR 0,86 512 2 /tmp
bash 634 root cwd VDIR 0,86 512 2 /tmp
pwebd 635 root cwd VDIR 0,86 512 2 /tmp
pwebd 635 root 3rW VREG 0,86 17993324 68 /tmp (/dev/ad0s1e)
pwebd 635 root 5u VREG 0,86 0 69 /tmp (/dev/ad0s1e)
lsof 693 root cwd VDIR 0,86 512 2 /tmp
grep 694 root cwd VDIR 0,86 512 2 /tmp

You can see 17993324K file is open on /tmp by pwebd (our in house software) but deleted accidentally by me. You can recreate above scenario in your Linux, FreeBSD or Unixish system as follows:

First, note down /home file system output:

# df -h /home
# du -d 0 -h /home

If you are using Linux then use du as follows:

# du -s -h /tmp

Now create a big file:

# cd /home/user
# cat /bin/* >> demo.txt
# cat /sbin/* >> demo.txt

Login on other console and open file demo.txt using vi text editor:

# vi /home/user/demo.txt

Do not exit from vi (keep it running).

Go back to another console and remove file demo.txt

# rm demo.txt

Now run both du and df to see the difference.

# df -h /home
# du -d 0 -h /home

If you are using Linux then use du as follows:

# du -s -h /tmp

Login to another terminal and close vi.

Now close the vi and the root cause of the problem should be resoled, the du and df outputs should be correct.

No comments: