This document describes why numbers returned by du -s and df are inconsistent. The document applies to AIX Versions 4.x.
Notice that du and df report on only the blocks allocated for data actually written. The ls command reports slightly different results depending on the type of file. See the section in this document called, "The ls command".
At AIX versions prior to 4.1, df reports its statistics in 1024-byte units and du reports in 512-byte units. At AIX 4.1 and later, both df and du default to 512-byte units. The following discussion addresses df and du in AIX 4.x; thus all units are in 512-byte blocks.
Sometimes du and df are used to get a free block value: df is used to report the total block count, and then the value returned by du -s /filesystem_path is subtracted from that total to calculate the free block value. However, this method of calculation yields a value that is greater than the free block value reported by df.
For example, sample output from executing du -s /tmp is as follows:
12920 /tmpSample output from executing df /tmp on the same system is as follows:
Filesystem 512-blocks Free %Used Iused %Iused Mounted on /dev/hd3 57344 42208 26% 391 4% /tmp
Here, the <total from df> - <used from du> = <false free block count>: 57344 - 12920 = 44424.
44424 is greater than 42208. The reason for this discrepancy involves the implementation of du and df.
du -s traverses the file tree, adding up the number of blocks allocated to each directory, symlink, and file as reported by the stat() system call. This is how du arrives at its total value.
df looks at the file system disk block allocation maps to arrive at its total and free values.
The file system allocates some of the disk blocks in the file system to record its data. This data is referred to as meta data. Meta data is not visible to most user-level programs. Examples of meta data are inodes, disk maps, indirect blocks, and super blocks.
1 4k block for the LVM 2 4k super blocks 2 4k blocks for disk maps 2 4k blocks for inode maps 2 4k blocks for .indirect 32 4k blocks for inodes ------------------------- 41 4k blocks for meta data on an empty 4MB file system
8 /foo/lost+found 16 /foo
The sixteen 512-byte blocks reported by du on this empty file system are the blocks used by the root directory.
To get the output from du to match that from df, we must add in the meta data. First, convert 41 4K blocks to 512-byte units:
41 * 8 = 328
328(meta data) + 16(from du) = 344
So there are 344 512-byte blocks allocated on this empty file system. For
example:
8192(total blocks) - 344(used from du + meta data) = 7848
This value does match the output from the free column reported by df /foo.
Filesystem 512-blocks Free %Used Iused %Iused Mounted on /dev/lv01 8192 7848 5% 16 2% /foo
This calculation was easy to perform on an empty file system. However, on a non-empty file system, the meta data for file indirect blocks comes into play and such calculations are tedious and impractical.
In conclusion, du -s produces a value that reflects the number of disk blocks that are allocated to files and directories. df reports on the actual allocation state of the file system. The true allocation state includes both user data (files and directories) plus meta data.
If someone is running an application with a file open in a directory and the open file is removed, the du output reflects a reduced size for this directory. However, df does not show a reduced size because all blocks in the file system remain allocated until the application that has the file open closes the file. After the file closure, df shows reduced usage for the file system.
The following compares ls output with that of du and df for sparse files.
An example sparse file can be created fairly easily. To do so, open the file, seek to a large address, and write some data. This can be demonstrated with the dd command, as follows:
date > notsparse
ls -l
The output of the ls command will be similar to the following:
total 8
-rw-r--r-- 1 root sys 29 Dec 21 08:12 notsparse
NOTE: Performance Analysis and Control Commands (perfagent.tools) must be installed for AIX Version 4.x.
fileplace notsparse
The output for AIX Version 4.x will look similar to the following:
File: notsparse Size: 29 bytes Vol: /dev/lv03
Bkl Size: 4096 Frag size: 4096 Nfrags: 1 Compress: no
Logical Fragment
----------------
00716 1 frags 4096 bytes, 100.0%
du -rs *
Example output looks similar to the following:
8 notsparse
touch sparse.1
dd if=notsparse of=sparse.1 seek=100
Example output looks similar to the following:
dd: 0+1 records in.
dd: 0+1 records out.
The dd command takes the data from the regular file and places it, in 100 512-byte blocks, into the sparse.1 file. Nothing is written to the initial 99 512-byte blocks. The following steps show the characteristics of the resulting file.
ls -l
Example output looks similar to the following:
total 16
-rw-r--r-- 1 root sys 29 Dec 21 08:12 notsparse
-rw-r--r-- 1 root sys 51229 Dec 21 08:13 sparse.1
fileplace sparse.1
Example output for AIX Version 4.1 looks similar to the following:
File: sparse.1 Size: 51229 bytes Vol: /dev/lv03
Blk Size: 4096 Frag Size: 4096 Nfrags: 1 Compress: no
Logical Fragment
----------------
unallocated 12 frags 49152 Bytes, 0.0%
0000769 1 frags 4096 Bytes, 100.0%
The du command reports the number of allocated blocks the file takes. For example:
du -rs *
The example output looks similar to the following:
8 notsparse
8 sparse.1
Each command correctly reports the data that is specific to its intended purpose. ls shows the range of offsets where data can be read from or written to a file. Reading from an offset where no data is written makes it appear to be zero-filled. du and df report only blocks allocated for data actually written.
NOTE: For more information about sparse files see prism document, "About Sparse Files."
[ Doc Ref: 90605201914642 Publish Date: Dec. 03, 2001]