How to Check Sparse Files with Perl

During my working period, my backup script is using rsync to copy files to remote server, and I have just realized that without some special parameter, rsync does not handle well on sparse files. (a.k.a Files with holes in them that are holding a file system and just take the space needed by their actual size). A consequence is that on the target side, sparse files will take their real size on the new file system.
Let me create quick example here, from this example we can see that sparse file are of different sizes in different commends. We use ls -l and ls -sh here, and ls -sh shows their real size.

[joseph@sparse test]$ mkdir a
[joseph@sparse test]$ mkdir b
[joseph@sparse test]$ ls
a  b
[joseph@sparse test]$ cd a
[joseph@sparse a]$ dd if=/dev/zero of=./sparse.file seek=1000 bs=4k count=1
1+0 records in
1+0 records out
[joseph@sparse a]$ ls -l
total 8
-rw-rw-r--  1 coremail coremail 4100096 Dec 17 21:52 sparse.file
[joseph@sparse a]$ ls -sh sparse.file
8.0K sparse.file
[joseph@sparse a]$ cd ..
[joseph@sparse test]$ ls
a  b
[joseph@sparse test]$ rsync -av a/ b/
building file list ... done
./
sparse.file

sent 4100736 bytes  received 40 bytes  8201552.00 bytes/sec
total size is 4100096  speedup is 1.00
[joseph@sparse test]$ du -sh *
12K     a
4.0M    b
[joseph@sparse test]$ ls -sh b/
total 4.0M
4.0M sparse.file

Under default options, rsync cannot handle these abnormal files, but when we add -S parameter, things will go well as expected:

[joseph@sparse test]$ rm -f b/sparse.file
rm -f b/sparse.file
[joseph@sparse test]$ rsync -av -S a/ b/
building file list ... done
./
sparse.file

sent 4100736 bytes  received 40 bytes  8201552.00 bytes/sec
total size is 4100096  speedup is 1.00
[joseph@sparse test]$ ls -sh b/
total 8
8.0K sparse.file
[joseph@sparse test]$ du -sh *
12K     a
12K     b

When you plan to use rsync to backup your issued file system, dont forget to add -S to handle these files with holes, as they are common to see on a corrupted file-system (or rotted file-system).

now, we face another question, how can we check these sparse files by script? As we said before, sparse files are of different size under different commands, it’s easy to create a shell script to do that.

Here, I’ll give a example in Perl:

#!/usr/bin/perl -w
# Created by Joseph.A.Chen (http://www.admon.org)
sub Check_Sparse_Files($) {
    my $file = shift;
    my @status = stat($file);
    if ( $status[7] > ($status[12] * 512) )
    {
        print "$file is sparse!n";
    }
}

Check_Sparse_Files("sparse.file");

$status[12] means the actual number of blocks allocated, and by default, on most linux distributions, the default block size is 512 Bytes. So we assume that the maximum size for the specified file is ($status[12] * 512).

For more question please registe an account here and admon.org, and raise your isses at Admon Forum.

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading ... Loading ...

2 Responses to “How to Check Sparse Files with Perl”

  1. GlenStef says:

    Can i take a one small picture from your site?
    Thanks

  2. anonSysAdmin says:

    Thanks, this is very useful

Leave a Reply

© 2014 Admon Linux. All rights reserved. Site Admin · Entries RSS · Comments RSS
Powered by DigitalOcean