Most of you probably noticed TAR.GZ compression isn’t very fast. Recently, during routine system backup I have realised TAR.GZ is limited not by disk read/write, but GZIP compression (98% of computation).
time sudo tar cpfz backup/ubuntu1404.tgz --one-file-system / real 6m20.999s user 6m1.800s sys 0m19.043s
GZIP in its standard implementation is single CPU bound, while most of modern computers can run 4-8 threads concurrently. But there are also multi-core implementations of GZIP ie. PIGZ. I have decided to install PIGZ and plug it with TAR as follows:
sudo apt-get install lbzip2 pigz time sudo tar cpf backup/ubuntu1404.pigz.tgz --one-file-system --use-compress-program=pigz / real 1m43.693s user 8m34.168s sys 0m20.243s
As you can see, TAR.GZ using PIGZ on 4-cores i7-4770k (using 8 threads) is 2.5 times faster than GZIP! And you get standard TAR.GZ archive as output:)
The same applies to BZIP2 compression using LBZIP2.