Speed up bzip2/xz compression

If you have a SMP system and wonder if exists such thing that can help you to use all CPUs to compress/decompress a file then the answer is YES, there is. It is PBZIP2 - the tool that speed up bzip2/xz compression using all your system's CPUs:

PBZIP2 is a parallel implementation of bzip2 block-sorting file compressor that uses pthreads and achieve near-linear speedup on SMP machines.

In fact there is even a version of BZIP2 that runs on cluster machines, it is called MPIBZIP2 which was designed for a distributed-memory message-passing architecture.

Edit: If you are going to use the Lempel-Ziv-Markov compression algorithm (LZMA) then even xz has a parallel implementation, called pxz.

Now, if you think that this article was interesting don't forget to rate it. It shows me that you care and thus I will continue write about these things.

The following two tabs change content below.

• Bio
• Latest Posts

Eugen Mihailescu

Founder/programmer/one-man-show at Cubique Software

Always looking to learn more about *nix world, about the fundamental concepts of math, physics, electronics. I am also passionate about programming, database and systems administration. 16+ yrs experience in software development, designing enterprise systems, IT support and troubleshooting.

Latest posts by Eugen Mihailescu (see all)

• Dual monitor setup in Xfce - January 9, 2019
• Gentoo AMD Ryzen stabilizator - April 29, 2018
• Symfony Compile Error Failed opening required Proxies - January 22, 2018