My *nix world

Speed-up system by SquashFs

It has been a while since I installed Gentoo. I always searched for new ways to improve the speed my system boot and run. After goggling little bit on net I found something about SquashFs and how to use it to squash your portage tree. But can I use the same technique to squash my entire /usr ? Well, this article says that you can.

Basically SquashFs allows you to create a compressed (zlib, lzo or xz) readonly filesystem. But we need to write to it, do we?

So we need another filesystem designed to allow us to write the filesystem content. That would be AnotherUnionFS (aufs). As Wikipedia says:

"aufs it's an advanced multi layered unification filesystem which implements union mount for Linux filesystem. It aimed to improve reliability and performance, but also introduced some new concepts, like writable branch balancing and other improvements".

So it seems something we can use. Let's speed-up system by SquashFs filesystem!

If you had installed SquashFS and aufs then it is a good time to do some modification in your system:

(in your DISTDIR is located outside of /usr then skip the next two steps below)

  1. define in your /etc/make.conf the path to DISTDIR, which should be outside /usr folder (ex: DISTDIR="/var/portage/distfiles")
  2. make sure you move the above folder from /usr/portage/distfiles location to the new DISTDIR location
  3. squash your /usr folder to a compressed filesystem image:
mksquashfs /usr /var/portage/portage-current.sqfs -b 65536
  1. create the following /etc/init.d/squash_usr init script that will help us to mount/unmount the compressed image filesystem when system starts/stops:
# Copyright 1999-2006 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $
# /etc/init.d/squashd allows efficient compression of Gentoo /$IMGNAME
# It requires support for the loop device and squashfs enabled in the kernel,
# module autoloading is also *highly* recommended.
# sys-fs/squashfs and sys-fs/aufs are necessary for read-write support.
# Author: Mathias Laurin <>
# 2006-11-28, v.0.1.5(4)
# 2009-02-24, v.0.1.6(1) Weedy <>
# 2009-03-20, v.0.1.7(1) j0inty <>
# 2009-07-10, v.0.1.8(1) j0inty
# 2009-09-01, v.0.1.9(1) nall <>
# 2012-02-25. v.0.1.10(2) Eugen <>

#source /etc/make.globals
source /etc/make.conf

SQFS_CUR="$SQFS_DIRNAME/$IMGNAME.sqfs" # the squashed filesystem's image
SQFS_NEW="$SQFS_DIRNAME/$IMGNAME-current.sqfs" # the current using version
SQFS_OLD="$SQFS_DIRNAME/$IMGNAME-old.sqfs" # the old version of image file

MNTDIR_RW="" # where to mount the rw portion of squashed filesystem
DEF_RW="/dev/shm/.$IMGNAME-rw" # if MNTDIR_RW not defined then use default one

SQFS_OPTS="-all-root -no-duplicates" # options used for mounting SquashFS

depend() {
need localmount modules $DEPENDS
before $BEFORE

check_support() {
if ! [ -w /dev/loop0 ] ; then
eerror "ERROR: loopback support is not available."
return 1
if ! [[ $(grep -s $'taufs$' /proc/filesystems) ]] ; then
eerror "ERROR: aufs filesystem support is not available."
return 1
if ! [[ $(grep -s $'tsquashfs$' /proc/filesystems) ]] ; then
eerror "ERROR: squashfs filesystem support is not available."
return 1
return 0

makeImage() {
mksquashfs $MNTDIR_RO $SQFS_NEW $SQFS_OPTS -b 65536 # 2>/dev/null
eend $retval

start() {
check_support || return 1
if [ -f "$SQFS_CUR" ]; then
ebegin "SQFS-$IMGNAME: Mounting $SQFS_CUR read-only squashfs image as $MNTDIR_RO"
mount -t squashfs -o loop,ro $SQFS_CUR $MNTDIR_RO
[ $retval -ne 0 ] "" return $retval
if [ ! -d $MNTDIR_RO ]; then
ebegin "SQFS-$IMGNAME: $SQFS_CUR looks empty or corrupted, using the original"
return 2

ebegin "Mounting $MNTDIR_RW read-write with aufs as $MNTDIR_RO"
if [ ! $MNTDIR_RW ] ; then
einfo " will be mounted in tmpfs (RAM)"
[ -d $MNTDIR_RW ] || mkdir -p $MNTDIR_RW
mount -t aufs -o udba=reval,br=$MNTDIR_RW:$MNTDIR_RO aufs $MNTDIR_RO
eend $?

stop() {
ebegin "SQFS-$IMGNAME: Stopping and unmounting"
rw_size=$(du -s --exclude={.w*$EXCLUDE} $MNTDIR_RW | cut -f 1)
if [ $rw_size -gt 4 ]; then
einfo " Changes detected ($rw_size bytes), updating image."
rm -f $SQFS_OLD
einfo " No changes detected, skipping update."
eend 0

einfo " Unmounting the $MNTDIR_RO aufs/squash image"
umount -l -t aufs $MNTDIR_RO
umount -l -t squashfs $MNTDIR_RO
rm -rf $MNTDIR_RW
eend 0
  1. make sure you tell your system to start that script after booting:
rc-update add squash_usr

That's all, it should works!

A more abstract version for this can be found here.

Note: don't delete the original /usr folder (located on the physical partition) in order to free some space unless you know what you are doing! Keep in mind that a lot of utilities are located on /usr/bin or /usr/sbin, a lot of libraries are located on /usr/lib, some configuration are stored event on /usr/share/.

I had deleted some folders inside the original /usr to free some space (that's why I am using a SquashFS in the first place), BUT I had kept the following folders:

  • bin
  • lib
  • lib64 (or lib32 or both , depending on if you are using multilib Gentoo system or not)
  • sbin
  • share/slim
  • tmp

Well, you are not supposed to keep the entire contents of those folders but only the necessary utilities/libraries/configuration files. So it is a matter of try and error if you want to get dirty on your hands.

Another issue (the biggest one I guess) you have to think about would be: how those utilities/libraries/configuration files from your original /usr folder (used just at boot-time) get updated when you rebuild your system? I'm asking this because you will have mounted the SquashFS image and not the physical /usr folder. So after you are successfully booting your system and your SquashFS image get mounted as /usr, everything you will update on your /usr will be stored inside the SquashFS image (ex: /var/portage/portage-current.sqfs) and not on the original /usr folder.

My advice is to analyze the whole scenario before starting and doing anything. What I have learned from here is that "you can" and "how to do it" but not "whether it can be applied to a particular situation or not".

Warning: before doing anything backup your data ;o)

Note: if you are using a SSD disk drive and not a HDD instead, I am not quite sure that you will gain in performance but for sure you will free some space on your disk.

Now, if you think that this article was interesting don't forget to rate it. It shows me that you care and thus I will continue write about these things.

The following two tabs change content below.
Speed-up system by SquashFs

Eugen Mihailescu

Founder/programmer/one-man-show at Cubique Software
Always looking to learn more about *nix world, about the fundamental concepts of math, physics, electronics. I am also passionate about programming, database and systems administration. 16+ yrs experience in software development, designing enterprise systems, IT support and troubleshooting.
Speed-up system by SquashFs

Latest posts by Eugen Mihailescu (see all)

Tagged on: , , , , ,

5 thoughts on “Speed-up system by SquashFs

  1. Petros

    How the system is supposed to boot and mount /usr if it squashed?

    After all mount is located in (the squashed) /usr and we face the chicken and egg situation.

    1. Eugen Mihailescu Post author

      Good question. From the manual:

      /usr usually contains by far the largest share of data on a system. Hence, this is one of the most important directories in the system as it contains all the user binaries, their documentation, libraries, header files, etc.... X and its supporting libraries can be found here. User programs like telnet, ftp, etc.... are also placed here. In the original Unix implementations, /usr was where the home directories of the users were placed (that is to say, /usr/someone was then the directory now known as /home/someone). In current Unices, /usr is where user-land programs and data (as opposed to 'system land' programs and data) are. The name hasn't changed, but it's meaning has narrowed and lengthened from "everything user related" to "user usable programs and data". As such, some people may now refer to this directory as meaning 'User System Resources' and not 'user' as was originally intended.

      Basically your system can boot without the user-land programs. All it needs is (love) the system land programs and data which happily are stored somewhere else 🙂

      So your compressed/squashed image will be decompressed as soon the Linux Core components are loaded (before X-Server or other user-lands programs). Please read this to learn how to squash any arbitrary folder.

    1. Eugen Mihailescu Post author

      Yes, squashmount is also a script that does the same thing (including the read-write aspect I mentioned in my post). My goal was to learn how things are done and also I tried to squeeze the Gentoo a bit. There is place for improvements, though 🙂

      I hope it helped and if it did then it was my pleasure 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *