Monitor file (size) progress

From time to time I do several maintenance tasks that involve file copying, compressing, disk image cloning, etc.

Because sometimes (in fact almost always) these administrative task involves several commands that use the output/result of each other, I cannot use the GUI applications (which usually provides end-user feedback, like progress) but only CUI applications:

• cp
• dd
• tar

etc.

These programs, despite their strengths, don't provide a way to monitor file size progress while they're working. Of course, there are some workarounds, sometimes there even exists some additional software that one can use to satisfy this requirement (see my notes at the bottom). But I don't want too many dependencies (I have to depend on myself, so I think that's enough) so, when I have such a situation I use the following strategy:

• knowing the source file size and watching continuously the target file size, I can approximate the percentage of work done

So what I do is basically:

1. get the size of the source file (expressed in MB)
2. list the target file and get its current size
3. print on screen (echo) the result of 100*current_size/source_file_size

E.g.:

ls -las --block-size=1M target.file | awk '{print 100*$1/xxxx;}'

where target.file is the file which progress I do monitor and xxxx is the size of the file being copied.

But if you want to watch this process continuously, every 1 second, then you can automate this little bit. You can use the watch command where you can specify the command to watch every N seconds. The problem with this combination (our command and watch) is that the progress's value (100*current/total) is evaluated at the beginning, before the watch is starting using it. So watch is no use for us. Instead we can use a while loop, and the sleep command, like below:

while [ true ];do ;sleep 1;done

where is just the command I shown you earlier.

E.g.:

while [ true ];do ls -las --block-size=1M target.file | awk '{print 100*$1/100;}';sleep 1;done

Can that be made easier than this? Yes, it can. I wrote a bash script named fwatch (file watch) that allows me to specify the source and the target file, optionally the frequency for displaying the progress, and it provides me the current progress, nothing more, nothing less. I don't need to remember the whole line of the code, I don't need to check the source file size, the script does this by itself.

#!/bin/bash

if [ $# -lt 2 ];then
	echo "Invalid number of arguments!"
	echo -e "Usage:\n\t$0   [frequency]"
	echo -e "where\n\tsource-file : the file that is copied (we're using to guess the final size)"
	echo -e "\tfile-to-monitor : is the file being created (we monitor its current size)"
	echo -e "\tfrequency (seconds): optional, how often to check/print the file status"
	exit 1
fi

SRC=$1 
DST=$2

FREQ=1
if [ -n "$3" ];then
	FREQ=$3
fi

SIZE=$(ls -las --block-size=1M ${SRC}|awk '{print $1}')

if [ -z "$SIZE" ];then
	SIZE=0
fi

while [ true ];do ls -las --block-size=1M $DST | awk -v size=$SIZE '{if (size == 0) done=0; else done=100*$1/size;printf "\rStatus: %.2f%% done",done;}';sleep $FREQ ;done

Using this script the only thing I have to do, instead the command I shown you earlier, is:

fwatch source.file target.file

Notes:

• to monitor the progress of something that goes via a pipe, one may use the Pipe Viewer (pv command), though I never tried
• to monitor the progress of dd command one may use the following trick:
  • find the process id for the dd command (like 'pgrep dd')
  • send the kill command to that process using a special USR1 signal; the dd command knows how to react to this particular signal, so in fact the dd command will print-out the current progress (i.e. kill -USR1 pid)

Now, if you think that this article was interesting don't forget to rate it. It shows me that you care and thus I will continue write about these things.

The following two tabs change content below.

• Bio
• Latest Posts

Eugen Mihailescu

Founder/programmer/one-man-show at Cubique Software

Always looking to learn more about *nix world, about the fundamental concepts of math, physics, electronics. I am also passionate about programming, database and systems administration. 16+ yrs experience in software development, designing enterprise systems, IT support and troubleshooting.

Latest posts by Eugen Mihailescu (see all)

• Dual monitor setup in Xfce - January 9, 2019
• Gentoo AMD Ryzen stabilizator - April 29, 2018
• Symfony Compile Error Failed opening required Proxies - January 22, 2018