My *nix world

How to choose the right kernel I/O scheduler

I was searching for a way to improve my system responsiveness knowing that I/O is a major factor that can influence this. Yes, I can change my HDD with a affordable SSD but I was looking for something that would not cost me money. So I have read little bit about kernel I/O schedulers.

I Linux kernel 3.0.6 I have found just 3 options:

How would I know which one is the best for my hardware and for the type of work I am doing on my system? Hard to say, BUT actually you can test each of them to see which one gives you a better result.

So, the first step is to set your kernel to use one of these three I/O schedulers. The second step is to run a I/O benchmark on your hard-disk using that particular scheduler.

How to choose the right kernel I/O scheduler

How can you check which is the current kernel I/O scheduler? Run at the console the following command:

It will return something like:

noop deadline [cfq]

The one which is included between parenthesis is the current kernel I/O scheduler. To change at runtime the kernel I/O scheduler you can run the following command at the console:

where is one of those which your kernel support.

Run a I/O benchmark on your disk

There exists many I/O benchmark tools on the net. The one which is my favourite is IOzone. By simple running the iozone -h command at your terminal you will get a comprehensive list with all available options (and there are dozens).

I found an interesting post about how to stress your hard-disk and how to get those benchmark's numbers that you can use in a spreadsheet to plot a visual chart which will help you understand which of these I/O schedulers behave better on your hardware:

https://bbs.archlinux.org/viewtopic.php?pid=969117

So I used the following bash script (eg: iozone-scheduler) which actually calls IOzone sequentially and then compiles a log file (a tab-delimited format) that one can use to plot a chart:

So, to test my disk I had used the following command at the terminal:

where:

  • sda is the name of my disk device as recognized by Linux (check your /dev/)
  • /mnt is the folder where the test file will be saved temporary
  • 1024 represent the size in MB of the test file (where I/O will run)
  • /dev/shm is the folder where will be saved the XLS spreadsheets and log files
  • first 3 is the number of runs (cycles) of the test
  • 8 is the record size in MB that will be used for IOzone test (I set 8 because my hdd buffer size is 8MB)
  • the last 3 represents the number of the maximum concurrent threads that will perform I/O operations

Well, I got mine iozone-sda-all-results.log which have the following structure (tab-delimited):

Test Throughput (KB/s) I/O Scheduler Threads n
Initial write 45654.45 cfq 1 1
Rewrite 49748.07 cfq 1 1
Random read 915288.75 cfq 1 1
Mixed workload 1243356.50 cfq 1 1
Random write 49748.07 cfq 1 1
Initial write 60800.41 cfq 1 2
Rewrite 64921.82 cfq 1 2
Random read 1242507.88 cfq 1 2
Mixed workload 1251540.75 cfq 1 2
...........

Using the above information I draw a chart for every of the following 5 I/O tests I've run:

  • Initial write
  • Re-write
  • Random write
  • Random read
  • Mixed workload
how to choose the right kernel

Sample chart based on IOzone result

I have 5 tests on 3 distinct runs, that means a total of 15 charts to plot. Well, I am not going to post all those here (make no sense) but I will tell you that: sometimes cfq behaves better then deadline which behaves better than noop, other time is vice-versa, other time is.... so I have got some interesting info every time. It is hard to decide which one is better than other (because one is better than other on reading, other on writing, other is better in the 2nd run than in the 1st run, etc) . In order to determine which one to choose I approached the problem with the following naive method:

  • I compared each runs individually
    • for each test I compared which of those I/O schedulers behaves better than others
    • I gave 2 point for the best, 1 for the average and 0 for the worst one
  • I added those points that each I/O scheduler have obtained
  • the one which added the most points I decreed as a winner.

I tested my laptop and my desktop workstation.

On my laptop where I have an TOSHIBA MK1637GSX disk the cfq I/O scheduler was the winner (20 points) and noop was the looser (11 points).

On my desktop where I have an WDC WD5000AAKS-60A7B2 disk the noop I/O scheduler was the winner (18 points) and deadline was the looser (11 points). Very close came the cfq (15 points) but noop was little better on all test so I decided to use noop in the future on that system.

Another interesting piece of information is "how much has improved the I/O by changing the kernel scheduler?"

Well, the difference is not a magnitude order but sometimes is 30% better, other time just 18% or only 2%. So the difference can vary between 0-30% or even over. But any improvement is welcome so when you get a positive improvement why not get it?

How to permanently change your kernel I/O scheduler

Well, the method I would prefer is to recompile the kernel, so:

  • Enable the block layer --->
    • IO schedulers --->
      • enable Deadline I/O scheduler (if you intend to use it)
      • enable CFQ I/O scheduler (if you intend to use it)
      • Default I/O schedulers --->
        • check on of the Deadline, CFQ or No-op available schedulers that fits your need.

After you recompile and install your new kernel your I/O operations should (hopefully) behave little better.

Now, if you think that this article was interesting don't forget to rate it. It shows me that you care and thus I will continue write about these things.

 
The following two tabs change content below.
How to choose the right kernel I/O scheduler

Eugen Mihailescu

Founder/programmer/one-man-show at Cubique Software
Always looking to learn more about *nix world, about the fundamental concepts of math, physics, electronics. I am also passionate about programming, database and systems administration. 16+ yrs experience in software development, designing enterprise systems, IT support and troubleshooting.
How to choose the right kernel I/O scheduler

Latest posts by Eugen Mihailescu (see all)

2 thoughts on “How to choose the right kernel I/O scheduler

  1. Shashwat Sher

    Hi Eugen
    Thanks for such an informative post. Actually I was searching for some comparative studies between different I/O Schedulers in Android when I came across your post. The results are informative but just 1 question here.... why aren't there are results with real world tasks/applications. All the benchmarks well, can only be good till a certain extent and can't always show the true picture. Do you know of any links or pages that provide a detailed comparison between I/O schedulers especially for android?

Leave a Reply

Your email address will not be published. Required fields are marked *

Follow

Get every new post on this blog delivered to your Inbox.

Join other followers: