Amazon EC2 I/O benchmarks

So, after my previous post where I used bonnie++ to benchmark my desktop machine, I decided to benchmark the I/O performance of the virtual server (or as Amazon like to call it, an "instance") with the same tool.

The results are as follows:
Note that I've taken a screenshot of the results as opposed to pasting them in this time, as the last time I couldn't get the columns to line up.

So, what can we tell from these numbers? How do they compare with my desktop SATA hard drive? Here are the same numbers from the SATA drive:


Well, we can see that when it comes to the Sequential Output numbers, the local SATA wins in all three of the numbers and the CPU utilization is exactly the same on both of the systems. It's the same story for the Sequential Input numbers, although there doesn't seem to be as much of a difference as with the Sequential Output numbers.

In the Random Seeks, the AWS machine blows away the local SATA drive, both in terms of Random seeks per second and the latency numbers. The only place where it loses out is in consuming much more CPU than the local system.

In the Sequential Create category, AWS also comes out on top, although, for some reason I wasn't able to get Read and Delete numbers for the local disk.

The Random Create category is pretty much even, with neither system really standing out.

So, there you have it, the AWS infrastructure isn't as bad as I had originally thought. Also, one thing to keep in mind is that the type of EC2 instance tested was just the basic "free" instance which Amazon provides. I would expect that better provisioned instances would perform even better when it comes to the same test.

I/O Benchmarks with Bonnie++

So, after my last article using the palimsest application, I decided to give a tool called Bonnie++ a try. From the description on the authors' website:

"Bonnie++ is a benchmark suite that is aimed at performing a number of simple tests of hard drive and file system performance. Then you can decide which test is important and decide how to compare different systems after running it. I have no plans to ever have it produce a single number, because I don't think that a single number can be useful when comparing such things."

So, after running the default program as provided by Ubuntu repositories, we get the following output:

$ bonnie++
Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version 1.96 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
srdan-desktop 8G 505 98 83083 10 36609 4 2514 86 96195 5 155.8 3
Latency 16674us 1003ms 1830ms 41497us 269ms 1014ms
Version 1.96 ------Sequential Create------ --------Random Create--------
srdan-desktop -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 11634 22 +++++ +++ 17458 25 16959 30 +++++ +++ 17864 23
Latency 55341us 601us 638us 753us 43us 123us
1.96,1.96,srdan-desktop,1,1308644057,8G,,505,98,83083,10,36609,4,2514,86,96195,5,155.8,3,16,,,,,11634,22,+++++,+++,17458,25,16959,30,+++++,+++,17864,23,16674us,1003ms,1830ms,41497us,269ms,1014ms,55341us,601us,638us,753us,43us,123us

So, overall we have a lot more detail, although not as many pretty graphs as we had with Palimpsest. Also, I'm afraid my blogger skills have let me down, because the output above actually looks a lot better from the command line, but I can't for the life of me figure out how to retain tabs when posting.

Looking at the output we can see that the tool tests five main categories:

* Sequential Output (Per Chr, Block, Rewrite)
* Sequential Input (Per Chr, Block)
* Random Seeks
* Sequential Create (Create, Read, Delete)
* Random Create (Create, Read, Delete)

According to the documentation the first three categories simulate the same kind of I/O load that a database would normally put onto a disk, while the last two categories simulate the kind of load that you would expect to see in an NNTP and/or web caching server. From what I could tell, the first three tests all create a single file to test on and the last two create a host of small files.

One thing to note is that this is a test of both the disk and the filsystem (and kernel), unlike the palimpsest benchmark, which only tests the disk. Another thing to note is that as well as getting the I/O throughput we're also seeing the %CP figure, i.e. how taxing the operation is on the CPU. This might be an important factor, when trying to determine what kind of CPU you need for your web caching server.

Overall, bonnie++ is a very good tool for looking at the holistic performance of any storage system.