Sunday, November 28, 2010

Some Hard Drive and File System benchmark tools

IOMeter

 

iozone

http://www.iozone.org

Document: http://www.iozone.org/docs/IOzone_msword_98.pdf
Manual: http://linux.die.net/man/1/iozone

Read, write, re-read, re-write, read backwards, read strided, fread, fwrite, random read/write, pread/pwrite variants

iozone -a | tee result.txt

iozone supports bunch of command line options. I summarized them in following table

Category Options Note
Auto mode -a: record size 4k - 16M, file size 64k - 512M.
-z: Used in conjunction with -a to test all possible record sizes. (Normally Iozone omits testing of small record sizes for very large files when used in full automatic mode. )
-A: more coverage 		 
Test file -f filename: the name for temporary file under test.
-F fn1 fn2: # of files should be equal to # of processors/threads		  
Output -b filename: output of an Excel compatible file
-R: Generate Excel report. 		 
Record size -r #: record size
-y #: minimum record size for auto mode
-q #: maximum record size for auto mode		  
File size -s #: size of the file to test
-g #: maximum file size for auto mode
-n #: minimum file size for auto mode		  
tests -i #: specifies which test to run
0=write/rewrite, 1=read/re-read, 2=random-read/write, 3=Read-backwards, 4=Re-write-record, 5=stride-read, 6=fwrite/re-fwrite, 7=fread/Re-fread, 8=mixed workload, 9=pwrite/Re-pwrite, 10=pread/Re-pread, 11=pwritev/Re-pwritev, 12=preadv/Re-preadv		 One will always need to specify 0 so that any of the following tests will have a file to measure. This means -I 0 creates files used by following tests.
-i # -i # -i # is also supported so that one may select more than one test.
  -+p percent_reads: the percentage of threads/processes that will perform read testing in the mixed workload test case  
  -+B: sequential mixed workload testing  
throughput tests -t #: Run Iozone in a throughput mode.
-T: Use POSIX pthreads for throughput tests		 This option allows the user to specify how many threads or processes to have active during the measurement.
processes/threads -l #: lower limit on number of processes to run
-u #: upper limit on number of processes to run		  
Timing -c: include close() in timing calculation
-e: include fflush(), fsync() in timing calculation		  
Other control -H #: Use POSIX async I/O with # async operations
-k #: Use POSIX async I/O (no bcopy) with # async operations.
-I: use direct I/O if possible
-m: use multiple buffers internally
-o: Writes are synchronously written to disk
-p: purges the processor cache before each file operation.
-W Lock file when reading or writing.
-K:  Inject some random accesses in the testing. 		 
     

Examples

It's important to use -I option to turn on DIRECT I/O. Otherwise, linux's page caches (buffer cache) may give you ridiculous fast read speed.

  • Auto Mode

    • iozone -a -n 512m -g 1g

  • Single Test

    • Sequential write, Sequential reads
      iozone -r 64k -s 1g -b excel.xls -R -i 0 -i 1 -I
    • Use two processes to test sequential writes, sequential reads
      iozone -r 64k -s 1g -b excel.xls -R -i 0 -i 1 -I -l 2 -u 2
    • Use one and two processes to tests (two runs. In first run, one process is created. In second run, two processes are created)
      iozone -r 64k -s 1g -b excel.xls -R -i 0 -i 1 -I -l 1 -u 2
    • Random writes, Random reads
      iozone -r 64k -s 1g -b excel.xls-R -i 0 -i 1 -K -I

  • Throughput Test

    • iozone -t 2
      roughly equivalent to "iozone -l 2 -u 2"

  • Mixed Workload Test

    • iozone -r 64k -s 1g -b excel.xls -R -i 0 -i 8 -+p 50

Result visualization

./Generate_Graphs  result.txt

It will a directory for each operation (e.g. read, write, fread, fwrite). In each directory, there are two files generated - iozone_gen_out.gnuplot and <operation>.ps (this file is generated after you view the corresponding result using gnuplot).  Under the hood, it uses gnu3d.dem to render the data using Gnuplot after those data files are generated for all tested operations. You can call it directly without regenerating separate data files.

gnuplot gnu3d.dem

There are two more scripts that can be used for visualization - report.pl and iozone_visualization.pl.
When I tried to run them in Linux (using command ./report.pl result.txt), I had following error:

    -bash: ./iozone_visualizer.pl: /usr/bin/perl^M: bad interpreter: No such file or directory

You can use command perl report.pl result.txt to run it successfully.
The solution is to change those two files from dos type to unix type (mainly the new line character conversion).

sudo aptitude install tofrodos
fromdos report.pl
fromdos iozone_visualizer.pl

Now, you should be able to report.pl and iozone_visualization.pl directly. When they are run, a directory named 'report_result' is created. In the directory are Gnuplot scripts (*.do files) and PNG images for all tested operations. Those PNG images are generated by running those Gnuplot scripts. For example, read.png is generated by running "gnuplot read.do". A HTML page (index.html) is generated by iozone_visualization.pl which contains all of those PNG images in the same page.

Bonnie

http://www.textuality.com/bonnie/

Resources

Posted by at
Labels: ,

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)