Home > Workload Solutions > High Performance Computing > White Papers > Enhance Availability of Storage Services with an NFS Storage System > IOzone benchmark
The IOzone benchmark was used to measure sequential read- and write throughput (MB/s) and random read- and write I/O operations per second (IOPS). You can download IOzone from http://www.iozone.org/. Version 3.492 was used for these tests and installed on both the NFS servers and all the compute nodes.
The IOzone tests were run from 1–16 nodes in clustered mode with one thread per node, and then the number of threads per node was increased on all nodes to simulate a higher number of clients. All tests were N-to-N. Meaning, N clients would read or write N independent files.
Between tests, the following procedure was followed to minimize cache effects:
• Unmount NFS share on clients.
• Drop the OS caches on the clients
• Mount NFS share on clients.
IOzone Argument |
Description |
-i 0 |
Write test |
-i 1 |
Read test |
-i 2 |
Random Access test |
-+n |
No retest |
-c |
Includes close in the timing calculations |
-t |
Number of threads |
-e |
Includes flush in the timing calculations |
-r |
Record size |
-s |
File size |
-t |
Number of threads |
+m |
Location of clients to run IOzone when in clustered mode |
-w |
Does not unlink (delete) temporary file |
-I |
Use O_DIRECT, bypass client cache |
-O |
Give results in ops/sec |
For the sequential tests, file size was varied along with the number of clients such that the total amount of data written was 512 GiB (number of clients * file size per client = 512GiB), such that the amount of data used for each test was twice the server RAM to eliminate cache effects on the server. By using -c and -e in the test, IOzone provides a more realistic view of what a typical application is doing.
IOzone Sequential Writes example (paths used may be different in your system)
# /usr/sbin/iozone -i 0 -c –e –w –r 1024k –s 16g –t 32 -+n -+m ./clientlist
IOzone Sequential Reads example (paths used may be different in your system)
# /usr/sbin/iozone -i 1 -c -e -w -r 1024k -s 16g -t 32 -+n -+m ./clientlist
For the random tests, after creating files with sequential transfers, each client read and then wrote randomly 4GiB of data. The record size used for the random tests was 4KiB to simulate small random data accesses.
IOzone file creation before random tests (paths used may be different in your system)
# /usr/sbin/iozone -i 0 -c –e –w –r 1024k –s 4g –t 32 -+n -+m ./clientlist
IOzone IOPs Random Reads and Writes (paths used may be different in your system)
# /usr/sbin/iozone -i 2 -w -r 4k -I -O -w -+n -s 4g -t 32 -+m ./clientlist
The O_Direct command line parameter allows us to bypass the cache on the compute node on which we are running the IOzone thread and since NFS was mounted as sync, it also forced writing to disk each transfer on the server before the operation was completed.
mpirun Argument |
Description |
-np |
Number of concurrent processes |
--map-by node |
Instructs mpirun allocate cores from nodes in a round robin fashion (one per node each time) to distribute evenly the threads among all nodes |
--allow-run-as-root |
Allow mpirun to run when executed by the root user |
--hostfile |
Tells mpirun where the hostfile is located |
MDtest Argument |
Description |
-d testdir[@testdir2] |
The directory in which the tests will run. For multiple paths, it must use fully qualified pathnames. [default: working directory of mdtest]. |
-i iterations |
The number of iterations the test will run [default: 1] |
-b branching factor |
The branching factor of the hierarchical directory structure [default: 1]. |
-z tree depth |
The depth of the hierarchical directory tree [default: 0]. |
-L |
Files/directories only created at the leaf level of the tree. |
-I items_per_directory |
The number of items per directory in the tree [default: 0]. |
-y |
Sync the file to disk after writing. |
-u |
Create a unique working directory for each task (presumes -d option). |
-t |
Include unique working directory management overhead in the results (presumes -u option). |