Home > Workload Solutions > High Performance Computing > White Papers > Dell Validated Design for HPC pixstor Storage—Joint Solution with Kalray > PowerEdge R650 metadata performance with MDtest using 3 KiB files
This test is almost identical to the previous test, except that we used small files of 3 KiB instead of empty files with the assumption that the files fit on the inodes and therefore avoid accessing other devices to store data. The following command was used to run the benchmark, where the Threads variable is the number of threads used (1 to 512 incremented in powers of 2), and my_hosts.$Threads is the corresponding file that allocated each thread on a different node, using the round-robin method to spread them homogeneously across the 16 compute nodes.
mpirun --allow-run-as-root -np $Threads --hostfile my_hosts.$Threads –map-by node --mca btl_openib_allow_ib 1 --oversubscribe --prefix /usr/mpi/gcc/openmpi-4.1.2a1 /usr/local/bin/mdtest -v -d /mmfs1/perf/mdtest -P -i 1 -b $Directories -z 1 -L -I 1024 -u -t -F -w 3K -e 3K
Now that some data must be transferred for each file, performance numbers decrease, even though the files should fit on the inodes. Stat and read operations showed good numbers, reaching their peak values at 256 threads with 9.2M op/s and approximately 4M op/s respectively. Remove operations attained the maximum of 431 K op/s and create operations achieved their peak of 211 K op/s, both at 64 threads. Stat and read operations have more variability, but when they reach their peak value, performance does not drop below 4M op/s for stat operations and 2.4M op/s for read operations. Create and remove operations have less variability, both keep increasing until reaching their peak values, and then slowly decreasing after.
Because these numbers are for a metadata module with a single PowerEdge R650 NVMe metadata pair, performance increases for each additional PowerEdge R650 NVMe pair, however we cannot assume a linear increase for all operations. Unless the whole file fits inside the inode for such files, data targets on other devices are used to store small files, limiting the performance to some degree.