Home > Workload Solutions > High Performance Computing > White Papers > Dell Validated Design for HPC pixstor Storage—Joint Solution with Kalray > PowerEdge R750 Metadata performance with MDtest using 3 KiB files
This test is almost identical to the previous test, except that we used small files of 3 KiB instead of empty files with the intention that such files fit on the inodes and therefore avoid accesses to other devices to store data. The following command was used to run the benchmark, where the Threads variable is the number of threads used (1 to 512 incremented in powers of 2), and my_hosts.$Threads is the corresponding file that allocated each thread on a different node, using the round-robin method to spread them homogeneously across the 16 compute nodes.
mpirun–-allow-run-as-root -np $Threads–-hostfile my_hosts.$Threads –map-by node–-mca btl_openib_allow_ib 1–-oversubscribe–-prefix /usr/mpi/gcc/openmpi-4.1.2a1 /usr/local/bin/mdtest -v -d /mmfs1/perf/mdtest -P -i 1 -b $Directories -z 1 -L -I 1024 -u -t -F -w 3K -e 3K
Now that some data must be transferred for each file, performance numbers decrease, even files that should fit on the inodes. Stat and read operations showed good numbers, reaching their peak values of 5.9M op/s at 32 threads and 3.2M op/s at 128 threads respectively. Remove operations attained the maximum of 474K op/s at 64 threads and create operations achieved their peak of 283.6K op/s, both at 64 threads. Stat and read operations have more variability, but when they reach their peak value, performance does not drop below 2.7M op/s for stat operations and 1.7M op/s for read operations. Create and remove operations have less variability, both keep increasing until reaching their peak values, slowly decreasing after that.
Because these numbers are for a metadata module with a one PowerEdge R650 NVMe metadata pair, performance increases for each additional PowerEdge R650 NVMe pair, however we cannot assume a linear increase for all operations. Unless the whole file fits inside the inode for such files, data targets on other devices will be used to store small files, limiting the performance.