Home > Storage > PowerScale (Isilon) > Industry Solutions and Verticals > Analytics > Deep Learning with Dell EMC Isilon > Dataset preprocessing pipeline optimizations
The training performance of the tested neural networks are affected by various flags in the benchmark. It was found that the training performance of Resnet50 could be improved by changing the default value of the flag datasets_num_private_threads to 4. This setting allows image preprocessing, which includes TFRecord parsing, JPEG decoding, distorting, and cropping, to be performed by multiple threads concurrently. The performance comparison between the baseline version and this tuned version for Resnet50 is shown in the following figure. There was no obvious performance improvement for AlexNet. With one compute node (4 GPUs), the performance was improved from 2,657 images/sec to 2,941 images/sec. With two compute nodes (8 GPUs), the performance was improved from 4,560 images/sec to 5,590 images/sec.