Home > Storage > PowerScale (Isilon) > Industry Solutions and Verticals > Analytics > Dell Technologies Solution: Distributed Deep Learning Infrastructure for Autonomous Driving > Data Transform
Training neural networks with images requires developers to first normalize those images. Moreover, images are often compressed to save storage. Developers have therefore built multi-stage data processing pipelines that include loading, decoding, cropping, resizing, and many other augmentation operators. Here are some key design considerations for data ingestion and data management for an ADAS DL workflow:
Offline augmentation is to create a new augmented data which stored on storage. This can help effectively increasing the training sample size many times over with variety of different augmentation technologies.
Online augmentation is to apply augmentation to data in real time before the images are fed to the neural network. CPUs are used heavily in background / parallel for online augmentations during the training.
Easy-to-use Python API
Transparent scaling across multiple GPUs
Accelerated image classification (ResNet-50), object detection (SSD) workloads and speech recognition models such as Jasper and RNN-T.
Flexible graphs let developers create custom pipelines
Supports multiple data formats - LMDB, RecordIO, TFRecord, COCO, JPEG, wav, flac, ogg,
H.264 and HEVC
Developers can add custom audio image and video processing operators For more information, refer to NVIDIA blog.