Developing and deploying deep learning models doesn’t have to be so complicated
Dell EMC Solution Insight for i-Abra
Thu, 14 Jan 2021 23:54:16 -0000|
Read Time: 0 minutes
Dell and Intel® recently evaluated a solution that greatly simplifies the process of developing and deploying deep learning models. The i-Abra AI system we tested automatically builds an optimized inference classifier trained with a customer supplied data set of labeled Images and deployable on FPGA accelerators – all in a single workflow. In this overview, we describe how the i-Abra Pathworks software works, along with a comparison of using traditional development and deployment practices. Our engineers worked with Pathworks in a lab environment to gain insights on the comparison between i-Abra Pathworks and traditional methods that we also share in this article.
The promise of creating applications that bring to light valuable insights from data is real. The data science community has been very successful over the last 10 years in improving the process of extracting information from data using deep learning (DL) - the practice of training artificial neural networks to solve specific analytics and business-driven problems. In fact, deep learning has become the de facto approach for developing highly accurate models for image classification, object detection, real-time video analytics, and many other problems. These accuracy advancements have however increased the complexity of model development, training, and deployment.
There are many factors that contribute to the overall complexity of developing and using DL models. Apart from the work of data preparation, the deep learning life-cycle consists of two phases: 1) the training phase where a neural network architecture is proposed by the data scientists and trained to perform a task, and 2) the deployment phase where the trained neural network is readied for deployment. Also, the deployment target may be a user-facing software application or embedded in a backend process oftentimes requiring a different hardware system.
During the training stage, the data science team can spend days, weeks, or months painstakingly selecting, crafting, and customizing a neural network architecture that produces the most accurate model for performing the specified task. It is a process that requires data scientists with a combination of training and experience coming from a talent pool that is shockingly small. This is an expensive process in time, money, and people resources that impacts TCO for many systems.
But remember, the training phase is only the first half of the story. In order to get the finalized trained model deployed - the deployment stage – the finalized trained neural network must go through an optimization process for the target user system. A DL model must be quantized (converted to reduced precision mathematical operations), pruned (eliminate any unnecessary neurons and connections), and fused (multiple layers, as well as neuron weights and biases, are frozen and merged together). This preparation phase (of the deployment phase) requires trial and error together with testing that can take as much time as the training stage. Many trained models never make it into production deployment due to insurmountable hurdles associated with getting high-quality DL models into production to support the AI applications that would benefit customers and/or employees. In short, the deployment process is an additional expensive process in time, money, and resources with the potential risk of not achieving production usage. This complexity impacts TCO decisions greatly and can threaten the value of the original investment in training.
There is another way to gain DL productivity while minimizing both the TCO and risk of deployment delays. i-Abra Pathworks is a productivity tool for data scientists that integrates and automates both the training and deployment phases into a single workflow. Pathworks eliminates several of the pain points noted above in the traditional neural network-based data science life cycle. First, instead of relying on a data scientist to select, fine-tune, and craft a neural network architecture during training, i-Abra uses an evolutionary architecture design approach which automatically crafts a custom neural network architecture for the labeled training data being used. That means that your highly trained data scientists can spend more time-solving problems with minimal time lost in the minutia of exploring every hyper-parameter combination while building your next AI model.
Additionally, i-Abra Pathworks performs this evolutionary architecture discovery and training on efficient Intel Field Programmable Gate Array (FPGA) accelerators, the same hardware acceleration that i-Abra deploys for boosting the performance of models in production. This full ecosystem approach, employing the same FPGA technology for both the training and deployment stages, means that the model created during the training stage is the same model used in production deployment. There is no quantization, no pruning, and no fusing to be performed. The model does not need to be converted, translated, or reevaluated since the training optimization produced the most efficient design for the data. You simply push that efficient model directly into production, because the model you deploy is the exact same model that you trained.
With i-Abra Pathworks productivity tools, users provide the labeled images to the process and the result is a deployable model. This change from traditional training and deployment approaches dramatically reduces time, money, and required resources. The result is improved TCO, lower risk, and more successful projects. It also reduces ongoing system complexity, improves overall financial performance, and increased technical agility to deploy newer models as the system needs to change with time.
The very best AI capabilities result from tightly coupled SW and HW to achieve optimal TCO, performance, power, and maintenance of the deployment AI system. For many years, i-Abra and Intel have collaborated through joint investment on the core technology to tightly couple i-Abra SW with Intel Architecture. The i-Abra Pathworks training environment relies on an integrated mesh of multiple Intel Xeon processors and Intel FPGAs for fast trained model generation. The i-Abra Synapse inference environment integrates the deployed runtime AI models on Intel FPGA, Xeon, and Atom processors creating many optimized deployment options. i-Abra GraphDB utilizes Pathworks and Synapse for an additional AI level of insights that incorporate multiple I-Abra Synapse inference classification results with additional metadata for a reasoning neural network built on Intel Architecture. Collectively the i-Abra products are tuned to Intel architecture bringing unique turnkey AI capability to multiple markets uses cases. Dell, i-Abra, and Intel have further jointly collaborated to integrate the i-Abra training capabilities into Dell production class training appliances and server technologies. Dell is also creating tightly integrated, full end to end i-Abra based AI edge and network solutions across multiple markets on Intel architecture.
The Dell EMC HPC & AI Innovation Lab has worked with i-Abra and Intel to develop a model training cluster for the i-Abra ecosystem, ensuring that the i-Abra Pathworks software stack functions in an efficient and performant manner while providing the robustness and reliability of the Dell EMC PowerEdge server and Dell EMC PowerSwitch network switch portfolio.
The model training cluster for i-Abra includes eight (8) Dell EMC PowerEdge R740 servers, each with two (2) second-generation Intel® Xeon® Scalable Gold 6248 processors and 768GB of high-speed DDR4 RAM. Each server also contains two (2) Intel® Programmable Acceleration Cards (PACs) with Intel Arria® 10 GX FPGA, as well as two (2) high capacity hard drives for storing and ingesting labeled training data. The PowerEdge servers are all connected with 10Gbps networking to ensure fast movement of training data and model information exchange during training.
The advantages of deep learning for extracting useful insights from many types of data have been proven across many industries and use cases. One of the most significant roadblocks hindering organizations that want to start or expand their use of deep learning models is too much complexity in moving from the training stage to the deployment stage. The i-Abra Pathworks solution simplifies that transition by optimizing models for deployment during training rather than using a traditional two-stage training followed by a deployment optimization process. The result is a more streamlined workflow with less room for errors and more models successfully deployed and a simplified long-term management experience. Dell and Intel have partnered with i-Abra to develop recommendations for assembling a model training cluster for the i-Abra ecosystem. Our results show that the i-Abra Pathworks software stack functions in an efficient and performant manner while providing the robustness and reliability of the Dell EMC PowerEdge server and Dell EMC PowerSwitch network switch portfolio.
From the core to the edge to the cloud, Dell Technologies solutions are the proven choice for the modern datacenter. Dell Technologies provides the most secure, innovative, and scalable solutions enabling our customers to solve today’s challenges wherever they are in their digital transformation. And that’s why they’re used together at scale, across the globe, more than any other platform. Dell EMC also partners with the largest, most innovative players in information technology (IT) to provide jointly engineered services and solutions that speed deployment, improve performance, and maximize return on investment (ROI).