Dozens or hundreds of iterations are produced as the models are tuned and new datasets are incorporated. Using automation to manage, build, and maintain the stages in a complex ML life cycle reduces the number of steps that must be performed manually, accelerating the ML process and minimizing mistakes.
A typical ML workflow includes the following steps: data acquisition, data analysis, data preparation, data validation, model building, model training, model validation, training at scale, model inference, and monitoring. The following diagram shows an example:
Figure 2. Machine learning workflow
Kubeflow supports the different lifecycle stages of an ML project, integrating commonly used ML tools such as TensorFlow and Jupyter notebooks into a single platform.