Choosing a Kubernetes distribution

Deploying open-source Kubernetes is a cost-effective way to deploy your first cluster. It provides the most capabilities for advanced users who are building complex systems of orchestrated containers on top of the orchestration engine. Open-source Kubernetes does not provide a prescriptive definition of all the components that make up an enterprise-class Kubernetes architecture. Instead, it provides a menu of options from which to choose the container storage interface, network overlay, and load balancer. These options enable a diverse set of capabilities but also presents yet another topic for administrators to research before deploying the cluster.

The HPC Engineering team at Dell EMC has been working with customers and partners to bring HPC workloads to Kubernetes as part of Project Argo. Argo is a reference architecture for powering Kubernetes orchestrated workloads in a heterogenous compute environment alongside traditional HPC schedulers such as Slurm. Argo uses Bright Cluster Manager to deploy and then manage the life cycle of Kubernetes nodes. To run Domino Data Science Platform, we can use the open-source Kubernetes architecture of Argo, as shown in the following figure:

Figure 10.       Open-source Kubernetes architecture (workloads shown in top-row boxes)