Modern applications—primarily microservices that are bundled with their dependencies and configurations—are increasingly being built using container technology. Kubernetes, also known as K8s, is an open-source platform for deploying and managing containerized applications at scale. Google open-sourced the Kubernetes container orchestration system in 2014.

The following figure shows the Kubernetes architecture:

Figure 3.                  Kubernetes architecture

For additional information about Kubernetes components and concepts, see Kubernetes Components.

Kubernetes features for container orchestration at scale include:

• Autoscaling, replication, and recovery of containers
• Intracontainer communication, such as IP sharing
• A single entity—a pod—for creating and managing multiple containers
• A container resource usage and performance analysis agent, Container Advisor (cAdvisor). cAdvisor provides container users an understanding of the resource usage and performance characteristics of their running containers. It is a daemon that collects, aggregates, processes, and exports information about running containers.
• Network pluggable architecture
• Load balancing
• Health check service

In a simulated development and testing scenario in Use Case 2, we used the Kubernetes container orchestration system to deploy two Docker containers in a pod.

Kubernetes clusters

A Kubernetes cluster consists of at least one cluster master and multiple worker machines called nodes. These master and node machines run the Kubernetes orchestration system. A cluster is the foundation of the Kubernetes objects including the containerized Oracle database application, all running within a cluster (Figure 12). The Kubernetes cluster has the following components:

• Load balancer—One NGINX load balancer runs on a dedicated Red Hat Enterprise Linux VM (along with a Docker registry), with VMware Fault Tolerance (VMware FT) enabled for the VM.
• Local Docker registry—One Docker registry container is deployed on the load balancer VM for simplicity. For better control and security, deploy the local Docker registry on its own dedicated VM or VMs that are configured with HA.
• Kubernetes master node—One dedicated Oracle Red Hat Enterprise Linux VM provides HA for Kubernetes master nodes if there is a failure. The etcd is deployed on the master node. You can also deploy separate etcd cluster nodes in their own dedicated VM. Figure 12 depicts the architecture.
• Kubernetes worker nodes—One dedicated Oracle Linux VM works as workload driver nodes. We put the databases and the customer Data Cluster pods on these VMs. Figure 10 depicts the details.

Kubernetes Container Storage Interface specification

The Kubernetes CSI plug-in implements the Container Storage Interface protocol, which enables containerized applications in Kubernetes clusters to use block storage. To address the challenges of persistent storage, PowerFlex provides its unique CSI plug-in. The CSI plug-in for PowerFlex enables our customers to deliver persistent storage for container-based applications on premises, for both development and production scale.

The Kubernetes CSI specification was developed as a standard for exposing arbitrary block and file storage systems to containerized workloads through an orchestration layer. Kubernetes previously provided a powerful volume plug-in that was part of the core Kubernetes code and shipped with the core Kubernetes binaries. Before the adoption of CSI, however, adding support for new volume plug-ins to Kubernetes when the code was “in-tree” was challenging. Vendors who wanted to add support for their storage system to Kubernetes, or even to fix a problem in an existing volume plug-in, were forced to align with the Kubernetes release process. Also, third-party storage code can cause reliability and security issues in core Kubernetes binaries. The code was often difficult—or sometimes impossible—for Kubernetes maintainers to test and maintain.

The adoption of the CSI specification makes the Kubernetes volume layer truly extensible. Using CSI, third-party storage providers can write and deploy plug-ins to expose new storage systems in Kubernetes without ever having to touch the core Kubernetes code. This capability gives Kubernetes users more storage options and makes the system more secure and reliable. Our Use Case 2 highlights these advantages by using the Dell EMC CSI Driver for Dell EMC PowerFlex to show the benefits of Kubernetes storage automation.

Figure 4.                  PowerFlex and Kubernetes architecture with the CSI driver

Kubernetes automation with the Dell EMC PowerFlex CSI driver

The PowerFlex CSI driver enables customers to automate storage activities while using Kubernetes. Capabilities include:

• Persistent volume (PV) actions—Create, list, delete, and create from a snapshot
• Dynamic volume provisioning—Create persistent volumes on demand without any manual steps
• Snapshot capabilities—Create, delete, and list

Volume prefixes enable LUN identification. For persistent volumes, the CSI plug-in supports both the ext4 and xfs file systems on worker nodes. The GitHub dell/csi-vxflexos page provides installation details and a download link for the latest PowerFlex CSI driver. You can also download the driver product guide from GitHub.

Note: Dell Technologies also offers CSI drivers on GitHub for the following systems:
• Dell EMC XtremIO
• Dell EMC PowerScale
• Dell EMC PowerMax
• Dell EMC Unity

Kubernetes implementations

Kubernetes is an open-source container orchestration system. Dell Technologies is a platinum member of the Cloud Native Computing Foundation (CNCF), which supports ongoing Kubernetes development. Companies such as VMware, Red Hat, and Canonical have created their own supported Kubernetes versions that are based on the common open-source version. In the use cases that we describe in this white paper, we used open-source Kubernetes because of its capability to run anywhere, to cover the broadest number of designs. For example, key supported platforms include most versions of Linux and clouds like Google GCP, Amazon AWS, and Microsoft Azure. There is no support cost for open-source Kubernetes, which is supported by the Kubernetes community; however, customers needing enterprise support must explore other versions.

VMware Enterprise PKS, which was jointly developed by VMware and Pivotal, is an enterprise implementation of Kubernetes with deep NSX-T integration and a built-in private registry. The commitment of VMware and Pivotal to support upstream Kubernetes means that customers can get a new version of PKS within weeks of a new Kubernetes release. For customers that have standardized on VMware vSphere, PKS is a natural extension of the platform.

Red Hat OpenShift is a platform for managing containers across on-premises data centers and clouds such as Azure Red Hat OpenShift. Red Hat OpenShift is part of the CNCF Certified Kubernetes program, ensuring compatibility for your container workloads. Ease of installation, a focus on security, and enterprise support make OpenShift a popular choice. The Dell Technologies Solutions Info Hub for the Red Hat OpenShift Container Platform has a library of related technical guides and papers.

Canonical offers a pure upstream Kubernetes platform for managing containers across a wide range of clouds, including all major public clouds, and in private data centers for both bare-metal and virtualized infrastructure. Canonical also offers enterprise support for Kubernetes on Ubuntu for public clouds, VMware, OpenStack, and bare metal.