Dell EMC can help you define an entry-level cluster that can scale as the business grows while you control your present capital and operating costs. Not all container ecosystems need hundreds of servers and no single cluster size fits all situations and circumstances. Customers who are new to containerization ask for a minimum platform configuration. A typical entry-level production platform in the corporate and enterprise markets has between 10 and 30 compute nodes. Large industrial-grade container ecosystems require several full racks of servers per cluster or multiple clusters per data center.

Dell EMC PowerEdge servers

This solution design uses PowerEdge R640 and PowerEdge R740xd servers for compute and storage.

PowerEdge R640 servers

The PowerEdge R640 general-purpose platform supports up to 7.68 TB of memory and twelve 2.5-in. drives and provides flexible I/O options. It is a dual-socket, 1U platform that is ideal for dense scale-out data center computing.

The PowerEdge R640 platform

features:

• Second-generation Intel Xeon Scalable processor product family (with up to 28 cores and two threads per core)
• Up to six DDR4 memory channels with two DIMMs per channel per CPU and 24 DIMMs (supports DDR4 RDIMM/LRDIMM/ NVDIMM-N/DCPMM)
• PCI Express (PCIe) 3.0 enabled expansion slots (with up to 48 lanes per CPU)

PowerEdge R640 servers are preferred for the CSAH and control plane node roles because the needs of these node types are easily accommodated in this 1U node configuration. We recognize that customers might prefer to use identical server configurations for all nodes in their cluster and might therefore choose PowerEdge R740xd servers instead.

PowerEdge R740xd servers

Dell EMC PowerEdge R740 and R740xd are two socket, 2U rack servers designed to run complex workloads using highly scalable memory, I/O capacity, and network options. The R740 and R740xd features the 2nd Generation Intel Xeon Scalable processor family, up to 24 DIMMs, PCI Express (PCIe) 3.0 enabled expansion slots, and a choice of network interface technologies to cover NIC and rNDC. The PowerEdge R740 general-purpose platform is capable of handling demanding workloads and applications such as data warehouses, e-commerce, databases, and high-performance computing (HPC). The PowerEdge R740xd platform adds exceptional storage capacity options, making it well-suited for data- intensive applications that require greater storage while not sacrificing I/O performance.

Dell EMC PowerSwitch S series switches

PowerSwitch S series switches provide the architectural agility and flexibility that are ideal for Ready Stack for Red Hat OpenShift Container Platform 4.2. This Ready Stack design uses the following switches:

• Data network—PowerSwitch S5200 series open networking (ON) switches (25/40/50/100 GbE)
• OOB management—PowerSwitch S3048-ON switch (1 GbE)

For more information, see Chapter 3 Networking Infrastructure and Configuration.

OpenShift Container Platform 4.2 can host the development and run-time execution of containerized applications, sometimes called “container workloads.” The platform uses the Kubernetes container orchestration toolchain that is core to modern automation container deployment, scaling, and management. OpenShift Container Platform 4.2 is designed to meet exacting demand-driven, scale-out capabilities for workloads. We expect the software platform to continue to mature and to expand rapidly, ensuring continued access to the tools you need to grow your business.

OpenShift Container Platform 4.2 is supported on Red Hat Enterprise Linux (RHE L) 7.6 as well as Red Hat Enterprise Linux CoreOS (RHCOS) 4.2. The OpenShift Container Platform 4.2 control plane can be deployed only on RHCOS. The control plane is hosted on control plane nodes. Either RHEL 7.6 or RHCOS can be deployed on compute nodes, known as worker nodes. Red Hat Enterprise Linux version 8 is not yet supported in OpenShift Container Platform.

Version 4.2 compared with version 3.11

Differences between OpenShift Container Platform 3.11 and OpenShift Container Platform 4.2 include:

• Separate infrastructure nodes have been deprecated: etcd is always on the cluster that is itself running on OpenShift Container Platform 4.2 control plane nodes.
• The web console has been significantly updated.
• RHCOS has replaced Atomic host.
• CRI-O is the new container run-time engine, replacing the Docker Container Engine in OpenShift Container Platform 3.11.
• Several CLI commands have changed.
• The Quay application has been introduced as the enterprise container registry.
• CoreDNS has replaced dnsmasq.
• Operator Lifecycle Manager (OLM) has replaced the OpenShift Service Broker and Service Catalog.

New features and enhancements

This section further describes the new features and enhancements in OpenShift Container Platform 4.2.

Operators

OpenShift Container Platform 4.x introduced an operator framework to replace much of the functionality that was previously available with Helm and Helm Charts. An operator is a method by which Kubernetes-native applications are packaged and deployed into the Kubernetes run-time environment. An operator provides a key method for management of repetitive Kubernetes functional operations.

Operator Lifecycle Manager

The functions that OLM supports include:

• Installing, upgrading, and granting access to operators running on their cluster
• Selecting from a catalog of curated operators, with the ability to load other operators into the cluster
• Performing rolling updates of all operators to new versions
• Implementing role-based access control (RBAC) that allows specific teams to use specific operators

For more information, see Understanding the Operator Lifecycle Manager in the Red Hat OpenShift documentation.

Installation and upgrade

We deployed OpenShift Container Platform 3.11 using the openshift-ansible tool. OpenShift Container Platform 4.2 uses ignition-based deployment, a new approach to getting your Kubernetes cluster operational quickly and simply. The ignition-based deployment tool is called openshift-install.

The ignition-based installation method supports two modes of deployment: installer-provisioned infrastructure and user-provisioned infrastructure.

For bare-metal deployment, the Dell EMC Ready Stack deployment process uses the User Provisioned Infrastructure (UPI) method. The openshift-install tool requires very few install-time configuration settings. A post-installation Customer Resource Definition (CRD) facility is used to specify run-time configuration settings.

Over-the-air upgrades for asynchronous z-stream releases of OpenShift Container Platform 4.x are available. Cluster administrators can perform an upgrade by using the Cluster Settings tab in the web console. Updates are mirrored to the local container registry before being pushed to the cluster.

Currently, no facility exists for performing an in-place upgrade of an OpenShift 3.11 cluster to OpenShift 4.2. You must redeploy the cluster to use OpenShift 4.2. After deployment, OpenShift 4.2 is capable of automatic updating, and it will likely be possible to enable automatic upgrading to later releases. Red Hat is developing tooling to enable migration of OpenShift 3.7 and later clusters to OpenShift 4.2. For more information, see this Red Hat documentation.

OperatorHub

OperatorHub helps administrators discover and install optional components and applications. It supports add-on tools and utilities from Red Hat, Red Hat partners, and the open source community.

Storage

OpenShift Container Platform 4.2 provides support for CSI 1.0, the container storage operator, and for the manila-provisioner/operator and snapshot operator.

Red Hat has added many other capabilities to the OpenShift Container Platform to make your container development process easier and more agile and to simplify deployment and management operations in production. For more information, see Understanding persistent storage in the OpenShift documentation.

Architecture

OpenShift Container Platform 4.2 introduces the three basic host types that make up every cluster: the bootstrap node, control plane nodes, and worker nodes.

The deployment process also requires a node called the Cluster System Admin Host (CSAH), but this node is not mentioned in Red Hat online documentation. The CSAH node is not part of the cluster but is required for OpenShift cluster administration. While you could log in to a control plane node to manage the cluster, this practice is not recommended. Although the OpenShift CLI administration tools are deployed onto the control plane nodes, the authentication tokens that are needed to administer the OpenShift cluster are installed only on the CSAH node as part of the deployment process.

Note: Control plane nodes are deployed using an immutable infrastructure, further driving the preference for an administration host that is external to the cluster.

Dell EMC recommends provisioning a dedicated host for administration of the OpenShift cluster. After the cluster is installed and started, the bootstrap node is repurposed as a worker node.

Bootstrap node

When your CSAH node is operational, installation of the cluster begins with the creation of a bootstrap node. This node is needed only during the bring-up phase of OpenShift cluster installation. When the initial minimum cluster—the control plane nodes and at least two worker nodes—is operational, you can redeploy the bootstrap node as a worker node. The bootstrap node is necessary to create the persistent control plane that is managed by the control plane nodes.

Control plane nodes

Nodes that run the control plane are referred to as “control plane nodes.” Three control plane nodes are required to control the operation of a Kubernetes cluster. In OpenShift Container Platform, these nodes are responsible for all control plane operations. The control plane operates outside the application container workloads and is responsible for ensuring the overall continued viability, health, and integrity of the container ecosystem.

Control plane nodes operate outside the MachineType framework. They consist of machines that provide an API for overall resource management. Control plane nodes cannot be removed from a cluster. These nodes provide HAProxy services and run etcd, the API server, and the Controller Manager Server.

Worker nodes

In an OpenShift Kubernetes-based cluster, all application containers are deployed to run on worker nodes. Worker nodes advertise their resources and resource utilization so that the scheduler can allocate containers and pods to worker nodes and maintain a reasonable workload distribution. The CRI-O Kubelet service runs on each worker node. This service receives container deployment requests and ensures that they are instantiated and put into operation. The Kubelet service also starts and stops container workloads. In addition, this service manages a service proxy that handles communication between pods that are running across worker nodes.

Logical constructs called MachineSets define worker node resources. MachineSets can be used to match requirements for a pod to direct deployment to a matching worker node. OpenShift Container Platform supports defining multiple machine types, each of which defines a worker node target type. A future release of OpenShift Container Platform will support specifically classified worker node types, such as AI hosts, infrastructure hosts, NFV hosts, and more.

Worker nodes can be added to or deleted from a cluster as long as the viability of the cluster is not compromised. A minimum of two viable worker nodes must be operating at all times. Further, sufficient compute platform resources must be available to sustain the overall cluster application container workload.

Deployment process

Dell EMC has simplified the process of bootstrapping your first OpenShift Container Platform 4.2 cluster. To use the simplified process, ensure that your rack has been provisioned with suitable network switches and servers, that network cabling has been completed, and that Internet connectivity has been provided to the rack. Internet connectivity is necessary for the installation of OpenShift Container Platform 4.2.

The deployment procedure begins with initial switch provisioning. This step enables preparation and installation of the CSAH node, which includes:

• Installation of Red Hat Enterprise Linux 8
• Subscription to necessary repositories
• Creation of an Ansible user account
• Cloning of a GitHub Ansible playbook repository from the Dell ESG container repository
• Running an Ansible playbook to initiate the installation process

Dell EMC has generated Ansible playbooks that fully prepare the CSAH node. Before installation of the OpenShift Container Platform 4.2 cluster begins, the Ansible playbook sets up a PXE server, DHCP server, DNS server, and HTTP server. The playbook also creates the ignition files that you need to drive your installation of the bootstrap, control plane, and worker nodes, and it configures HAProxy so that the installation infrastructure is ready for the next step. The Ansible playbook presents a list of node types that must be deployed in top-down order.

The Ansible playbook creates an installconfig file that is used to control deployment of the bootstrap node. The following figure shows the workflow to generate the installconfig file:

Figure 1.                  Generating the installconfig file

An ignition configuration control file starts the bootstrap node, as shown in the following figure:

Figure 2.                  OpenShift Container Platform 4.2 installation workflow: Creating the bootstrap, control plane, and worker nodes

Note: Ignition configuration-driven installation generates security certificates that expire after 24 hours. The cluster must be completely installed before the certificates expire. The cluster must operate in a viable (nondegraded) state so that the first certificate rotation can be completed.

The cluster bootstrapping process involves the following phases:

1. Once booted, the bootstrap node creates the resources that are needed to start the control plane nodes. Do not interrupt this process.

2. The control plane nodes pull resource information from the bootstrap node to bring them up into a viable state. This resource information is used to form the etcd control plane cluster. For more information, see Operating etcd clusters for Kubernetes.
3. The bootstrap node instantiates a temporary Kubernetes control plane that is under etcd control.
4. A temporary control plane loads the application workload control plane to the control plane nodes.
5. The temporary control plane is shut down, handing control over to the now viable control plane operating on the control plane nodes.
6. OpenShift Container Platform components are pulled into the control of the control plane nodes.
7. The bootstrap node is shut down.

The control plane node (control plane) now drives creation and instantiation of the worker nodes.

8. The control plane adds operator-based services to complete the deployment of the OpenShift Container Platform ecosystem.

Your cluster is now viable and can be placed into service in readiness for Day-2 operations. You can expand the cluster by adding worker nodes.