As mentioned above, a Kubernetes deployment requires several server nodes to accommodate the master and worker roles and redundancy for each. By default, it might be assumed that these nodes are equal in platform and hardware specifications. However, Kubernetes can accommodate a mixed or hybrid environment and that was leveraged in a couple of different ways in this study.
In this environment, the control-plane/master nodes were run as virtual machines running on VMware ESXi 7.0u3, some worker nodes were also virtual machines, and the worker nodes used for our test workload were running on physical PowerEdge servers. Because the control-plane is not resource intensive, especially on smaller clusters, this enabled efficient use of resources by allowing the control plane to run in a shared environment rather than dedicating physical machines.
To direct our test workload to run on physical hosts, we leveraged the capabilities of Kubernetes and PowerStore CSI. Kubernetes architecture uses labels and selectors for controlling orchestration and objects within the environment. The PowerStore CSI driver leverages this architecture, allowing a node selector label to be used. This was employed by first labeling our cluster nodes that were physical and configured for Fibre Channel storage connections to PowerStore with the label “fc=true”.
Next, the PowerStore CSI driver configuration is configured with the following:
nodeSelector:
fc: “true”
This configuration ensures that the CSI driver will only run on nodes with the specified label. Because the Storage Class that references this will only be available on those nodes, Kubernetes will only consider these nodes for scheduling workloads that require the PowerStore storage classes. In this manner we have selectively marked specific nodes in our mixed cluster for the Arc-enabled SQL MI workload. The output from kubectl get nodes –-show-labels is shown in the following table, with the additional labels shown in bold on node4, node5, and node6.
Table 6. Kubernetes nodes and labels
node1 | Ready | control-plane, master | 37d | v1.23.5 | beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=node1,kubernetes.io/os=linux,node-role.kubernetes.io/control-plane=,node-role.kubernetes.io/master=,node.kubernetes.io/exclude-from-external-load-balancers= |
node2 | Ready | control-plane, master | 37d | v1.23.5 | beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=node2,kubernetes.io/os=linux,node-role.kubernetes.io/control-plane=,node-role.kubernetes.io/master=,node.kubernetes.io/exclude-from-external-load-balancers= |
node3 | Ready | <none> | 37d | v1.23.5 | beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=node3,kubernetes.io/os=linux |
node4 | Ready | <none> | 37d | v1.23.5 | beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,csi-powerstore.dellemc.com/xxx.xxx.xxx.xxx-nfs=true,csi-powerstore.dellemc.com/xxx.xxx.xxx.xxx-fc=true,csi-powerstore.dellemc.com/xxx.xxx.xxx.xxx-nfs=true,fc=true,kubernetes.io/arch=amd64,kubernetes.io/hostname=node4,kubernetes.io/os=linux |
node5 | Ready | <none> | 36d | v1.23.5 | beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,csi-powerstore.dellemc.com/xxx.xxx.xxx.xxx-nfs=true,csi-powerstore.dellemc.com/xxx.xxx.xxx.xxx-fc=true,csi-powerstore.dellemc.com/xxx.xxx.xxx.xxx-nfs=true,fc=true,kubernetes.io/arch=amd64,kubernetes.io/hostname=node5,kubernetes.io/os=linux |
node6 | Ready | <none> | 36d | v1.23.5 | beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,csi-powerstore.dellemc.com/xxx.xxx.xxx.xxx-nfs=true,csi-powerstore.dellemc.com/xxx.xxx.xxx.xxx-fc=true,csi-powerstore.dellemc.com/xxx.xxx.xxx.xxx-nfs=true,fc=true,kubernetes.io/arch=amd64,kubernetes.io/hostname=node6,kubernetes.io/os=linux |
