AKS-HCI can be installed using PowerShell or Windows Admin Center. We used Windows Admin Center in the lab for ASHCLUSTER. We verified that we met all the requirements in the Quickstart: Set up Azure Kubernetes Service on Azure Stack HCI using Windows Admin Center documentation before stepping through the setup. The automated deployment of AKS-HCI using Windows Admin Center removed all the complexity of a typical Kubernetes deployment using predefined configurations from Microsoft. The deployment process involved the creation of two types of clusters: 

• Management cluster (or AKS host) – The cluster consisted of the core orchestration components of the AKS-HCI service, and it deploys and manages one or more workload clusters. Core components include the API server, which allows for management using tools like Windows Admin Center, PowerShell modules, Azure CLI, and kubectl. A load balancer directs traffic to the API server. The following figure depicts the configuration decisions that are made in the AKS-HCI host workflow.

Figure 7.        Azure Kubernetes Service host configuration

The following figure shows how the Azure Kubernetes Service tool appears in Windows Admin Center after the deployment is complete.

Figure 8.        Successful deployment of AKS-HCI host VM

• Workload clusters (or target clusters) – Software developers deploy and run containerized applications – including the Azure Arc-enabled SQL Managed Instances and SQL databases – in these clusters. The cluster in the lab consisted of Linux VMs that hosted the Kubernetes control plane, which ran an API server and etcd for life cycle management. Another Linux VM ran HAProxy and KeepAlive to provide load-balancing services for the workload cluster. Then, we deployed two Linux-based worker nodes that ran Kubernetes system and user pods – including pods for the Azure Arc-enabled Kubernetes agents, Azure Arc-enabled data services, and all the management and operations capabilities. The following figure depicts configuration choices that we made when deploying the workload cluster.

Figure 9.        Workload cluster configuration

The following figure shows how the Azure Kubernetes tool appears in Windows Admin Center after the workload cluster is successfully deployed.

Note: Azure Arc-enabled Kubernetes agents were installed automatically as part of the deployment using Windows Admin Center.

Figure 10.    Successful deployment of AKS-HCI workload cluster

The following figure presents the logical architecture of the initial setup in the lab before any other resources were deployed.

Figure 11.    AKS-HCI architecture deployed in the lab

The following table summarizes all the configuration details about the initially deployed environment.

Table 4.      AKS-HCI configuration details

The following figure shows the service host and workload clusters appearing in the Azure Arc-enabled Kubernetes blade in the Azure portal. 

Figure 12.    AKS-HCI management and workload clusters in Azure Arc

Management and operations

The following table lists the primary tools that we used to perform the deployment, life cycle management, and monitoring of the AKS-HCI layer. 

Note: For some of the tools listed – especially helm, Azure CLI, and the Azure CLI extensions – we always verified that we were using the latest version that was available.   

Table 5.      Tools used at AKS-HCI layer

We used Azure Arc-enabled Kubernetes to integrate with other Azure services like Azure Monitor Container Insights. This provided us with workload cluster processor, memory, and storage performance data. We used the metrics at the AKS-HCI layer combined with data that was captured from the Azure Arc-enabled SQL Managed Instances to get a holistic view into the overall responsiveness of the SQL databases during the load testing. To set up integration with Azure Monitor Container Insights we checked the prerequisites and installed a new extension to the workload cluster. We already had created a Log Analytics workspace. We had the correct permissions assigned to the Azure subscription and workspace. 

We issued the following Azure CLI commands to install the azuremonitor-containers extension:

# Authenticate to Azure interactively

az login

# Switch to the subscription hosting the Log Analytics workspace

az account set -s <subscription ID>

# Obtain the Log Analytics workspace ID

az resource list --resource-type Microsoft.OperationalInsights/workspaces -o json

# Create extension instance

az k8s-extension create --name azuremonitor-containers --cluster-name aks-lab-workloads-1 --resource-group AksHciResources-rg --cluster-type connectedClusters --extension-type Microsoft.AzureMonitor.Containers --configuration-settings logAnalyticsWorkspaceResourceID="<workspace ID>"

# Verify installState shows Installed for new extension

az k8s-extension show -c aks-lab-workloads-1 --cluster-type connectedClusters -n azuremonitor-containers -g AksHciResources-rg

The following figure shows the aks-lab-workloads-1 cluster appearing in Azure Monitor after we installed the extension.

Figure 13.    Workload cluster in Azure Monitor Container Insights

There was also the option to use open-source software to perform monitoring and logging on-premises. Prometheus is an open-source application used for monitoring and alerting of containerized workloads. Prometheus can be deployed during initial creation of the workload cluster or deployed on an existing workload cluster using a single PowerShell command. Grafana is another tool for visualizing metrics, logs, and other data in a wide variety of formats. Microsoft provides guidance on installing Grafana with AKS-HCI and configuring it to use Prometheus as the data source using a helm chart. 

For on-premises logging, AKS-HCI uses a solution that is based on Elasticsearch, Fluent Bit, and Kibana (EFK). You can find the details for setting up the solution in the AKS-HCI-Apps GitHub repository. We used Azure Monitor Container Insights for monitoring the aks-lab-workloads-1 cluster in the lab.  The commands that we ran to install the open-source tools are listed in Appendix A: On-premises monitoring set-up commands. 

Note: For the EFK installation, we ran the required command in PowerShell 5.1 and not PowerShell Core, as we experienced failures when using Core.