The final step required to complete the DBaaS platform buildout in the lab was to deploy the Azure Arc-enabled data services data controller into dbaas-databases-1. The data controller provided the ability for IT administrators, DBAs, and software developers to request Azure Arc-enabled SQL Managed Instances in a self-service manner. From there, additional automation could be written to provision SQL databases into the SQL MIs. For example, software developers could include SQL database instantiation as part of a microservices-based application’s continuous deployment pipeline. The following figure depicts many key concepts of the Azure Arc-enabled data services architecture.

Figure 15.    Azure Arc-enabled data services logical architecture

The data controller is the orchestrator in the Azure Arc-enabled data services architecture. The data controller is a collection of Kubernetes resources that provide services that are related to provisioning, elasticity, recoverability, monitoring, and high availability. The bootstrapper pod plays an important role in the creation of new Azure Arc-enabled SQL Managed Instances. The bootstrapper instructs AKS hybrid to create Kubernetes resources like pods, services, and persistent volumes by communicating with the Kubernetes API. In short, the data controller shields IT staff and developers from the complexities of Kubernetes using automation.

Two types of connectivity modes are available for the data controller:

• Direct mode – This is for organizations who maintain network connectivity to Azure and prefer to leverage Azure Resource Manager – including Azure portal and Azure CLI – for management of this DBaaS platform. Usage data, metrics, and logs are automatically sent to Azure. We used this connectivity mode in the lab.
• Indirect mode – This is for organizations with limited or no connectivity to Azure because of data sovereignty, data gravity, or other similar security requirements. DBAs use tools such as Azure Data Studio and Azure CLI (az) with the arc data extension to manage the data controller directly. This mode requires manual uploads of usage data, metrics, and logs to Azure using the Azure CLI.

Deploy data controller in direct connectivity mode

The data controller was deployed in the lab in direct connectivity mode using the Azure CLI. See Appendix A for the listing of specific commands used to perform the deployment. We chose direct connectivity mode to provide the most self-service experience for the various consumers of the DBaaS platform. For example, software developers would appreciate the deployment automation available using the Azure CLI and Azure Resource Manager APIs. Some IT and DBAs might also appreciate the ability to provision the SQL MIs through the Azure portal. Azure Active Directory and Azure Role-Based Access Control can also be used in the directly connected mode only because there is consistent connectivity with Azure.

Note: Currently, you can deploy only one data controller per AKS hybrid workload cluster. Future releases of Azure Arc-enabled data services provide the opportunity for more than one data controller in a workload cluster.

During our testing, we had to upgrade the data controller once because a new version became available from Microsoft. In direct connectivity mode, we accomplished this using the Azure CLI.

First, we listed all the available upgrades using the Azure CLI. Using the Azure portal, we could only view the latest upgrade version.

az arcdata dc list-upgrades --k8s-namespace arc-services-ns

Then, we ran the following command to perform the upgrade to the latest version:

az arcdata dc upgrade -n dbaasdc01 --resource-group rg-A7525R06C01-dev-ps2 --desired-version v1.15.0_2023-01-10

Figure 16.    Data controller upgrade management in the Azure portal

Management and operations

The following table lists the primary tools that we used to perform the deployment, life cycle management, and monitoring Azure Arc data services.

Table 6.      Tools used at the Azure Arc data services layer

An Azure Arc-enabled SQL MI is the same evergreen Azure SQL MI engine that is compatible with any production release of SQL Server database engine. An Azure Arc-enabled SQL MI combines the best and most widely used RDBMS, SQL Server, with the declarative power of Kubernetes. Many features and services that you already use with SQL Server are ready and available. We used the Azure CLI to create the Azure Arc-enabled SQL Managed Instances.

Two service tiers are available for Azure Arc-enabled SQL Managed Instances:

• General purpose – This suffices for many workloads requiring standard performance and availability. The AKS hybrid layer provided high availability for workload clusters using a single stateful set and Azure Stack HCI Cluster Shared Volumes (CSV) for persistent storage. If a SQL MI pod failed, the AKS hybrid workload cluster control plane would automatically redeploy the pod. In this case, consumers of that SQL MI would be unable to connect to their SQL databases until the pod was fully recovered.

The general-purpose tier deployed a single pod, Contained Always On Availability Group (CAG) Cluster, which included the master and msdb system databases.

Figure 17.    SSMS - General Purpose Contained Availability Group

• Business critical – The business-critical tier is suitable for databases requiring higher performance, low latency, and the highest level of availability. High availability is achieved by deploying multiple replicas and using Microsoft SQL Always On availability groups. The data controller creates an availability group during the provisioning of the SQL MI pods, and all the SQL databases are automatically added to it. Applications can also benefit from read scale out to offload read-intensive workloads to secondary replicas.

The business critical tier can automatically deploy up to three replicas as part of the CAG topology. This entire multi-replica CAG deployment is completed in minutes, which includes all the internal load balancer configuration and read-only synchronous configuration replicas. Microsoft does recommend a three-replica configuration to achieve near-zero data loss.

In Management and operations of the AKS hybrid section of this white paper, we discussed using Prometheus and Grafana to monitor and visualize performance date at the workload cluster level. The data controller was also deployed with its own instance of Grafana for visualizing performance data within the SQL MIs. The data controller also has its own instance of Kibana for viewing logging data. The following figure shows where the URLs to Grafana and Kibana can be found in Azure Data Studio.  

Figure 18.    Azure Arc-enabled SQL Managed Instance monitoring URLs

Here is how the data controller’s Grafana instance visualized one of the SQL MI’s performance metrics.

Figure 19.    Data Controller Grafana Instance