The final step required to complete the DBaaS platform buildout in the lab was to deploy the Azure Arc-enabled data services data controller into dbaas-databases-1. The data controller provided the ability for IT administrators, DBAs, and software developers to request Azure Arc-enabled SQL Managed Instances in a self-service manner. From there, additional automation could be written to provision SQL databases into the SQL MIs. For example, software developers could include SQL database instantiation as part of a microservices-based application’s continuous deployment pipeline. The following figure depicts many key concepts of the Azure Arc-enabled data services architecture.
Figure 15. Azure Arc-enabled data services logical architecture
The data controller is the orchestrator in the Azure Arc-enabled data services architecture. The data controller is a collection of Kubernetes resources that provide services that are related to provisioning, elasticity, recoverability, monitoring, and high availability. The bootstrapper pod plays an important role in the creation of new Azure Arc-enabled SQL Managed Instances. The bootstrapper instructs AKS hybrid to create Kubernetes resources like pods, services, and persistent volumes by communicating with the Kubernetes API. In short, the data controller shields IT staff and developers from the complexities of Kubernetes using automation.
Two types of connectivity modes are available for the data controller:
The data controller was deployed in the lab in direct connectivity mode using the Azure CLI. See Appendix A for the listing of specific commands used to perform the deployment. We chose direct connectivity mode to provide the most self-service experience for the various consumers of the DBaaS platform. For example, software developers would appreciate the deployment automation available using the Azure CLI and Azure Resource Manager APIs. Some IT and DBAs might also appreciate the ability to provision the SQL MIs through the Azure portal. Azure Active Directory and Azure Role-Based Access Control can also be used in the directly connected mode only because there is consistent connectivity with Azure.
Note: Currently, you can deploy only one data controller per AKS hybrid workload cluster. Future releases of Azure Arc-enabled data services provide the opportunity for more than one data controller in a workload cluster.
During our testing, we had to upgrade the data controller once because a new version became available from Microsoft. In direct connectivity mode, we accomplished this using the Azure CLI.
First, we listed all the available upgrades using the Azure CLI. Using the Azure portal, we could only view the latest upgrade version.
az arcdata dc list-upgrades --k8s-namespace arc-services-ns
Then, we ran the following command to perform the upgrade to the latest version:
az arcdata dc upgrade -n dbaasdc01 --resource-group rg-A7525R06C01-dev-ps2 --desired-version v1.15.0_2023-01-10
Figure 16. Data controller upgrade management in the Azure portal
The following table lists the primary tools that we used to perform the deployment, life cycle management, and monitoring Azure Arc data services.
Table 6. Tools used at the Azure Arc data services layer
Tool | Purpose | Version |
Data Controller | Provides consistency across platforms along with a range of management capabilities including provisioning, monitoring, elastic scaling, automated updates, high availability, and backup. | v1.15.0_2023-01-10 |
Azure Data Studio (including required extensions) | Cross-platform database management tool for hybrid cloud environments. The following extensions were also installed: Azure Arc (v0.9.6) and Azure CLI (v0.1.0). | 1.40.2 |
Azure CLI (including required extensions) | Used to interact with Azure Arc data services resources. "extensions": { "arcdata": "1.4.10", "azurestackhci": "0.2.6", "connectedk8s": "1.3.8", "customlocation": "0.1.3", "k8s-configuration": "1.7.0", "k8s-extension": "1.3.7", "resource-graph": "2.1.0" } | 2.44.1 |
Grafana | Grafana is another tool for visualizing metrics, logs, and other data in a wide variety of formats. | Version deployed with data controller |
Kibana | Elasticsearch, Fluent Bit, and Kibana are used for SQL-MI logging. | Version deployed with data controller |
An Azure Arc-enabled SQL MI is the same evergreen Azure SQL MI engine that is compatible with any production release of SQL Server database engine. An Azure Arc-enabled SQL MI combines the best and most widely used RDBMS, SQL Server, with the declarative power of Kubernetes. Many features and services that you already use with SQL Server are ready and available. We used the Azure CLI to create the Azure Arc-enabled SQL Managed Instances.
Two service tiers are available for Azure Arc-enabled SQL Managed Instances:
The general-purpose tier deployed a single pod, Contained Always On Availability Group (CAG) Cluster, which included the master and msdb system databases.
Figure 17. SSMS - General Purpose Contained Availability Group
The business critical tier can automatically deploy up to three replicas as part of the CAG topology. This entire multi-replica CAG deployment is completed in minutes, which includes all the internal load balancer configuration and read-only synchronous configuration replicas. Microsoft does recommend a three-replica configuration to achieve near-zero data loss.
In Management and operations of the AKS hybrid section of this white paper, we discussed using Prometheus and Grafana to monitor and visualize performance date at the workload cluster level. The data controller was also deployed with its own instance of Grafana for visualizing performance data within the SQL MIs. The data controller also has its own instance of Kibana for viewing logging data. The following figure shows where the URLs to Grafana and Kibana can be found in Azure Data Studio.
Figure 18. Azure Arc-enabled SQL Managed Instance monitoring URLs
Here is how the data controller’s Grafana instance visualized one of the SQL MI’s performance metrics.
Figure 19. Data Controller Grafana Instance