The testing methodology that is used for this paper considered all the architectural layers of the DBaaS integrated architecture holistically. Rather than stressing an individual SQL MI to its resource limits, the goal was to balance the utilization of the storage, cluster, CPU, and memory. This demonstrates the primary benefits of the DBaaS integrated architecture - rapid deployment, ease of management, and elastic resource scalability while also optimizing cluster resource utilization and delivering a performance that meets SLAs. To generate the appropriate load on the SQL MIs, an industry-accepted pattern of an Online Transaction Processing (OLTP) workload from HammerDB was used. 

A robust set of scripts were used to automate the different benchmark tests and collect their results. These include T-SQL logging and reporting framework to harvest HammerDB transactions per minute (TPM) metrics from a jump host outside of the DBaaS environment. Also, the HammerDB CLI, with the associated. tcl scripting was constructed to manage the autopilot timed execution. Doing so allows for the capture of TPM metrics from a multi-instance SQL workload at SQL MI instance granularity with individualized batch request/second logging.

Note: The test suite that is used is not part of the DBaaS integrated architecture offering and was used internally to perform robust stress and performance tests of the deployment. It allows for testing the deployment, at scale, of multiple SQL MI instances, each running a stress workload, while monitoring all aspects of their database performance and environment.

With an Azure Arc-enabled SQL Server Managed Instance, there is no SQL Server instance to deploy HammerDB against. However, SQL MIs are near 100% compatible with SQL Server database engine. Therefore, the HammerDB workload can run against each SQL MI, where the system databases are used to log metrics to a single table. Then these metrics are pulled back to a master reporting SQL instance for aggregation and review.

A dedicated host (virtual machine) was used to initiate the HammerDB tests. This host also resides on the same PowerFlex cluster.

The HammerDB software and SQLCMD are deployed on the HammerDB host.

The HammerDB host had the following virtual hardware profile and specifications:

• Operating System: Windows 10
• 4 vCPU
• 16 GB memory

The metrics that are described in the testing section below are aggregated TPM numbers for each SQL MI and user count for the entire run duration. The following figure shows the architectural layers of the DBaaS integrated architecture used for the tests.

Figure 3.        The architecture followed in the lab