Validation

Performance analysis and characterization (PAAC) testing on Dell EMC VDI solutions is carried out using a carefully designed, holistic methodology that monitors both hardware resource utilization parameters and end-user experience (EUE) during load-testing. This ensures the optimal combination of EUE and cost-per-user.

Login VSI performance testing process and monitoring

We tested each user load against four runs:

A pilot run to validate that the infrastructure was performing properly and that valid data could be captured.
Three subsequent runs to enable data correlation.

During testing, while the environment was under load, we logged in to a session and completed tasks that correspond to the user workload. This test is subjective, but it helps to provide a better understanding of the EUE in the desktop sessions, particularly under high load. It also helps to ensure reliable data gathering.

To ensure that the user experience was not compromised, we monitored the following important resources:

Compute host servers—Solutions based on VMware vCenter for VMware vSphere gather key data (CPU, memory, disk, and network usage) from each of the compute hosts during each test run. This data is exported to .csv files for single hosts and then consolidated to show data from all hosts. While the report does not include specific performance metrics for the management host servers, these servers are monitored during testing to ensure that they are performing at an expected level with no bottlenecks.

Hardware resources—Resource overutilization can cause poor EUE. We monitored the relevant resource utilization parameters and compared them to relatively conservative thresholds. The thresholds are shown in the following table. They were selected based on industry best practices and our experience to provide an optimal trade-off between good EUE and cost-per-user while also allowing sufficient burst capacity for seasonal or intermittent spikes in demand.

Table 4. Resource utilization parameters
Parameter	Pass/fail threshold
Physical host CPU utilization	85% 1
Physical host memory utilization	85%
Network throughput	85%
Disk latency	20 milliseconds
Login VSI failed sessions	2%

Load generation

Login VSI installs a standard collection of desktop application software, including Microsoft Office and Adobe Acrobat Reader, on each VDI desktop testing instance. It then uses a configurable launcher system to connect a specified number of simulated users to available desktops within the environment. When the simulated user is connected, a login script configures the user environment and starts a defined workload. Each launcher system can launch connections to several VDI desktops (target machines). A centralized management console configures and manages the launchers and the Login VSI environment.

We used the following login and boot conditions:

Users were logged in within a login timeframe of 1 hour.
All desktops were started before users were logged in.

Login VSI workloads

The following table describes the Login VSI workloads that we tested:


Login VSI workload name	Workload description
Task Worker	A light workload that runs fewer applications and starts/stops them less frequently than the other workloads. This results in lower CPU, RAM, and IO usage. The Task Worker workload uses the following applications: Adobe Reader Internet Explorer Microsoft Excel Microsoft Outlook Microsoft Word It also uses native Windows apps (Notepad and 7-Zip) to complete the print and zip actions used by the workload meta functions.
Knowledge Worker	Designed for virtual machines with 2 vCPUs. This workload includes the following activities: Microsoft Outlook—Browse messages. Internet Explorer—Browse websites and open a YouTube style video (480p movie trailer) three times in every loop. Microsoft Word—Start one instance to measure response time and another to review and edit a document. Doro PDF Printer and Acrobat Reader—Print a Word document and export it to PDF. Microsoft Excel—Open a large, randomized sheet. Microsoft PowerPoint—Review and edit a presentation. FreeMind—Run a Java-based Mind Mapping application. Other—Perform various copy and zip actions.

Desktop VM test configurations

The following table summarizes the desktop VM configurations used for the Login VSI workloads that we tested:

Table 5. Desktop VM test configurations
Login VSI workload	vCPUs	ESXi configured memory	ESXi reserved memory	Screen resolution	Operating system
Task Worker	8	32 GB	32 GB	1280 x 720	Windows Server 2019 64-bit
Knowledge Worker	2	4 GB	2 GB	1920 x 1080	Windows 10 Enterprise 64-bit

Summary of test results

The following table summarizes the host utilization metrics for the different Login VSI workloads that we tested, and the user density derived from Login VSI performance testing:

Table 6. Summary of test results
Server configuration	Login VSI workload	Operating system	User density	Average CPU	Average GPU	Average active memory	Average IOPS per user	Average network Mbps per user
Density Optimized	Task Worker	Server 2019, 1809	370	85%	N/A	96 GB	1.84	0.73 Mbps
Density Optimized	Knowledge Worker	Windows 10, 21h1	240	85%	N/A	281 GB	10.23	1.97 Mbps

Note: VM densities in excess of 200 were used for benchmarking purposes only. A limit of 200 VMs per vSAN-enabled host should be used in a production environment. For additional information, see VMware Configuration Maximums.

The host utilization metrics mentioned in the table are defined as follows:

User density—The number of users per compute host that successfully completed the workload test within the acceptable resource limits for the host. For clusters, this number reflects the average of the density achieved for all compute hosts in the cluster.
Average CPU—The average CPU usage over the steady state period. For clusters, this number represents the combined average CPU usage of all compute hosts. On the latest Intel processors, the ESXi host CPU metrics exceed the rated 100 percent for the host if Turbo Boost is enabled, which is the default setting. An additional 35 percent of CPU is available from the Turbo Boost feature, but this additional CPU headroom is not reflected in the VMware vSphere metrics where the performance data is gathered.
Average active memory—For ESXi hosts, the amount of memory that is actively used, as estimated by the VMKernel based on recently touched memory pages. For clusters, this is the average amount of physical guest memory that is actively used across all compute hosts over the steady state period.
Average IOPS per user—IOPS calculated from the average cluster disk IOPS over the steady state period divided by the number of users.
Average network usage per user—Average network usage on all hosts calculated over the steady state period divided by the number of users.

1 The Dell Technologies Validated Design for VDI team recommends that average CPU utilization not exceed 85 percent in a production environment. A 5 percent margin of error was allocated for this validation effort. Therefore, CPU utilization sometimes exceeds our recommended percentage. Because of the nature of Login VSI testing, these exceptions are reasonable for determining our sizing guidance.