Your Browser is Out of Date

Nytro.ai uses technology that works best in other browsers.
For a full experience use one of the browsers below
VDI Validation Guide—Citrix Virtual Apps and Desktops on VxRail and vSAN Ready Nodes
Executive Summary Test Environment Configuration and Best Practices Login VSI Performance Testing NVIDIA nVector Performance Testing
nVector performance testing process
nVector performance test results and analysis
Conclusion References Appendix A: Login VSI metrics

nVector performance testing process

Home > Integrated Products > VxRail > Guides > VDI Validation Guide—Citrix Virtual Apps and Desktops on VxRail and vSAN Ready Nodes > nVector performance testing process

nVector performance testing process

  • NVIDIA nVector is a performance testing tool for benchmarking VDI workloads. The nVector tool creates a load on the system by simulating a workload that matches a typical VDI environment. The tool assesses the experience at the endpoint device rather than the response time of the virtual desktop.

    The nVector tool captures performance metrics from the endpoints that quantify user experience, including image quality, frame rate, and user latency. These metrics, when combined with resource utilization information from the servers under test, enable IT teams to assess their VDI graphics-accelerated environment needs.

    We tested multiple runs for each user load scenario to eliminate single-test bias. We used a pilot run to validate that the solution was functioning as expected, and we validated that testing data was being captured. We then tested subsequent runs to provide data that confirmed that the results we obtained were consistent.

    To assess the EUE experience, we logged into a VDI session and completed several tasks that are typical of a normal user workload. This small incremental load on the system did not significantly affect our ability to provide reproducible results. While the assessment undoubtedly is subjective, it helps to provide a better understanding of the EUE under high load. It also helps to assess the reliability of the overall testing data.

    Load generation

    The nVector tool runs the simulated workflow of a typical VDI workload at a predesignated scale. This part of the test requires performance monitoring to measure resource utilization. Acting as an execution engine, nVector orchestrates the following necessary stages that are involved in measuring EUE for a predefined number of VDI instances:

    1. Provision VDI instances with predefined settings such as vCPU, vRAM, and frame buffer, and provision an equal number of VMs that act as virtual thin clients.
    2. Establish remote connections to VDI desktops using virtual clients.
    3. Measure resource utilization statistics on the server and on the guest operating system of the VDI desktop.
    4. Run the designated workload on all the VDI instances.
    5. Collect and analyze performance data and EUE measurements.
    6. Generate a report that reflects the trade-off between EUE and user density (scale).

    The following figure shows the stages in the NVIDIA benchmarking tool's measurement of user experience:

    Figure 39 NVIDIA nVector workflow

    nVector Knowledge Worker workload

    We performed this testing exercise with NVIDIA's nVector Knowledge Worker workload. This workload contains a mixture of typical office applications, including some multimedia usage. This workload is representative of what a typical office worker does during the working day. The activities performed include:

    • Working on Microsoft Excel files
    • Scrolling through PDFs
    • Opening and working on Microsoft Word documents
    • Opening and presenting a Microsoft PowerPoint presentation
    • Opening and viewing web pages and web videos using Google Chrome
    • Opening and closing applications and saving or copying content

    Resource monitoring

    Host metrics

    We used VMware vCenter to gather key host utilization metrics, including CPU, GPU, memory, disk, and network usage from the compute host during each test run. This data was exported to .csv files for each host and then consolidated for reporting.

    Resource overutilization can cause poor EUE. We monitored the relevant resource utilization parameters and compared them to relatively conservative thresholds. The thresholds were selected based on industry best practices and our experience to provide an optimal trade-off between good EUE and cost-per-user while also allowing enough burst capacity for seasonal or intermittent spikes in demand. The following table shows the pass/fail threshold for host utilization metrics:

    Table 12 Resource utilization thresholds
    Parameter Pass/fail threshold
    Physical host CPU utilization 85%
    Physical host memory utilization 85%
    Network throughput 85%
    Disk latency 20 milliseconds
    Physical host CPU readiness 10%

    Measuring EUE

    This section explains the EUE metrics measured by the nVector tool. These metrics include image quality, frame rate, and end-user latency.

    Metric 1: Image quality—NVIDIA nVector uses a lightweight agent on the VDI desktop and the client to measure image quality. These agents take multiple screen captures on the VDI desktop and on the thin client to compare later. The structural similarity (SSIM) of the screen capture taken on the client is computed by comparing it to the one taken on the VDI desktop. When the two images are similar, the heatmap reflects more colors above the spectrum, with an SSIM value closer to 1.0, as shown on the right-hand side in the following figure. As the images become less similar, the heatmap reflects more colors down the spectrum, with a value of less than 1.0. More than a hundred pairs of images across an entire set of user sessions are obtained. The average SSIM index of all pairs of images is computed to provide the overall remote session quality for all users.

    Figure 40 SSIM as a measure of image quality

    Metric 2: Frame rate—Frame rate is a common measure of user experience that defines how smooth the experience is. It measures the rate at which frames are delivered on the screen of the endpoint device. During the workload testing, NVIDIA nVector collects data on the frames per second (FPS) sent to the display device on the end client. This data is collected from thousands of samples, and the value of the 90th percentile is taken for reporting. A larger FPS indicates a more fluid user experience.

    Metric 3: End-user latency—The end-user latency metric defines the level of response of a remote desktop or application. It measures the duration of any lag that an end-user experiences when interacting with a remote desktop or application.

    Desktop VM configurations

    The following table summarizes the compute VM configurations for the nVector workloads that we tested:

    Table 13 Desktop VM specifications
    Test case nVector workload vCPUs Memory Reserved memory vGPU profile Operating system bit level HD size Screen resolution
    Non-GPU Knowledge Worker 4 8 GB 8 GB N/A Windows 10, 64-bit 60 GB 1920 x 1080
    GPU Knowledge Worker+GPU 4 8 GB 8 GB T4-2B Windows 10, 64-bit 60 GB 1920 x 1080