This section describes the performance validation of both Secureworks VDR and XDR. We describe the details of our approach and the corresponding results. Our objective was to estimate the magnitude of the additional load created on our VxRail system under test (SUT) when we added vulnerability and threat detection scanning concurrently with a normal VMS plus CV workload. To be able to recommend this solution architecture to our customers and partners, we needed to ensure that none of the critical VMS and CV workloads would be significantly impacted by adding the additional Secureworks services on the common platform.

We first configured all the applications on a single VxRail cluster including the previously validated Smart Transportation applications plus the Secureworks VDR virtual appliance and XDR Red Cloak agents installed on 10 systems. All simulated camera streaming applications were hosted outside of the VxRail SUT. We initiated a comprehensive simulation of both VMS recording and computer vision analysis workloads while capturing our baseline metrics for comparison with the remaining scenarios. The Secureworks XDR and VDR scans were added independently (VDR only and XDR only) and simultaneously (both VDR and XDR) for our impact analysis. We then assessed the additional resource consumption impacts of Secureworks' vulnerability and threat monitoring scanning.

The VxRail cluster used for our earlier CV platform validation consists of 5 nodes running dozens of VMs running application roles and infrastructure management. Our Secureworks testing was completed on a total of 10 systems that comprise a representative sample of the various roles used in the system. For example, we only choose 1 of the 4 available alert processing servers from the two CV application vendors included in the original platform designs.

The complete list of servers included in the validation testing is as follows:

The VMS and CV systems hosted on the VxRail were processing a comprehensive simulated workload load before initiating any Secureworks scanning. The full details of the simulated workload are described in previously published Computer Vision for Transportation Design Guides referenced several times earlier in the document. To give an indicative example of the scale of the baseline workloads, we had 12 Milestone XProtect recorders processing 70 camera streams each. The two CV application servers we included in our sample were processing real-time alerting on between 30 and 40 of the camera streams being recorded by Milestone XProtect.

The following steps were performed for metrics collection on each of the 10 servers in the sample identified in the table above.

This resulted in over 40 hours of performance data recorded at 10-second intervals capturing dozens of metric attributes per server.

The charts below show resource utilization metrics collected from the Windows Performance Monitor data collector. We use a PowerShell script and configuration file to control the list of counters collected and the duration of the data collection window. This eliminates the resource overhead of connecting with Remote Desktop Protocol tools to run data collection interactively. The charts included below were developed using Python data analysis tools including Pandas and seaborn charting.

The four color line charts below have the x-axis labeled Time Interval. The Time Interval is the number of 10-second intervals that have passed since the beginning of the test. A Time Interval of 360 is equivalent to a total collection duration of 60 minutes (6 ten-second intervals per minute times 60 minutes).

The four colored lines (blue, gold, green, and red) each represent one of the four 1-hour testing scenarios listed above.

The chart below is from one of the 10 servers tested that represent the general shape of the data we collected. All 10 of the servers showed similar patterns.

Findings

• VDR scanning (blue line compared to green line) shows no discernible impact on CPU utilization.
• XDR scanning with the Red Cloak Agent (red line compared to blue line) shows a sustained increase in CPU utilization.
• BOTH VDR and XDR scanning (gold line compared to red line) are consistent with the individual results that XDR accounts for most, if not all the increase.
• This behavior is consistent and representative of the other nine servers tested.

Overall

• Due to the increased XDR CPU consumption, further analysis is needed to understand the impact.

This analysis provides more insight into the CPU Utilization metrics for all 10 servers we tested comparing the XDR ONLY scenario to the BASELINE. The goal is to create a distribution of increased CPU utilization to show the relative percentage of total test time associated with higher levels of CPU utilization increase attributed to XDR activity.

We collected two hours of testing data for this analysis, one hour of BASELINE workload and one hour of BASELINE workload plus XDR scanning. The data was collected every 10 seconds. We then subtracted the BASELINE time series from the BASELINE PLUS XDR time series at each 10-second interval. We grouped the data by counting the measured differences between 0 and 5 percent to get the total number of 10-second intervals where the CPU increase was in the 0-5% range. That data is presented in the center column of the table below. The right column converts the minutes to % of the time using 60 minutes for the total test.

The first row shows that the CPU % utilization increased between 0 and 5% for 21 minutes of the 60-minute test or 35% of the test time. The bottom row of data shows that an increase in CPU utilization of 30-35% was only observed for 2 minutes or 4% of the hour.

Findings

• 55% of the time the CPU increase impact is between 0-10% when running XDR.
• 17% of the time the CPU increase impact is over 20% when running XDR.

Overall

• The overhead of running XDR is generally less than 10%.
• A new Taegis XDR agent is being released by Secureworks that will replace the "Red Cloak" agent and has a lower CPU overhead.

The chart below is from one of the 10 servers tested that represent the general shape of the data we collected. All 10 of the servers showed similar patterns.

Findings

• The y-axis scale covers only a 4% difference between tic marks, so the increase in memory utilization (<200 MB) was negligible when enabling XDR
• VDR scanning had no noticeable impact on the percentage of memory consumed.

Overall

• These memory consumption results were consistent for all the servers tested.
• Memory consumption by VDR and XDR should have no impact on system sizing guidance and is consistent with the research expectations prior to testing.

The chart below is from one of the 10 servers tested that represent the general shape of the data we collected. All 10 of the servers showed similar patterns.

Findings

• The impact on network consumption from running both XDR and VDR scanning simultaneously shows the least change of any metric comparison in this testing.
• These network consumption results were consistent for all the servers tested.

Overall

• Network consumption by VDR and XDR should have no impact on system sizing guidance.

The charts below are from one of the 10 servers tested that represent the general shape of the data we collected. All 10 of the servers showed similar patterns.

Findings

• A minor increase in Reads/sec and Writes/sec was observed when XDR was enabled.
• Disk operations spike periodically but only for very short durations. This is expected behavior as the operating system flushes caches to disk.
• No additional disk operations were observed when VDR scanning was active.

Overall

• A very minor increase in Reads or Writes with XDR scanning.
• For systems with VSAN configured with SSD disks, there is no impact on sizing recommendations.

• Our testing of VDR scanning shows no impact on the CPU utilization and therefore could be enabled for all servers without impacting sizing guidance.
• Our testing of the XDR Red Cloak agents shows a noticeable impact on CPU utilization but was less than 10% for approximately 50% of the total testing duration. Secureworks is soon to release a new agent through online distribution which is expected to reduce this overhead significantly.
• There is no significant impact on Memory utilization from either VDR or XDR that will impact sizing guidance.
• There is no significant Network impact of running VDR or XDR that will impact sizing guidance.
• There is no significant impact on Disk IO when running VDR or XDR that will impact sizing guidance.

Overall, the benefits of protection provided by VDR and XDR are significant and should be enabled on your Computer Vision for Smart Transportation environments to provide robust cybersecurity protection.