Unreducible Capacity Alerting for PowerMax for Possibly Indicating a Ransomware Attack

With the release of PowerMaxOS 6079.175.0042 and Unisphere 10.0.1 (March 2023), you can now get anomaly detection for your PowerMax storage environment. This blog explores the various ways you can set up capacity anomaly detection for your PowerMax storage environments that could possibly indicate a ransomware attack occurring on your PowerMax.

Unisphere 10.0.1 provides a new performance metric called Unreducible Capacity, which reports on the unreducible capacity in the storage array. With data reduction capabilities enabled by default, this metric monitors your unreducible capacity for a system and Storage Group. To chart the unreducible capacity within a PowerMax environment:

• Log in to Unisphere 10.0.1.
• Select the PowerMax for which you would like to chart unreducible capacity (this assumes that the PowerMax is leveraging DRR capabilities).
• On the left-hand side of Unisphere, expand the Performance section and select Charts (example shown here).

• Locate the Time Frame section and select a time frame.   Please note, the ‘Unreducible Capacity’ metric is selectable with all ‘Diagnostic’ and ‘Historical’ time frames, however the metric is not selectable with the Realtime time frame.
• Under Category, Unreducible Capacity is eligible for System and Storage Group (which is under the System & Hosts category).  There are benefits to charting unreducible capacity for system and Storage Groups. The benefits are:
  1. For System – Gives customers an idea if a massive ransomware attack is occurring across the whole system.
  2. For Storage Groups – Gives customers a more surgical approach because Storage Groups are typically created on an application-by-application basis. Therefore, customers can chart the unreducible capacity per Storage Group and give an indication if a ransomware attack may be occurring on the application.
• Change Data Format from the default setting of Avg (for average) to Max.
• If charting the System, check under Instances for the PowerMax array serial number you want to chart.
• Under Metric, select All for all the available metrics.
• In the All Metrics field, enter unreducible and Unreducible Capacity appears. Select Unreducible Capacity, and click Create at the bottom of the screen.

Here is how the screen now appears:

This figure shows how customers can chart the Unreducible Capacity metric in Unisphere. Customers can also set a Performance Threshold and Alert within the Settings section of Unisphere. Doing this enables customers (after monitoring the System and Storage Group for the appropriate thresholds) to set an alert within Unisphere. Unisphere will generate an alert within Unisphere and could email and/or have SNMP send the alert to stakeholders if the thresholds set for Unreducible Capacity are exceeded. To set an alert for Unreducible Capacity:

• Log in to Unisphere.
• Navigate to and select the gear icon in the top right-hand corner of Unisphere. The gear icon is the Settings section for Unisphere for this PowerMax instance.
• On the left-hand side, navigate to the Alerts section and expand it.
• Select the Performance Thresholds and Alerts section. This opens a panel to configure performance thresholds and alerts within Unisphere.
• By default, the category System is selected. To create a threshold for Unreducible Capacity for System:
  1. Select the Create button.
  2. On the top right-hand side, select the All setting to view all the Performance Metrics available within Unisphere for the category System.
  3. The metrics are listed in alphabetical order, so scroll down to the Unreducible Capacity metric and select it.
  4. Enter a number for First Threshold and a higher number for Second Threshold.
  5. On the bottom right-hand side, click OK to create a Performance Threshold alert for unreducible capacity being measured against the PowerMax system.

Here is how the information appears with these steps completed:

You can change the Category shown in this screenshot from System to Storage Groups to set a performance threshold alert for an individual storage group. Here is a screenshot of creating a performance threshold and alert for Storage Groups (Note: I selected the radio button Specific to pick specific Storage Groups that I want to monitor and alert on).

If you have SRS enabled on your PowerMax array, you can onboard CloudIQ through Unisphere.  CloudIQ is a monitoring and reporting tool for all supported Dell assets that feed data into CloudIQ.  With PowerMaxOS 6079.175.0042, you can use CloudIQ for Capacity Anomaly Detection on a PowerMax storage array.

For more information about the technologies mentioned in this blog, see the following resources:

• For Unisphere information, review the Dell Technologies Support page
• PowerMaxOS Release Notes

Authors: Justin Bastin (Senior Principal Engineer) and Richard Pace (Principal Engineering Technologist)

Cyber security remains a priority for organizations. A cyber or ransomware attack occurs every 11 seconds¹, causing organizations to continually implement security requirements in order to safeguard mission critical and sensitive data. There is an important need not only to protect this data but have the ability to recover and restore data in the event of a ransomware attack. PowerMax SnapVX Snapshots are a powerful tool to help protect, recover, and restore in the event of a cyber-attack.

SnapVX provides space saving and efficient local replication in PowerMax arrays. SnapVX snapshots are a pointer-based structure that preserves a point-in-time view of a source volume. Snapshots provide the ability to manage consistent point-in-time copies for storage groups. Host accessible target volumes can be linked if a point-in-time snapshot needs to be accessed without affecting the point-in-time of the source.

SnapVX snapshots can be set as secure snaps. Secure snaps are snapshots that cannot be deleted, either accidentally or intentionally. They are retained in resource-limited situations in which conventional snapshots are placed in a failed state to release resources.

SnapVX snapshot users can take advantage of automated scheduling using Snapshot Policies. Snapshot Policies are customizable with rules that specify when to take snapshots, how many to take, how long to keep them, and whether they are standard or secure snaps.  

The following is an example snapshot policy dashboard:

SnapVX snapshots with Snapshot Policies allows for 1024 snapshots per source device and 65 million per PowerMax array. Users can take advantage of the frequency and large snapshot scale in policy-driven snapshots to provide enhanced data resiliency. 

Because secure snaps cannot be maliciously or accidentally deleted prior to any planned expiration date, they can be leveraged for organizations to preserve multiple point in time copies that can be recovered from, in the event of a malware or ransomware attack. Snapshot policies can be automated to take secure snaps with a high frequency and a short retention duration for fine granularity, with a lower frequency and longer retention for added security, or a mixture of both. If an attack occurs, the user can review the secure snaps to determine which point in time has the most relevant and up to date copy of data without malware impact. When the precise point in time is identified, restoring critical data can be done almost instantaneously by bringing application data back to the original state prior to any attack. 

Secure snaps also provide an additional layer of security in the case of multiple attacks and can be used for forensic work to help determine what happened during the attack and when it originally occurred. With the lower frequency and longer retention period, secure snaps can be used to validate data and data change rate to help identify any suspicious activity.

The following figure provides an example of creating secure snaps with snapshot policies:

Traditional snapshots can be set with a policy to take snapshots at a frequency and retention that works best for the organization. These snapshots can be used for daily business continuity, such as development, operations, and data analytics. They can also assist in any forensic analysis and can be compared against secure snaps to help determine what changed and when it started to change. Unlike secure snaps, traditional snapshots can be deleted or fail in array resource constraint situations. However, the data on an existing snapshot cannot be changed and could be used for additional recovery options. 

Both secure and traditional snaps are a power tool for organizations to leverage to help protect and restore data rapidly, to minimize any impact of a malware or ransomware attack. The large scalability of snapshots can be easily managed using Snapshot policies for scheduling frequency and retention time duration to fit any size organization. 

The following is an operational example of the frequency, retention, and scale out of the value of SnapVX secure snaps. The numbers are based on an average of 5000 production volumes in a PowerMax array.

• Secure snaps every 10 minutes with a 48-hour retention
  • 288 per volume point-in-time copies
  • Fine grain protection and recovery
• Secure snaps every 60 minutes with a 7-day retention
  • 168 per volume point-in-time copies
  • Extended protection and data validation

Total of 2,040,000 secure point-in-time copies

The flexible and scalable features for PowerMax SnapVX traditional and secure snapshots are powerful tools for protecting against ransomware attacks.

Resources

• Dell PowerMax: Cyber Security
• Dell PowerMax: Cyber Security for Mainframe Storage
• Dell PowerMax and VMAX All Flash: TimeFinder SnapVX Local Replication
• Dell PowerMax and VMAX All Flash: Snapshot Policies

¹ "Cybercrime To Cost The World $10.5 Trillion Annually By 2025," by Cybercrime Magazine, November 2020.

Author: Richard Pace, PowerMax Engineering Technologist

Twitter: @pace_rich

Dell Technologies takes a comprehensive approach to cyber resiliency and is committed to helping customers achieve their security objectives and requirements. Storage Engineering Technologists Richard Pace, Justin Bastin, and Derek Barboza worked together, cross platform, to deliver three independent cyber security white papers for PowerMax, Mainframe, and PowerStore:

• Dell PowerMax: Cyber Security
• Dell PowerMax: Cyber Security for Mainframe Storage
• Dell PowerStore: Cyber Security

Each paper acts as a single point where customers can gain an understanding of the respective robust features and data services available to safeguard sensitive and mission critical data in the event of a cyber crime. All three papers leverage CloudIQ and the CyberSecurity feature to provide customers insight in anomaly detection.  

The following figure shows a CloudIQ anomaly that indicates unusual behavior in a customer’s environment:

Backed by CyberSecurity in CloudIQ, we can see how quickly CloudIQ detects the issue and provides the details for manual remediation.

Dell has an ingrained culture of security. We follow a 'shift-left' approach that ensures that security is baked into every process in the development life cycle. The Dell Secure Development Lifecycle (SDL) defines security controls based on industry standards that Dell product teams adopt while developing new features and functionality. Dell’s SDL defines security controls that our product teams adopt while developing new features and functionality. Our SDL includes both analysis activities and prescriptive proactive controls around key risk areas.   

Dell strives to help our customers minimize risk associated with security vulnerabilities in our products. Our goal is to provide customers with timely information, guidance, and mitigation options to address vulnerabilities. The Dell Product Security Incident Response Team (Dell PSIRT) is chartered and responsible for coordinating the response and disclosure for all product vulnerabilities that are reported to Dell. Dell employs a rigorous process to continually evaluate and improve our vulnerability response practices, and regularly benchmarks these against the rest of the industry. 

Resources

• Dell PowerMax: Cyber Security 
• Dell PowerMax: Cyber Security for Mainframe Storage
• Dell PowerStore: Cyber Security

Authors: Richard Pace, Justin F. Bastin