Conquer Your Challenges with Dell PowerProtect Data Manager Dynamic NAS Protection
Wed, 16 Feb 2022 21:53:54 -0000
|Read Time: 0 minutes
Challenges when protecting NAS
For years, NAS and backup vendors have used Network Data Management Protocol (NDMP) to protect NAS data. But NDMP has its own limitations, such as manual slicing of a NAS share to achieve multi-stream backup, limited parallel streams, and requires periodic full backups. Customers also face challenges to protect their growing amounts of data and to back up that data within their specified backup windows. With NDMP, full image restores are required for a file-level recovery and restore to any NAS device, such as NFS/CIFS, are not supported. These challenges lead to missed data protection and Service Level Agreements (SLAs).
PowerProtect Data Manager Dynamic NAS protection
Dell PowerProtect Data Manager for NAS protection addresses today’s customer challenges of protecting evolving NAS environments. In the PowerProtect Data Manager 19.9 software release, we introduced a new NAS data protection solution called Dynamic NAS protection. Unlike NDMP-based solutions, Dynamic NAS protection is a NAS-vendor-agnostic solution. With Dynamic NAS protection, customers can overcome some of the challenges they faced with NDMP.
Dynamic NAS protection addresses the challenges with the following capabilities:
- Vendor-agnostic solution for NAS protection
- Forever incremental backup
- High number of parallel streams and multiple virtual containers to address scale and performance
- Index, search, and restore
- Restore to any NAS device, such as NFS/CIFS
Dynamic NAS protection provides a non-NDMP, crawl- and backup-based solution by leveraging the NAS Protection Engine internally using Filesystem Agents (FSA) file based-backup (FBB) technology.
Dynamic NAS protection uses the NAS Protection Engine for backup and recovery orchestration. This solution is easy to use, providing automatic discovery, orchestration, and management through the Data Manager UI.
Data Manager for NAS protection solution supports all the Data Manager objectives such as DD Replication, Cloud Tier, progress monitoring, and SLA compliance.
Dynamic NAS protection - Intelligent auto slicer
The NAS file share auto slicer is a new library that is embedded in the Data Manager NAS agent. The slicer splits NAS assets (NAS share, a file system) into multiple sub-assets in preparation for multi-stream data movement to a Dell PowerProtect DD series appliance. Slices are created using parallel threads, and each slice is backed up concurrently using available NAS Protection Engine containers and moved to a PowerProtect DD series appliance.
The slicer partitions NAS assets dynamically before each backup. Based on backup history and changes in the content of the NAS asset being sliced, relevant slices are added, removed, or rebalanced. Periodically, unbalanced trees are automatically managed as content changes over time. No manual reconfiguration is required. The default slice size is 200 GB or 1 million files.
For a full backup, a complete share is traversed in parallel to create slices. For an Incremental backup, only modified slices are traversed based on backup history.
Auto distribution of backup streams
Dynamic NAS protection enables automated load balancing of protection engine hosts, and automatic scaling for containers to achieve maximum backup streams and reduce manual management overhead. The NAS protection containers spin up and tear down, depending upon the workload. Each NAS Protection Engine can run multiple containers. Each container is pre-installed with a NAS agent and an FSA agent.
Protecting NAS assets with Data Manager
With the Data Manager 19.9 software release, the Dynamic NAS solution supports protection for Dell PowerStore, Dell Unity, and Dell PowerScale (Isilon) NAS products, and any NFS or CIFS share using generic NAS for vendors such as NetApp, Windows, and Linux file servers.
Data Manager can protect NAS assets in two ways:
- Appliances: Automatic discovery of shares on supported Dell PowerStore, Dell Unity, and Dell PowerScale (Isilon) products.
- Shares: Network File System (NFS) and Common Internet File System (CIFS) shares from other NAS platforms.
Restoring the NAS assets with Data Manager
Data Manager provides support to restore a NAS asset to the original location or to an alternate location. Data Manager also supports FLR using the search engine to restore individual files and folders from NAS backups. Once the Search Engine is deployed and NAS protection policy is enabled with indexing, individual files and folders can be restored from one or more NAS backups by using the File Search option.
Data Manager for NAS protection supports the following restore use cases:
- Share-level restore
- Restore to any device, NFS or CIFS
- Restore to original and alternate NAS shares
- File-Level Recovery (FLR): NAS backups are indexed on the Search Engine for search and restore operations.
With many enhancements across our Dell data protection software offerings, Dell Technologies continues to drive innovation without compromise. We stop at nothing to give you technology innovations that modernize the protection of your NAS infrastructure.
Easily automate and optimize with Dynamic NAS protection available with Data Manager. With its snapshot technology and intelligent slicing, Data Manager protects NAS data efficiently within the required backup window. Dynamic NAS protection offers up to 3x faster backups[1] and up to 2x faster restores[2].
For more details on Data Manager Dynamic NAS protection, see the white paper Dell PowerProtect Data Manager: Dynamic NAS Protection and visit the Dell PowerProtect Data Manager web site.
Author: Vinod Kumar Kumaresan, Senior Engineering Technologist, Data Protection Division
[1] When comparing PowerProtect Data Manager 19.9 with Dynamic NAS protection backup performance to NDMP backup performance with Avamar. Based on Dell internal testing. August 2021.
[2] When comparing PowerProtect Data Manager 19.9 with Dynamic NAS protection restore performance to NDMP restore performance with Avamar. Based on Dell internal testing. August 2021.
Related Blog Posts
Using Dell PowerFlex and Google Distributed Cloud Virtual for Postgres Databases and How to Protect Them
Fri, 03 Nov 2023 23:27:04 -0000
|Read Time: 0 minutes
Did you know you can get the Google Cloud experience in your data center? Well now, you can! Using Google Distributed Cloud (GDC) Virtual and Dell PowerFlex enables the use of cloud and container workloads – such as Postgres databases – in your data center.
Looking beyond day one operations, the whole lifecycle must be considered, which includes assessing how to protect these cloud native workloads. That’s where Dell PowerProtect Data Manager comes in, allowing you to protect your workloads both in the data center and the cloud. PowerProtect Data Manager enhances data protection by discovering, managing, and sending data directly to the Dell PowerProtect DD series virtual appliance, resulting in unmatched efficiency, deduplication, performance, and scalability. Together with PowerProtect Data Manager, the PowerProtect DD is the ultimate cyber resilient data protection appliance.
In the following blog, we will unpack all this and more, giving you the opportunity to see how Dell PowerFlex and GDC Virtual can transform how you cloud.
What is Google Distributed Cloud Virtual?
We will start by looking at GDC Virtual and how it allows you to consume the cloud on your terms.
GDC Virtual provides you with a consistent platform for building and managing containerized applications across hybrid infrastructures and helps your developers become more productive across all environments. GDC Virtual provides all the mechanisms required to bring your code into production reliably, securely, and consistently while minimizing risk. GDC Virtual is built on open-source technologies pioneered by Google Cloud including Kubernetes and Istio, enabling consistency between cloud and on premises environments like PowerFlex. Anthos GKE (on GCP and on-prem), Anthos Service Mesh, and Anthos Config Management are the core building blocks of Anthos, which has integrations with platform-level services such as Stackdriver, Cloud Build, and Binary Authorization. GDC Virtual users purchase services and resources from the GCP Marketplace.
Figure 1. GDC Virtual components.
GDC Virtual puts all your IT resources into a consistent development, management, and control framework, automating low-value tasks across your PowerFlex and GCP infrastructure.
Within the context of GCP, the term ‘hybrid cloud’ describes a setup in which common or interconnected services are deployed across multiple computing environments, which include public cloud and on-premises. A hybrid cloud strategy allows you to extend the capacity and capabilities of your IT without the upfront capital expense investments of the public cloud while preserving your existing investments by adding one or more cloud deployments to your existing infrastructure. For more information, see Hybrid and Multi-Cloud Architecture Patterns.
PowerFlex delivers software defined storage to both virtual environments and bare metal hosts providing flexible consumption or resources. This enables both two-tier and three-tier architectures to match the needs of most any environment.
PowerFlex container storage
From the PowerFlex UI – shown in the following figure – you can easily monitor the performance and usage of your PowerFlex environment. Additionally, PowerFlex offers a container storage interface (CSI) and container storage modules (CSM) for integration with your container environment. The CSI/CSM allows containers to have persistent storage, which is important when working with workloads like databases that require it.
Figure 2. PowerFlex dashboard provides easy access to information.
To gain a deeper understanding of implementing GDC Virtual on Dell Powerflex, we invite you to explore our recently published reference architecture.
Dell engineers have recently prepared a PostgreSQL container environment deployed from the Google Cloud to a PowerFlex environment with GDC Virtual in anticipation of Kubecon. For those who have deployed Postgres from Google Cloud, you know it doesn’t take long to deploy. It took our team maybe 10 minutes, which makes it effortless to consume and integrate into workloads.
Once we had Postgres deployed, we proceeded to put it under load as we added records to it. To do this, we used pgbench, which is a built-in benchmarking tool in Postgres. This made it easy to fill a database with 10 million entries. We then used pgbench to simulate the load of 40 clients running 40 threads against the freshly loaded database.
Our goal wasn’t to capture performance numbers though. We just wanted to get a “warm” database created for some data protection work. That being said, what we saw on our modest cluster was impressive, with sub-millisecond latency and plenty of IO.
Data protection
With our containerized database warmed up, it was time to protect it. As you probably know, there are many ways to do this, some better than others. We’ll spend just a moment talking about two functional methods of data protection – crash consistent and application consistent backups. PowerProtect Data Manager supports both crash-consistent and application consistent database backups.
A “crash consistent” backup is exactly as the name implies. The backup application captures the volume in its running state and copies out the data regardless of what’s currently happening. It’s as if someone had just pulled the power cord on the workload. Needless to say, that’s not the most desirable backup state, but it’s still better than no backup at all.
That’s where an “application consistent” backup can be more desirable. An application consistent backup talks with the application and makes sure the data is all “flushed” and in a “clean” state prior to it being backed up. At least, that’s the simple version.
The longer version is that the backup application talks to the OS and application, asks them to flush their buffers – known as quiescing – and then triggers a snapshot of the volumes to be backed up. Once complete, the system then initiates a snapshot on the underlying storage – in this case PowerFlex – of the volumes used. Once the snapshots are completed, the application-level snapshots are released, the applications begin writing normally to it again, and the backup application begins to copy the storage snapshot to the protected location. All of this happens in a matter of seconds, many times even faster.
This is why application consistent backups are preferred. The backup can take about the same amount of time to run, but the data is in a known good state, which makes the chances of recovery much greater than crash consistent backups.
In our lab environment, we did this with PowerProtect Data Manager and PowerProtect DD Virtual Edition (DDVE). PowerProtect Data Manager provides a standardized way to quiesce a supported database, backup the data from that database, and then return the database to operation. This works great for protecting Kubernetes workloads running on PowerFlex. It’s able to create application consistent backups of the Postgres containers quickly and efficiently. This also works in concert with GDC Virtual, allowing for the containers to be registered and restored into the cloud environment.
Figure 3. An application consistent backup and its timing in the PowerProtect Data Manager UI
It’s great having application consistent backups of your cloud workloads, “checking” many of those boxes that people require from their backup environments. That said, just as important and not to be forgotten is the recovery of the backups.
Data recovery
As has been said many times, “never trust a backup that hasn’t been tested.” It’s important to test any and all backups to make sure they can be recovered. Testing the recovery of a Postgres database running in GDC Virtual on PowerFlex is as straightforward as can be.
The high-level steps are:
- From the PowerProtect Data Manager UI, select Restore > Assets, and select the Kubernetes tab. Select the checkbox next to the protected namespace and click Restore.
- On the Select Copy page, select the copy you wish to restore from.
- On the Restore Type page, select where it should be restored to.
- Determine how the Persistent Volume Claims (PVCs) and namespace should be restored.
- When finished, test the restore.
You might have noticed in step 4, I mentioned PVCs, which are the container’s connections to the data and, as the name implies, allow that data to persist across the nodes. This is made possible by the CSI/CSM mentioned earlier. Because of the integration across the environment, restoring PVCs is a simple task.
The following shows some of the recovery options in PowerProtect Data Manager for PVCs.
Figure 4. PowerProtect Data Manager UI – Namespace restore options
The recovery, like most things in data protection, is relatively anticlimactic. Everything is functional, and queries work as expected against the Postgres database instance.
Dell and Google Cloud collaborated extensively to create solutions that leverage both PowerFlex and GDC Virtual. The power of this collaboration really shows through when recovery operations just work. That consistency and ease enables customers to take advantage of a robust environment backed by leaders in the space and helps to remove one nightmare that keeps developers and IT admins awake at night, allowing them to rest easy and be prepared to change the world.
If any of this sounds interesting to you and you’ll be at Kubecon in Chicago, Illinois on November 6-9, stop by the Google Cloud booth. We’ll be happy to show you demos of this exciting collaboration in action. Otherwise, feel free contact your Dell representative for more details.
Resources
Authors:
Authors: | Tony Foster, | Vinod Kumar Kumaresan, | Harsha Yadappanavar, |
LinkedIn: | |||
X (formerly Twitter): |
| @harshauy | |
Personal Blog: |
|
|
Smart Scale for Dell PowerProtect Appliances, Part III: MSU Migration, Replication, Recoverability, Security
Tue, 18 Jul 2023 15:44:25 -0000
|Read Time: 0 minutes
In the first blog post of this series, we covered what Smart Scale is and why we need it. In the second part of the series, we covered the architecture deep dive of the solution and how to deploy, set up, and configure workflows.
In this part, let’s discuss Smart Scale Mobile Storage Unit (MSU) migration, replication topology, recovery, and security.
Migrating Mobile Storage Units
Migrating a Mobile Storage Unit (MSU) is a new feature that helps to migrate a MSU from one DD series appliance to another. In the following figure, notice that “MSU7” has been migrated to DD series appliance 3 from DD series appliance 2 and the client continues to use the “MSU7” on DD series appliance 3 for backup and recovery operations.
When you initiate the migration, the Migration and Placement Service provides the recommended list of DD series appliances from the available list of DD series appliances in the Data Center. Based on that list, you can select the target system and initiate the migration.
You can select which Network Group to use, and can select from the following migration transfer priorities:
- Balanced Transfer – Balances the resources between the backup and the migration
- Fast Transfer – Uses more resources for migration
- Minimum system impact – Uses fewer resources for the migration
The final step of the migration provides a Review and Commit screen, which includes several migration statistics (such as logical capacity used, physical capacity used and available, and compression factor). Using these analytics, users can optimize appliance and workload lifecycles. When the migration job reaches 100%, it is ready to commit. All the operations (backup or restore) to the source MSU need to be canceled or stopped before committing the migration. There are two options available for the source MSU: you can either mark the MSU to be deleted after a successful migration or you can keep the MSU on the source system. When the commit is done, the remaining data that needs to be synced with the target DDR will be updated. The source MSU is marked as read only; the target MSU is marked as read write. The source MSU is demoted to an Mtree and target system has a new MSU with the same name. The Migration and Placement Service in PowerProtect DD Management Center updates the Namespace Redirection Service Data Manager (NRSDM) about the new location of the MSU. NRSDM then provides that information to the NRS. When a boost client now requests the physical location of the MSU to write the data, NRS provides the target DDR’s IP address. Critically, none of these operations requires reconfiguring the backup software.
Note: Smart Scale is an additional feature of DDMC designed to make administration simple, agile, and flexible. When Smart Scale services are enabled, the customer’s existing environment remains undisturbed. This means that traditional backup operations and storage units co-exist with operations that system pools and mobile storage units support.
MSU affinity group migration
Starting with DDMC version 7.12, migration of a group of MSUs in an affinity group (or a subset of the group) from one DD system to another in a system pool is supported. With MSU affinity group migration, a DDMC administrator can migrate a group of MSUs without having to manually set up multiple migration jobs. There can be up to 32 MTrees in an affinity group.
Replicating Mobile Storage Units
DD series appliances provide automated, policy based, network efficient, and encrypted replication for disaster recovery and multi-site backup and archive consolidation. Smart Scale supports Managed File Replication (MFR) on Mobile Storage Units (MSUs) with Dell PowerProtect Data Manager, Dell NetWorker, and Veritas NetBackup. Types of replications supported are:
- Storage Unit to Storage Unit
- MSUs to Storage Unit, and Storage Unit to MSUs
- MSUs to MSUs
Replication is supported between storage units or MSUs that are within the same data center or not, within the same system pool or not, within the same appliance or not, and within the management domain of a single DDMC or across DDMC deployments.
Recoverbility of Smart Scale services
In the era of performance on appliances, what organizations often ignore is the recoverability of the appliance or service when failures occur. Dell recognizes the importance of your SLA and RPO, keeping that in mind we have built Smart Scale architecture so that it provides an automated response to any disaster, by providing options for quick recovery to ensure minimum downtime.
Let’s divide the Smart Scale architecture into two parts: the Management plane which has all the Smart Scale management services, and the Data plane which stores the actual data (that is, the DD series appliance). Now let’s examine the various recovery abilities available in both planes.
In the management plane
- Smart Scale has intelligent Analytics Services that provide capacity predictions, initial placement, and rebalancing recommendations.
- Micro services in PowerProtect DD Management Center (DDMC) and DD Namespace VM (DDNVM) automatically restart and recover as needed.
- Automated alert notifications are triggered by periodic monitoring of services in DDMC and Namespace VM. In the worst case, DDMC administrators may need to redeploy DDNVM using DDMC (which is quick, by just re-entering the VM credentials).
- The reconciliation service automatically resynchronizes and fixes discrepancies between DDMC and DDNVM.
- DDMC backs up the configuration regularly on a timed basis and whenever administrators make a configuration change.
- DDMC can be recovered with a clean reinstall using configuration backup. If there are no issues in the DDNVM, DDMC can then resynchronize with the running DDNVM, with no need to redeploy DDNVM.
In the data plane
- Data Center services hold a copy of the overall namespace for each pool of DD series appliances, to enable the redirection of backup clients to the appropriate DD series appliance in the pool
- After it is redirected, the backup client connects to the DD series appliance and starts data transfer, then (unless it is restarted) it continues to work even if DDMC or DDNVM have any issues.
- DDMC VM will redeploy DDNVM if necessary
- DDNVM continues working even if DDMC is not available
- Reinstalling DDMC does NOT require reinstalling DDNVM
Security options with Smart Scale services
With the increased rate of cyber-attacks and ransomware threats, it is essential to have services and appliances that have adequate security restrictions for shielding your data from malicious attacks. Smart Scale enables security at every level of the solution. Let’s take a look at the security options available:
Data plane
- D@RE Encryption at rest is supported within Smart Scale. A System Pool may have a mix of encrypted and non-encrypted systems. During migration a warning is given if the selected migration is from encrypted to non-encrypted systems.
- Encryption in flight is supported both for backup/restore and replication, with the same modes of operation as regularly available for PowerProtect appliances.
- MSUs are protected by Mobile Boost Users. These users are migrated between systems automatically as required when MSUs are migrated.
Management plane
- Overall security is applied and managed from DDMC, both in the creation of System Pools and in the creation and mobility of MSUs.
- Certificates are enabled to secure communications between DDMC and DDNVM.
- Most services run as non-root user.
- DDNVM is guest OS enabled, and no customer login is allowed.
Network security
- When creating and enabling MSUs, the supported network groups are selected. Redirection from Pool Access IP to MSUs is restricted only to those network groups enabled for specific MSUs.
- VLANs are supported (but not mandated) using either trunk or access mode (that is, tagging on the node or the network switch) for both appliances and DDNVM.
Smart Scale is about simplifying capacity management across multiple DD series appliances, but it's more than just management and reporting. While it includes analytics, insights, and recommendations around capacity needs and placement, the real value is the system pooling and management it provides. Smart Scale services deliver the next generation of data protection scale, mobility, and insights for PowerProtect DD series appliances.
Thank you for taking a moment to read this series of Smart Scale blog posts. We hope they were useful and helped you to understand the Smart Scale feature comprehensively.
To catch up on the previous Smart Scale blog posts in this series, see:
- Smart Scale for Dell PowerProtect Appliances, Part I: Innovative Technology to Manage Multi-Exabyte Data
- Smart Scale for Dell PowerProtect Appliances, Part II: Architecture Deep Dive
Additional resources for Smart Scale
- Dell PowerProtect DD series appliances
- Smart Scale for PowerProtect Appliances – Technical white paper
- Dell PowerProtect DD Management Center Interactive Demo
Authors: