
PowerFlex Native Asynchronous Replication RPO with Oracle
Tue, 18 Aug 2020 17:07:05 -0000
|Read Time: 0 minutes
PowerFlex software-defined storage platform provides a reliable, high-performance foundation for mission-critical applications like Oracle databases. In many of these deployments, replication and disaster recovery have become a common practice for protecting critical data and ensuring application uptime. In this blog, I will be discussing strategies for replicating mission-critical Oracle databases using Dell EMC PowerFlex software-defined storage.
The Role of Replication and Disaster Recovery in Enterprise Applications
Customers require Disaster Recovery and Replication capabilities to meet mission-critical business requirements where SLAs require the highest uptime. Customers also want the ability to quickly recover from physical or logical disasters to ensure business continuity in the event of disaster and be able to bring up the applications in minimal time without impact to data. Replication means that the same data is available at multiple locations. For Oracle database environments, it is important to have local and remote replicas of application data which are suitable for testing, development, reporting, and disaster recovery and many other operations. Replication improves the performance and protects the availability of Oracle database application because the data exists in another location. Advantages of having multiple copies of data being present across geographies is that, critical business applications will continue to function if the local Oracle database server experiences a failure.
Replication enables customers in various scenarios such as:
- Disaster Recovery for applications ensuring business continuity
- Distributing with one type of use case such as analytics
- Offloading for mission-critical workloads such as BI, Analytics, Data Warehousing, ERP, MRP, and so on
- Data Migration
- Disaster Recovery testing
PowerFlex Software-Defined Storage – Flexibility Unleashed
PowerFlex is a software-defined storage platform designed to significantly reduce operational and infrastructure complexity, empowering organizations to move faster by delivering flexibility, elasticity, and simplicity with predictable performance and resiliency at scale. The PowerFlex family provides a foundation that combines compute as well as high performance storage resources in a managed unified fabric.
PowerFlex is designed to provide extreme performance and massive scalability up to 1000s of nodes. It can be deployed as a disaggregated storage / compute (two-layer), HCI (single-layer), or a mixed architecture. PowerFlex inclusively supports applications ranging from bare-metal workloads and virtualized machines to cloud-native containerized apps. It is widely used for large-scale mission-critical applications like Oracle database. For information about best practices for deploying Oracle RAC on PowerFlex, see Oracle RAC on PowerFlex rack.
PowerFlex also offers several enterprise-class native capabilities to protect critical data at various levels:
- Storage Disk layer: PowerFlex storage distributed data layout scheme is designed to maximize protection and optimize performance. A single volume is divided into chunks. These chunks will be striped on physical disks throughout the cluster, in a balanced and random manner. Each chunk has a total of two copies for redundancy.
- Fault Sets: By implementing Fault sets, we can ensure the persistent data availability at all time. PowerFlex (previously VxFlex OS) will mirror data for a Fault Set on SDSs that are outside the Fault Set. Thus, availability is assured even if all the servers within one Fault Set fail simultaneously. Fault Sets are subgroup of SDSs installed on host servers within a Protection Domain.
PowerFlex replication overview
PowerFlex software consists of a few important components - Meta Data Manager (MDM), Storage Data Server (SDS), Storage Data Client (SDC) and Storage Data Replicator (SDR). MDM manages the PowerFlex system as a whole, which includes metadata, devices mapping, volumes, snapshots, system capacity, errors and failures, system rebuild and rebalance tasks. SDS is the software component that enables a node to contribute its local storage to the aggregated PowerFlex pool. SDC is a lightweight device driver that exposes PowerFlex volumes as block devices to the applications and hosts. SDR handles the replication activities. PowerFlex has a unique feature called Protection Domain. A Protection Domain is a logical entity that contains a group of SDSs. Each SDS belongs to only one Protection Domain.
Figure 1. PowerFlex asynchronous replication between two systems
Replication occurs between two PowerFlex systems designated as peer systems. These peer systems are connected using LAN or WAN and are physically separated for protection purposes. Replication is defined in scope of a protection domain. All objects which participate in replication are contained in the protection domain, including volumes in Replication Consistency Group (RCG). Journal capacity from storage pools in the protection domain is shared among RCGs in the protection domain.
The SDR handles replication activities and manages I/O of replicated logical volumes. The SDR is deployed on the same server as SDS. Only I/Os from replicated volumes flows through SDR.
Replication Data Flow
Figure 2. PowerFlex replication I/O flow between two systems
- At the source, application I/O are passed from SDS to SDR.
- Application I/O are stored in the source journal space before it is sent to target. SDR packages I/O in bundles and sends them to the target journal space.
- Once the I/O are sent to target journal and get placed in target journal space, they are cleared from source.
- Once I/O are applied to target volumes, they are cleared from destination journal.
- For replicated volumes, SDS communicates to other SDS via SDR. For non-replicated volumes, SDS communicates directly with other SDS.
For detailed information about Architecture Overview, see Dell EMC PowerFlex: Introduction to Replication White Paper.
It is important to note that this approach to replication allows PowerFlex to support replication at extreme scales. As the number of nodes contributing storage are scaled, so are the SDR instances. As a result, this replication mechanism can scale effortlessly from 4 to 1000s of nodes while delivering RPOs as low as 30 seconds and meeting IO and throughput requirements.
Oracle Databases on PowerFlex
The following illustration demonstrates that the volumes participating in replication are grouped to form the Replication Consistency Group (RCG). RCG acts as the logical container for the volumes.
Figure 3. PowerFlex replication with Oracle database
Depending on the scenario, we can create multiple RCGs for each volume pair or combine multiple volume pairs in a single RCG.
In the above Oracle setup, PowerFlex System-1 is the source and PowerFlex System-2 is the destination. For replication to occur between the source and target, the following criteria must be met:
- A volume pair must be created in both source and target.
- Size of volumes in both source and target should be same. However, the volumes can be in different storage pools.
- Volumes are in read-write access mode on the source and read-only access mode in secondary. This is done to maintain data integrity and consistency between two peer systems.
The PowerFlex replication is designed to recover from as low as a 30 seconds RPOs minimizing the data-loss if there is a disaster recovery. During creation of RCG, users can specify RPO starting from 30 seconds to maximum of 60 minutes.
All the operations performed on source will be replicated to destination within the RPO. To ensure RPO compliance, PowerFlex replicates at least twice for every RPO period. For example, setting RPO to 30 seconds means that PowerFlex can immediately return to operation at the target system with only 30 seconds of potential data loss.
The following figures depicts the replication scenario under steady state of workload:
Figure 4. 100% RPO compliance for RPO of 30s for an Oracle database during a steady application workload
Figure 5. Replication dashboard view of PowerFlex
Disaster Recovery
In the case of disaster recovery, the entire application can be up and running by failover to secondary, with less than 30 seconds of data loss.
When we do a planned switchover or failover, the volumes on secondary system are automatically changed to read-write access mode and the volumes on source will be changed to read-only. Consequently, we can bring up Oracle database on secondary by setting up the Oracle environment variables and starting the database.
Once we have RCG in the failover or switchover mode, user can decide how to continue with replication:
- Restore replication: Maintains the replication direction from original source to destination.
- Reverse replication: Changes the direction so that original destination becomes the source and replication will begin from original destination to original source.
PowerFlex also provides various other options:
- Pause and Resume RCG: If there are network issues or user need to perform maintenance of any of the hardware. While paused, any application I/O will be stored at source journal and is replicated to the destination only after the replication is resumed.
- Freeze and Unfreeze RCG: If the user requires consistent snapshot of the source or target volumes. While frozen, replication will still occur between source journal and destination journal, nonetheless the target journal holds on to the data and do not apply them to the target volumes.
PowerFlex native volume replication is a unique solution and provides customers with easy to configure and setup without worrying about disaster.
Irrespective of workload and application, it is designed to support massive scale while providing RPOs as low as 30 seconds.
For more information, please visit: DellTechnologies.com/PowerFlex.
Related Blog Posts

Flexible Windows Bare-Metal Recovery with PowerProtect Data Manager
Tue, 24 Jan 2023 11:00:27 -0000
|Read Time: 0 minutes
In today’s enterprise, it is certainly not a surprise to hear of a mission-critical application server experiencing downtime or degraded state due to disaster recovery situations, such as hardware failures and cyberattacks. In such situations, bare-metal recovery (BMR) is obviously one of the best disaster recovery solutions. Most users rely on the BMR procedure to restore mission-critical applications, operating environments, and data.
With Dell PowerProtect Data Manager, BMR for a Windows server can be performed efficiently with just a few clicks. Before we explore more about Windows server BMR with PowerProtect Data Manager, let us briefly take a look at BMR.
What is BMR?
BMR, also known as offline recovery, is used as part of a disaster recovery plan that provides protection when a server or a computer will not start after a catastrophic failure. The term bare metal is in reference to a computer without a base operating system or applications. The goal of BMR is to bring a server or computer to the state it was in before the failure.
When is BMR required?
BMR can be used to recover from the following situations:
- To recover a server or a computer entirely after a hardware failure that has been repaired.
- To recover data to a new server or a computer after a hardware failure that cannot be repaired. The new computer does not have an operating system, and the operating system files must also be recovered from the old computer.
BMR of Windows host with PowerProtect Data Manager
Starting with version 19.10, PowerProtect Data Manager supports the file system agent to back up the disaster recovery asset and perform a BMR of a Windows host.
You can use BMR for the following operations:
- Physical machine to physical machine (P2P)
- Physical machine to virtual machine (P2V)
- Virtual machine to virtual machine (V2V)
With PowerProtect Data Manager, you can perform BMR by backing up the disaster recovery asset. When a file system agent backup is performed, there is an extra asset—”disaster recovery”—that is backed up. This asset includes the information required to rebuild the Windows system back to its state at the time of the backup. The data in the disaster recovery asset, plus volume information for those file systems that contain operating system data (critical volumes), are also backed up.
After the disaster recovery asset backup is successful, you can perform the BMR using the customized PowerProtect Data Manager WinPE ISO image. By default, each BMR backup is system state enabled.
Backing up Windows disaster recovery assets
After you install the file system agent on the Windows file system host and it is approved in the PowerProtect Data Manager UI, the disaster recovery asset is discovered along with the other file system assets.
After the disaster recovery asset is discovered in PowerProtect Data Manager UI, you can create a file system protection policy and configure it to back up the disaster recovery asset. A disaster recovery protection policy should contain objects to be backed up, which include critical volumes and system state recovery files.
After the disaster recovery asset backup is successful, you can perform BMR using the customized PowerProtect Data Manager WinPE ISO image.
BMR data consists of the following:
- The operating system files and all data except user data on critical volumes
Note: Critical volumes include the boot volume, the system volume, and the volume that hosts system state data, such as Active Directory and application services.
- All system state information
By default, each BMR backup is system state enabled.
To protect a Windows host entirely, we recommend that you back up BMR data for critical volumes and separately back up regular assets that contain user data.
Performing Windows BMR
PowerProtect Data Manager provides a custom WinPE image that allows you to recover a source host to a target host without installing an operating system. Because local disks are not in use by the operating system, the recovery process can replace files without conflict. The custom PowerProtect Data Manager WinPE image is based on Windows PE 10.0 and contains the NIC and disk drivers for the Windows versions that the WinPE image supports.
Before you perform a BMR, verify that the environment meets the requirements and that you have the necessary information.
Note: For BMR requirements, see the the PowerProtect Data Manager File System User Guide.
When a recovery of a system is required, you can download the Windows BMR ISO image from the PowerProtect Data Manager UI. The image contains the necessary files to boot and create a WinPE system. The image includes a PowerProtect Data Manager Bare Metal Recovery Wizard that is launched and used as part of the restore.
Note: Ensure that the hardware on the target computer is operational and that the target computer is similar in make, model, and hardware configuration to the source computer to be recovered. For more details about the BMR requirements, see the PowerProtect Data Manager File System User Guide.
The target host boots with the custom WinPE image, either locally or over the network. The Welcome page of the PowerProtect Data Manager Bare Metal Recovery Wizard is displayed.
On the Select NIC page, you can select the network interface for communication with Data Manager during the BMR. If the required NIC driver is not in the list, click Load Driver to browse to it.
Note: The driver must not require a restart. The WinPE environment loads only in memory, and changes are not persistent across a restart. If a restart prompt appears, you might be able to ignore the prompt. Most NIC drivers are plug-and-play.
On the Host and Network Configuration page, enter the hostname of the target host and the domain name for the host.
On the Available Disks page, verify the disk configuration. The size and number of hard disks that are added to the target machine should be either equal to or greater than the size and number of disks on the source machine.
On the Select Server page, enter the PowerProtect Data Manager server and source hostname details. In the Server Name or IP field, add the IP of the server or FQDN only.
On the Select Backup page, select the respective host from the Source Host Name list and BMR data to restore to the destination host. Backups appear in the list in descending order from the most to least recent.
The Critical Volumes page displays the volumes that will be restored and the option to enable a quick disk format.
The PowerProtect Data Manager BMR wizard fetches information to perform a BMR, and the Summary page is displayed. To add custom BMR options, next to Custom restore options, click Options.
Confirm the quick format of disks and restore the backup.
The Status page shows the restore progress.
You can also monitor the Bare Metal Recovery job status in the PowerProtect Data Manager UI at Jobs > Protection Jobs.
The BMR wizard displays the results. After the recovery is successful, you can reboot the system and restore the application data as required.
With this easy BMR solution with PowerProtect Data Manager, Dell Technologies empowers Windows administrators to recover their business-critical Windows servers quickly and resume their operations.
For more details about disaster recovery for Windows with PowerProtect Data Manager, see the technical white paper and PowerProtect Data Manager: File System User Guide.
For more details about data protection with PowerProtect Data Manager, see the PowerProtect Data Manager website.
Author: Vinod Kumar Kumaresan, Principal Engineering Technologist, Data Protection Division

Driving Innovation with the Dell Validated Platform for Red Hat OpenShift and IBM Instana
Wed, 14 Dec 2022 21:20:39 -0000
|Read Time: 0 minutes
“There is no innovation and creativity without failure. Period.” – Brené Brown
In the Information Technology field today, it seems like it’s impossible to go five minutes without someone using some variation of the word innovate. We are constantly told we need to innovate to stay competitive and remain relevant. I don’t want to spend time arguing the importance of innovation, because if you’re reading this then you probably already understand its importance.
What I do want to focus on is the role that failure plays in innovation. One of the biggest barriers to innovation is the fear of failure. We have all experienced some level of failure in our lives, and the costly mistakes can be particularly memorable. To create a culture that fosters innovation, we need to create an environment that reduces the costs associated with failure – these can be financial costs, time costs, or reputation costs. This is why one of the core tenets of modern application architecture is “fail fast”. Put simply, it means to identify mistakes quickly and adjust. The idea is that a flawed process or assumption will cost more to fix the longer it is present in the system. With traditional waterfall processes, that flaw could be present and undetected for months during the development process, and in some cases, even make it through to production.
While the benefits of fail fast can be easy to see, implementing it can be a bit harder. It involves streamlining not just the development process, but also the build process, the release process, and having proper instrumentation all the way through from dev to production. This last part, instrumentation, is the focus of this article. Instrumentation means monitoring a system to allow the operators to:
- See current state
- Identify application performance
- Detect when something is not operating as expected
While the need for instrumentation has always been present, developers are often faced with difficult timelines and the first feature areas that tend to be cut are testing and instrumentation. This can help in the short term, but it often ends up costing more down the road, both financially and in the end-user experience.
IBM Instana is a tool that provides observability of complete systems, with support for over 250 different technologies. This means that you can deploy Instana into the environment and start seeing valuable information without requiring any code changes. If you are supporting web-based applications, you can also take things further by including basic script references in the code to gain insights from client statistics as well.
Announcing Support for Instana on the Dell Validated Platform for Red Hat OpenShift
Installing IBM Instana into the Dell Validated Platform for Red Hat OpenShift can be done by Operator, Helm Chart, or YAML File.
The simplest way is to use the Operator. This consists of the following steps:
- Create the instana-agent project
- Set the policy permissions for the instana-agent service account
- Install the Operator
- Apply the Operator Configuration using a custom resource YAML file
You can configure IBM Instana to point to IBM’s cloud endpoint. Or for high security environments, you can choose to connect to a private IBM Instana endpoint hosted internally.
Figure 1. Infrastructure view of the OpenShift Cluster
Once configured, the IBM Instana agent starts sending data to the endpoint for analysis. The graphical view in Figure 1 shows the overall health of the Kubernetes cluster, and the node on which each resource is located. The resources in a normal state are gray: any resource requiring attention would appear in a different color.
Figure 2: Cluster View
We can also see the metrics across the cluster, including CPU and Memory statistics. The charts are kept in time sync, so if you highlight a given area or narrow the time period, all of the charts remain in the same context. This makes it easy to identify correlations between different metrics and events.
Figure 3: Application Calls View
Looking at the application calls allows you to see how a given application is performing over time. Being able to narrow down to a one second granularity means that you can actually follow individual calls through the system and see things like the parameters passed in the call. This can be incredibly helpful for troubleshooting intermittent application issues.
Figure 4: Application Dependencies View
The dependencies view gives you a graphical representation of all the components within a system and how they relate to each other, in a dependency diagram. This is critically important in modern application design because as you implement a larger number of more focused services, often created by different DevOps teams, it can be difficult to keep track of what services are being composed together.
Figure 5: Application Stack Traces
The application stack trace allows you to walk the stack of an application to see what calls were made, and how much time each call took to complete. Knowing that a page load took five seconds can help indicate a problem, but being able to walk the stack and identify that 4.8 seconds was spent running a database query (and exactly what query that was) means that you can spend less time troubleshooting, because you already know exactly what needs to be fixed.
For more information about the Dell Validated Platform for Red Hat OpenShift, see our launch announcement: Accelerate DevOps and Cloud Native Apps with the Dell Validated Platform for Red Hat OpenShift | Dell Technologies Info Hub.
Author: Michael Wells, PowerFlex Engineering Technologist
Twitter: @SqlTechMike
LinkedIn