Home > Storage > Unity XT > Data Protection > Dell Unity: Replication Technologies > Theory of Operation
Native asynchronous replication is consisted of several physical and software components. Each of these components will be discussed in the following sections. When creating asynchronous replication between Pools within a system, only a replication session needs to be created. To fully configure asynchronous replication between systems, you must do each of following:
This section will also discuss the various block and file resources which support asynchronous replication. The replication modes and roles will also be discussed. The sections below outline the different functions, requirements, and how these components interact with each other. Configuration and management of these components will be completed using Unisphere, which will be discussed in the Unisphere Management section.
Replication interfaces are used to transport data to a destination system for remote replication sessions and must be defined for replication to work. Asynchronous replication between Pools within a system does not require replication interfaces to be configured. Asynchronous remote replication requires a replication interface be created on each SP on the system. For asynchronous replication, replication is performed over ethernet interfaces configured on the system.
Any front-end ethernet port on Dell Unity physical systems can be used for replication, such as onboard 10 GbE BaseT ports, CNA ports with ethernet personality, or ethernet based I/O modules. Supported front-end port configurations and capabilities varies by system model. Replication interfaces can also be created on link aggregated ports for high availability, increased maximum throughput, and load balancing of replication traffic across physical ports in the aggregation. It is suggested to only configure replication interfaces on ports of the same type and speed. It is also recommended to dedicate the replication interfaces strictly for replication purposes, but it is not required and hosts can also use these ports. In Dell UnityVSA systems, any I/O port can be used for asynchronous replication.
In UnityOS versions prior to the 5.1 release, in order to have a successful asynchronous replication connection, all asynchronous replication ports on a source system must be able to see and/or route to all asynchronous replication ports on the destination system, and the other way around. This also means that if more than two systems were part of the replication topology, all interfaces on all systems could communicate with each other. This requirement limited the possible replication topologies that could be supported on the system.
In Dell UnityOS version 5.1 the asynchronous replication interface pairing requirement that all asynchronous replication interfaces must be able to connect to each other has been removed. In UnityOS 5.1 and later, the replication connection requires that replication interfaces on each SP of the source system be able to see and/or route to replication interfaces on both SPs of the destination system, and vice versa. When the remote connection is being created, only valid paths between the systems will be used for the connection. This feature allows for more advanced replication topologies to be created. Management connectivity is still required between the local system and all remote systems.
With the change in the UnityOS 5.1 release and later, replication connections to multiple remote systems can be created over different physical networks. This allows a common system to remotely connect to peer systems that are physically isolated from each other. In Figure 20 below, System 2 has replication interfaces created on physical ports that spread across Network A and Network B. Interfaces connecting to Network A are used to connect to System 1, while interfaces created on Network B ports only connect to System 3. In this example, System 1 and System 3 are on different data networks and cannot reach each other. In this topology, the management port on System 2 is still required to have access to both systems simultaneously.
Figure 20. Physical network isolation example
Another feature in UnityOS version 5.1 and later is the ability to create replication interfaces on different VLANs. Replication interfaces created with different VLAN IDs can be used for connectivity to different systems. As shown in Figure 21, multiple replication interfaces can be created on the same physical port but can be tagged with different VLAN IDs. In this example System 2 makes a remote connection with System 1 over VLAN 1000, while System 2 is using VLAN 1100 to connect to System 3. In this configuration, System 1 and System 3 would not be able to connect to each other as they as tagged with different VLANs. Duplicate IP addresses are not allowed in this configuration. Topologies including a mix of physical network isolation and VLAN isolation is also supported. As mentioned previously, management connectivity is still required between the local system and all remote systems.
Figure 21. VLAN isolation example
To support the new replication interface topologies supported in the UnityOS 5.1 release, all systems in the configuration must be running Dell UnityOS version 5.1 or later. If a system is not running version 5.1 or later, the remote connections within the topology cannot validate as the previous verification rules will apply. When changes to the replication configuration occur, the remote connections need to be re-validated to pick up any changes. Changes include adding or removing a replication interface or changing the settings within an existing replication interface.
Before a remote replication session can be created, a trusted link must be created between systems participating in replication. A replication connection establishes this logical link between remote systems. This trusted link is used for replication management operations and the data path between the pair of remote systems. After the replication interfaces are created, you must create the replication connection between the systems. Once created, all asynchronous remote replication sessions will use the replication connections to transport data to the remote system.
Figure 22 below shows an example configuration of an asynchronous replication connection between two physical systems. In the figure, the source of the replication session is labeled “Production System”, and the destination is labeled “DR System”. For each of these example systems, the Dell Unity 480 system has 10 GbE ports configured as replication interfaces. The green lines in the figure show the connections for the 10GbE replication interfaces, and the orange lines are the system management connections. Once the replication interfaces are created and cabled to the network on both systems, the replication connection between the systems can be made. Once a replication connection is configured on one of the systems participating in replication, it is automatically created on the peer system.
Verify and Update is a single operation that is used to update the replication connection information about the system it is issued on. This operation is performed on the replication connection itself, as opposed to an individual replication session. Verify and Update can be used to test a replication connection to a remote system or update the replication information if changes to the system have been made. Verify and Update should be issued to reestablish the replication connection to a remote system after an outage. A use case for using Verify and Update is if an Asynchronous Replication Interface has been added or an IP Address has been changed on the system.
Figure 22. Native asynchronous replication example
In Dell UnityOS version 5.1 and later asynchronous replication traffic can be throttled to reduce the rate at which data is replicated to a destination system. Asynchronous replication throttling is configured at the replication connection level, which allows each remote system connection to be controlled independently of each other. Also, only outbound replication traffic to a remote system is throttled. This not only allows different throttles to and from a remote system over the replication connections, but also allows replication traffic to be throttled from a system running UnityOS 5.1 and later to a system running an earlier release. This feature can be managed using Unisphere, Unisphere CLI, and REST API.
Asynchronous replication throttling is controlled by bandwidth schedules configured directly on a remote system connection. Bandwidth schedules can be configured when creating a replication connection to a new system or added and modified at any time on an existing connection. By default, no bandwidth schedules exist. Figure 23 below shows the Bandwidth Schedules step within the Create Replication Connection window. Here, one or more bandwidth schedules can be added when creating the replication connection. Available options on this screen include Add, Delete, View/Edit, Move up, and Move down.
Figure 23. Create Replication Connection – Bandwidth Schedules step
When creating a bandwidth schedule, the following information can be customized:
Figure 24 below shows an example of the Add Bandwidth Schedule window. In this window, the Maximum Bandwidth and Hours of the day have been customized. As the Days of the Week checkboxes are not set, this schedule will run on all days once created. When configuring the Start Hour and the End Hour, the timing entered directly depends on the Schedule Time Zone set on the system. As shown in this example, when the system’s Schedule Time Zone setting is configured as UTC Legacy (default), the drop-down options must be configured with the UTC Legacy offset. Based on the selections, the timing is also displayed taking into account the current time zone offset of the computer Unisphere is being accessed from. This schedule will run as configured between 8:00 AM (UTC-04:00) and 5:00 PM (UTC-04:00). The Schedule Time Zone setting is discussed later in this section.
Figure 24. Add Bandwidth Schedule
Multiple bandwidth schedules can be created and can overlap. This allows replication traffic to be controlled differently depending on the day of the week and time of day. When multiple bandwidth schedules are present on the connection, the first schedule matching the current day and time will be enforced. The Move up and Move down arrows can be used to reorder the bandwidth schedules list as needed. If one schedule limits the replication traffic during a specific time of the day, and another is a limit for all days and times, the specific schedule should be listed first. Duplicate schedules can be created, but only the first occurrence of the schedule will be enforced. If duplicate schedules exist on the connection, a warning is displayed and a hyperlink to select the duplicates is presented so they can easily be deleted.
Figure 25 below shows an example configuration on the Bandwidth Schedules tab within the properties of the replication connection. In this window all bandwidth schedules for the replication connection are shown, along with the Current bandwidth limit, which lists the current limit in KB/sec. A note about the display times for the window is also shown.
In this example, on Saturdays and Sundays the replication connection uses all available bandwidth of the link as no limit is defined. On Mondays, Tuesdays, Wednesdays, Thursdays, and Fridays, limits are created for various times during the day. In this example, the 7:00 AM – 9:00 AM and 4:00 PM – 7:00 PM schedules are listed and do not overlap. Both schedules will be enforced on the replication connection. The fourth schedule on the list is configured for 0 KB/sec between 7:00 AM and 7:00 PM. This the replication sessions from running on the connection. As this schedule overlaps with day and times defined earlier in the list, this schedule will only run during times that are not previously configured (9:00 AM to 4:00 PM). The last schedule in this example is set to run at all times on all days and enforces a 204800 KB/sec limit. As this is the last schedule in the list, all other schedules would run before this schedule is encountered. If this schedule was listed first, then no other schedules would run for the remote connection.
Figure 25. Replication Connection properties – Bandwidth Schedules tab
When a bandwidth schedule with a KB/sec limit above zero is defined, all active replication sessions on the replication connection will share the bandwidth allowance. Using the 102400 KB/sec limit as an example, all sessions actively replicating to the remote system will share the 102400 KB/sec limit. As replication session updates end and begin, the bandwidth allowance for each active session is adjusted. Bandwidth schedules are enforced at the replication session level not at the replication interface level and are in no way impacted by the number of replication interfaces used by the remote connection on the system.
Due to seasonal time changes in certain regions of the world, the bandwidth schedules can become out of sync with the local time. For example, a schedule starting at 8:00AM local time in one part of the year may start at 9:00AM in another part of the year. To avoid this issue, a user must manually validate and correct the settings of the bandwidth schedules when a seasonal time change has occurred.
In Dell UnityOS version 5.1 and later the Schedule Time Zone option can be set to correct bandwidth schedule timing. This feature automatically adjusts the timing of bandwidth schedules with the seasonal time changes of the assigned time zone. The Schedule Time Zone option not only applies to asynchronous replication throttling, but also snapshot schedules. This option can be found in Settings > Management > Schedule Time Zone, as shown in Figure 26. A link to this page is also available within the Create Schedule page, as shown in Figure 24.
Figure 26. Configure Schedule Time Zone
After installing or upgrading to the 5.1 release or later, the system is set to UTC Legacy setting by default. UTC Legacy keeps the default handling and does not adjust schedules due to seasonal time changes. The user has the option to choose from over 100 times zones, covering over 300 cities around the world. The time zone of a system can be changed at any time, which may be required if the system is relocated from one site to another.
After changing the time zone, it is recommended to verify the timing is correct for bandwidth schedules that are in use on the system. When the Schedule Time Zone option is changed, the timing of schedules is not updated to reflect the new time zone. This may cause bandwidth throttle to run at the incorrect time. If the timing is no longer correct for a bandwidth schedule, it should be modified to reflect the required timing.
In the following example, the Schedule Time Zone is set to (UTC-05:00) Eastern Time (US & Canada). The Add Bandwidth Schedule window displays the time zone setting on the system. When configuring the Start Hour and End Hour, the timing will be based on the time zone of the system. At the timing of this update, daylight saving time / summer time is in effect. The Add Bandwidth Schedule window also displays the schedule time based on the current time zone offset of the computer Unisphere is being accessed from. In this case the current time is based on UTC-4:00. Both times are displayed to confirm proper timing.
Figure 27. Add Bandwidth Schedule
A replication session is used as a logical link between storage resources participating in replication. When using asynchronous remote replication, a configured replication connection is used to transfer data to the remote system. A replication connection is not required when asynchronously replicating data between Pools within a system. When a replication session is created in Unisphere, a storage resource of the same size and type is created on the destination Pool, whether it is locally within the same system or on a destination system. Snapshot schedules are not replicated on the destination storage resource.
Asynchronous replication image synchronizations are triggered by a user-defined Recovery Point Objective (RPO) or at any time manually by the user. The following characteristics define asynchronous replication:
Manual replication operates the same as asynchronous replication, except all replication updates must be started manually by the user as opposed to using an RPO. Updates occur when the sync operation is initiated on the session, which is described later. When writing to a storage resource configured with a manual replication session, the data is stored locally and acknowledged and only replicated when a sync is issued.
When an asynchronous replication session is created, a full synchronization of the source and destination storage resource may be required. If replication is configured when a new resource is being created, the synchronization for block resource is quick as no data needs to be copied to the destination storage resource. For file resource, the synchronization copies, at minimum, 1.5 GB worth of meta data to the destination storage resource. If replication is added to an existing storage resource, a full synchronization may be required between the source and destination storage resource if no common base snapshot exists. In Dell UnityOS 5.1 and later, a replicated snapshot may be used to avoid a full synchronization under certain circumstances. This feature is discussed in the Common Base Snapshots section of this paper. Writes occurring during the initial synchronization period are not copied to the destination storage resource at this time, but rather tracked for a later synchronization. Once the initial synchronization is complete, a common base is established between the source storage resource and the destination. When creating a manual replication session, an initial synchronization does not automatically start. To synchronize the local and remote images, a manual sync needs to be initiated. Host write operations which occur after the initial synchronization are acknowledged with the host normally, and no data is replicated to the destination until the next sync. Manually at a later time, or at the RPO interval for asynchronous replication, all changes made to the source storage resource since the last synchronization will be replicated to the destination. A new common base is then established. If a failure is encountered on the source, all data not copied to the destination will be lost as the changes have not been copied to the destination.
Asynchronous replication in Dell Unity leverages Unified Snapshots to maintain the common base images explained in the previous section. Figure 28 below shows how Unified Snapshots are used with asynchronous and manual replication. This section assumes no common base snapshots exist at time of session creation.
Figure 28. Asynchronous and Manual Replication Theory
Dell Unity Asynchronous Replication operates in the following way:
Each time the RPO is reached or a manual update is started, the common base image will alternate between Snapshot 1 and Snapshot 2.
Snapshots used for asynchronous replication behave the same as user Unified Snapshots and are based on redirect on write technology. Space needed to preserve the point-in-time snapshot is allocated from the same Pool as the source storage resource. Although user snapshots and replication snapshots share the same technology, replication snapshots have restrictions on their usage. Replication snapshots can be viewed in Unisphere, but user operations, such as restore, or mount operations are not allowed. Snapshots allocated for replication purposes do not count towards user snapshot maximums.
When configuring asynchronous replication, the source and destination storage resource must be of the same type. In Dell Unity, native asynchronous replication is supported on the following storage resources:
In Dell Unity, asynchronous replication for LUNs, thin clones, and VMware VMFS Datastores function the same. When configuring asynchronous replication on a LUN in Unisphere, a single replication session is created, and the destination storage resource will be created the same size and type as the source storage resource. When configuring replication session on a thin clone, the destination storage resource will be a regular storage resource and not a clone. While replication is configured, you can extend the size of LUNs, thin clones, and VMware VMFS Datastores, and the changes will be reflected on the destination storage resource after the next sync. Options such as the LUN’s, thin clone’s, or VMware VMFS Datastore’s name or tiering policy may be configured differently between systems.
In Dell Unity, a consistency group is a storage resource which contains one or more LUNs within a storage system. consistency groups help organize storage resources allocated for a particular host or hosts. Consistency groups are treated as a single entity when they are replicated, meaning that a single replication session is created for the entire consistency group no matter how many LUNs it contains. When replication is configured in Unisphere for a consistency group, the destination storage resource and its contents are created automatically. While a consistency group is part of an asynchronous replication session, LUNs within the consistency group can be expanded. All changes to LUNs within a consistency group will be reflected on the destination image after the next completed synchronization. LUNs cannot be added or removed while replication is configured. When pausing or resuming replication on a consistency group, the entire group is affected by the replication operation.
When replicating existing file systems or VMware NFS Datastores with asynchronous replication, you must first configure replication for the NAS Server it is mounted on. If replication is configured in Unisphere for the NAS Server, all file systems and NFS Datastores on the NAS Server will also be replicated to the destination. Replication sessions for resources which do not require replication can be deleted later. All replication sessions automatically configured when the NAS Server is replicated will have the same RPO. The RPO for the individual replication sessions can be changed later. While replicated, all size changes to the file systems and NFS Datastores will be reflected on the destination after the next synchronization. If a different IP address for NAS Server access needs to be specified on the destination NAS Server, you may specify an Override Address by editing a NAS Server Network Interface.
For asynchronous replication to operate, two storage resources are required for replication:
When a replication session is created in Unisphere, the destination storage resource is automatically created. In code versions prior to the UnityOS 5.1 release, you may not choose a previously provisioned storage resource as a replication destination when configuring a session in Unisphere. In Dell UnityOS version 5.1 and later, Unisphere includes an option to reuse a destination resource if one exists. This feature is discussed in detail in the Common Base Snapshots section. Upon creation, the destination resource is marked as a destination image. This restriction blocks Read/Write access on the destination storage resource. To view the data contained in the destination storage resource, you may take a snapshot of the resource and provide host access to the snapshot.
In Unisphere, you can easily determine which storage resources are replicated and which are destination images from any of the storage resource pages, such as the LUNs tab on the Block page. While in this tab, select the gear icon, and hover over the Columns option to view the available columns that can be viewed. Select the check boxes for “Replication Type” and “Restricted Replication Access”. Replication Type displays what type of replication the storage resource is participating in, whether it is None, Remote, or Local. The Restricted Replication Access column will display “Yes” if the storage resource is labeled as a replication destination resource, or “No” if it is not. If a replication session is deleted, the destination storage resource will still be labeled as a replication destination image. The replication destination label must be edited manually over Unisphere CLI before the resource is allowed to receive I/O. For example, to remove the replication destination setting from a LUN, use the uemcli /stor/prov/luns/lun -id <value> set -replDest no command. For more information about removing the replication destination setting on other storage resource types, consult the Dell Unity Unisphere Command Line Interface User Guide.
In Dell UnityOS versions 5.1 and later, Source, Destination, and All filtering buttons on the replication sessions page and various storage resource pages help the user easily identify replication source and destination resources/sessions without adding columns to the view. When All is selected, all resources/sessions on the current page are displayed. When Source is selected on a resource page, all resources that are the source of a replication session are displayed. Resources that are not replicated are also shown when Source is selected. When Source is selected on the replication sessions page, only replication session originating on the system are shown. When Destination is selected on a resource page, only resources that are the destination images of a replication session are shown. While on the sessions page, Destination will only show the sessions replicating to the current system. Also, sessions that are part of local replication are displayed regardless of which view is selected.