A SyncIQ policy may be configured to run with four different options. Each of those options is explained in this section.

Note: Although SyncIQ offers many options for configuring a SyncIQ policy, as explained in this section, Whenever a snapshot of the source directory is taken is the best practice and recommended configuration. For more information about this configuration, see Whenever a snapshot of the source directory is taken.

Manually

The manual option allows administrators to have a SyncIQ Policy completely configured and ready to run when a workflow requires data replication. If continuous data replication is not required and on an as-needed basis, the manual option is the best option. Administrators can just select the policy to run when it is required, limiting cluster overhead and saving bandwidth.

Note: Manual SyncIQ jobs still maintain a source snapshot that accumulates changed blocks. Therefore, it is recommended to run the manual job frequently, to ensure that the source snapshot growth is limited.

On a schedule

Running a SyncIQ Policy on a schedule is one of the more common options. When this option is selected, another drop-down appears, to specify the frequency of the job, as displayed in the following figure.

Figure 13.    SyncIQ Job on a schedule

Options include daily, weekly, monthly, or yearly. When the frequency is selected, further options appear to refine the frequency selection.

Before OneFS 8.0, a snapshot is always taken for scheduled jobs, even if no data changes have occurred since the previous execution. In OneFS 8.0, a policy parameter can be specified so that SyncIQ checks for changes since the last replication as the first step in the policy. If there are no changes, no further work will be done on that policy iteration, and the policy will report as skipped. If there are changes, the source data snapshot will be taken, and the policy will proceed. This capability reduces the amount of work performed by the cluster if there is no changed data to be replicated. To enable this behavior, select Only run if source directory contents are modified on the WebUI or specify –-skip-when-source-unmodified true on the CLI.

Note: As a best practice, avoid the overlap of policy start times or have several policies running during the same time period. As explained in SyncIQ design considerations, consider policy start times and cluster resources. As policies complete, monitor completion times and adjust policy start times to minimize overlap. Staggering policy start times is especially critical for a high-volume dataset.

RPO alerts

An option for sending RPO alerts is available when On a Schedule is selected for a running a job. Administrators can specify an RPO (recovery point objective) for a scheduled SyncIQ policy and trigger an event to be sent if the RPO is exceeded. The RPO calculation is the interval between the current time and the start of the last successful sync job.

Note: The RPO option only appears if RPO is enabled under SyncIQ global settings. From the web interface, select Data Protection > SyncIQ, and then select the Settings tab. The Enable RPO Alerts checkbox is displayed.

For example, consider a policy scheduled to run every 8 hours with a defined RPO of 12 hours. Suppose the policy runs at 3 pm and completes successfully at 4 pm. Thus, the start time of the last successful sync job is 3 pm. The policy should run next at 11 pm, based on the 8-hour scheduled interval. If this next run completes successfully before 3 am, 12 hours since the last sync start, no alert will be triggered, and the RPO timer is reset to the start time of the replication job. If for any reason the policy has not run to successful completion by 3 a.m., an alert will be triggered because more than 12 hours elapsed between the current time (after 3 a.m.) and the start of the last successful sync (3 p.m.).

If an alert has been triggered, it is automatically canceled after the policy successfully completes.

The RPO alert can also be used for policies that have never been run, as the RPO timer starts at the time the policy is created. For example, consider a policy created at 4 pm with a defined RPO of 24 hours. If by 4 pm the next day, the policy has not successfully completed at least one synchronization operation, the alert will be triggered. As stated previously, the first run of a policy is a full synchronization and will probably require a longer elapsed time than subsequent iterations.

An RPO can only be set on a policy if the global SyncIQ setting for RPO is already set to enabled: isi sync settings modify –rpo-alerts true|false. By default, RPO alerts are enabled.

Individual policies by default have no RPO alert setting. Use –-rpo-alert <duration> on the isi sync policies create or modify command to specify the duration for a particular policy.

Whenever the source is modified

The Whenever the source is modified option is also referred to as, “SyncIQ continuous mode” or “Replicate on Change.” When the Whenever the source is modified policy configuration option is selected (or –-schedule when-source-modified on the CLI), SyncIQ will continuously monitor the replication dataset and automatically replicate changes to the target cluster. Continuous replication mode is applicable when the target cluster dataset must always be consistent with the source, or if data changes at unpredictable intervals.

Figure 14.    SyncIQ source modified option

Note: Practice extreme caution with the Whenever the source is modified option because it can trigger a large amount of replication, snapshot, and network traffic if the data is volatile. The source modified option is not synchronous data replication. Consider the cluster resources and frequency of dataset updates when applying this option. It may result in SyncIQ policies constantly running and excessive resource consumption. Another factor to consider is, by default, snapshots of the source directory are taken before each SyncIQ job. If the dataset is frequently modified, many snapshots are triggered, possibly conflicting with other snapshot activity. If selecting this option is necessary, ensure that the sync delay is configured with ample time to encapsulate new data and allows for the policy to complete.

Events that trigger replication include file additions, modifications and deletions, directory path, and metadata changes. SyncIQ checks the source directories every ten seconds for changes, as illustrated in the following figure.

Figure 15.    SyncIQ source modified policy triggers

Before OneFS 8.0, jobs in Continuous Replication mode run immediately after a change is detected. OneFS 8.0 introduces a policy parameter to delay the replication start for a specified time after the change is detected. The delay allows a burst of updates to a dataset to be propagated more efficiently in a single replication event rather than triggering multiple events. To enable the delay for a continuous replication policy, specify the delay period in the Change-Triggered Sync Job Delay option on the UI, as shown in Figure 14, or specify –-job-delay <duration> on the CLI.

Note: As a best practice, if the Whenever the source is modified option is selected, configure the Change-Triggered Sync Job Delay option for a reasonable delay to propagate multiple updates into a single update.

Whenever a snapshot of the source directory is taken

A SyncIQ policy can be configured to trigger when the administrator takes a snapshot of the specified source directory and matching a specified pattern as displayed in the following figure.

Figure 16.    Whenever a snapshot of the source directory is taken

If this option is specified, the administrator-taken snapshot will be used as the basis of replication, rather than generating a system snapshot. Basing the replication start on a snapshot is useful for replicating data to multiple targets. They can all be simultaneously triggered when a matching snapshot is taken, and only one snapshot is required for all the replications. To enable this behavior, select the Whenever a snapshot of the source directory is taken policy configuration option on the UI. Alternatively, from the CLI, use the flag, --schedule=when-snapshot-taken.

All snapshots taken of the specified source directory trigger a SyncIQ job to start, replicating the snapshot to the target cluster. An administrator may limit all snapshots from triggering replication by specifying a naming convention to match in the Run job if snapshot name matches the following pattern: field. By default, the field contains an asterisk, triggering replication for all snapshots of the source directory. Alternatively, from the CLI, if the flag --snapshot-sync-pattern <string> is not specified, the policy automatically enters an asterisk, making this flag optional.

The checkbox Sync existing snapshots before policy creation time is displayed only for a new policy. If an existing policy is edited, this option is not available. Alternatively, from the CLI, the flag --snapshot-sync-existing is available for new policies. The Sync existing snapshots before policy creation time option replicates all snapshots to the target cluster that were taken on the specified source cluster directory.

Note: The Whenever a snapshot of the source directory is taken is the best practice and recommended policy for scheduling SyncIQ policies. Further, the when-snapshot-taken SyncIQ policy schedule should be driven by first creating a SnapshotIQ policy on the source directory with the desired schedule. After configuring the SnapshotIQ policy, the when-snapshot-taken SyncIQ policy can be created or modified to use the SnapshotIQ schedule and the --snapshot-sync-existing option. For more information about SnapshotIQ and SyncIQ, see SnapshotIQ and SyncIQ.

When snapshots are replicated to the target cluster, by default, only the most recent snapshot is retained and the naming convention on the target cluster is system generated. However, to prevent only a single snapshot from being overwritten on the target cluster and the default naming convention, select the Enable capture of snapshots on the target cluster as stated in Target snapshots. When this checkbox is selected, specify a naming pattern and select the Snapshots do not expire option. Alternatively, specify a date for snapshot expiration in the Snapshots expire after option. Limiting snapshots from expiring ensures that they are retained on the target cluster rather than overwritten when a newer snapshot is available. The target cluster snapshot options map to --target-snapshot-archive, --target-snapshot-alias, --target-snapshot-expiration, and --target-snapshot-pattern in the CLI.

Another feature of snapshots and SyncIQ is the option to retain an original snapshot’s name and creation time from the source cluster in the field, Existing snapshot naming pattern, as shown in the following figure. The default pattern in the Existing snapshot naming pattern field is %(SnapName)-%SnapCreateTime. SnapName refers to the original snapshot’s name and SnapCreateTime refers to the original snapshot creation time. Both naming patterns are based on the name and creation time originally from the source cluster, allowing administrators to manage snapshots on the target cluster easily. Alternatively, from the CLI, the Existing snapshot naming pattern field maps to –-sync-existing-target-snapshot-pattern.

Also, to have the snapshots use the expiration specified on the source cluster, select Snapshots use same expiration dates as source. Alternatively, from the CLI, this  option maps to --sync-existing- -snapshot-expiration in the CLI. Furthermore, the Snapshots use same expiration dates as source checkbox only appears if the Sync existing snapshots before policy creation time checkbox (or the snapshot-sync-existing flag in the CLI) was selected under the “Basic Settings” tab when the policy is configured.

Figure 17.    Target cluster snapshot naming and retention

Note: Configuration of snapshots for automatic capture based on a time-frequency triggers the SyncIQ policy to run. If SyncIQ policies are constantly running, consider the impact on system resources before configuring them. As with any major storage infrastructure update, test the configuration in a lab environment before updating a production cluster to ensure that all resource impacts are considered and calculated.

Alternatively, SyncIQ also provides an option for manually specifying an existing snapshot for SyncIQ replication, as explained in SnapshotIQ and SyncIQ.