Snapshots always carry a trade-off between cluster resource consumption (CPU, memory, disk) and the benefit of increased data availability, protection, and recovery.
Figure 12. SnapshotIQ integration with NDMP backups
OneFS SnapshotIQ creates snapshots at the directory-level instead of the volume-level, thereby providing improved granularity.
There is no requirement for reserved space for snapshots in OneFS. Snapshots can use as much or little of the available file system space as desirable.
Snapshots can either be manually taken on-demand or automated with a snapshot schedule.
Snapshot scheduling allows cluster administrators to automatically generate snapshots according to a predefined itinerary. OneFS snapshot schedules can be configured at daily, weekly, monthly, or yearly intervals, with single or multiple job frequency per schedule, and down to a per-minute granularity. Similarly, automatic snapshot deletion can be configured per defined schedule at an hourly through yearly range.
- An ordered deletion schedule is simple to configure but retains a larger number of snapshots and is recommended for datasets with a lower rate of change.
- For more active data, an unordered deletion schedule can prove more effective. The configuration and monitoring overhead is slightly higher, but fewer snapshots are retained.
The following table provides a suggested snapshot schedule for both ordered and unordered deletion configurations.
Table 6. Snapshot schedule recommendations
Ordered deletion (for mostly static data) | Every four hours | Start at 12:00AM End at 11:59AM | 1 month | 180 |
Unordered deletion (for frequently modified data) | Every other hour | Start at 12:00AM End at 11:59AM | 1 day | 27 |
Every day | At 12:00AM | 1 week |
Every week | Saturday at 12:00AM | 1 month |
Every month | First Saturday of month at 12:00AM | 3 months |
For optimal cluster performance, we recommend observing the following SnapshotIQ best practices.
- Use an ordered snapshot deletion strategy where viable.
- Configure the cluster to take fewer snapshots, and for the snapshots to expire more quickly, so that less space will be consumed by old snapshots. Take only as many snapshots as you need and keep them active for only as long as you need them.
- Using SmartPools, snapshots can physically reside on a different disk tier than the original data. The recommendation, however, is to keep snapshots on the same tier on which they were taken.
- The default snapshot limit is 20,000 per cluster. We recommend limiting snapshot creation to 1,024 per directory.
- Limit snapshot depth to a maximum of 275 directories.
- Avoid creating snapshots of directories that are already referenced by other snapshots.
- It is recommended that you do not create more than 1000 hard links per file in a snapshot to avoid performance degradation.
- Creating snapshots of directories higher on a directory tree will increase the amount of time it takes to modify the data referenced by the snapshot and require more cluster resources to manage the snapshot and the directory.
- Avoid taking snapshots at /ifs level. Taking snapshots at a parent dataset level is recommended, enabling faster snapshot deletions and avoiding management complexities. In particular, avoid taking nested snapshots, redundant snapshots, or overly scoped snapshots. For example, if you schedule snapshots of /ifs/data and /ifs/data/foo and/ifs/data/foo/bar, consider taking snapshots of only the intermediate or most granularly scoped part (/ifs/data/foo or /ifs/data/foo/bar).
- If you intend on reverting snapshots for a directory, it is recommended that you create SnapRevert domains for those directories while the directories are empty. Creating a domain for a directory that contains less data takes less time.
- Delete snapshots in order, beginning with the oldest. Where possible, avoid deleting snapshots from the middle of a time range. Newer snapshots are mostly pointers to older snapshots and deleting them will not free up much space. Deleting the oldest snapshot ensures you will actually free up the space. You can determine snapshot order (if not by name or date) by using the isi snapshots list command. The snapshot IDs (first column) are non-conserved, serial values.
- Configure the SSD Strategy to “Use SSDs for metadata read/write acceleration” for faster snapshots deletes.
- Quotas can be used to calculate a file and directory count that includes snapshot revisions, provided the quota is configured to include snaps in its accounting using the “--snaps=true” configuration option.
- SnapshotDelete will only run if the cluster is in a fully available state, (no drives or nodes are down).
- A snapshot schedule cannot span multiple days: To generate snapshots from 5:00 PM Monday to 5:00 AM Tuesday, create one schedule that generates snapshots from 5:00 PM to 11:59 PM on Monday, and another schedule that generates snapshots from 12:00 AM to 5:00 AM on Tuesday.
- If a directory is moved, you cannot revert any snapshots of that directory that were taken before its move.
- Do not delete SyncIQ snapshots (snapshots with names that start with SIQ), unless the only remaining snapshots on the cluster are SyncIQ snapshots, and the only way to free up space is to delete those SyncIQ snapshots.
Writable snapshots, first available in OneFS 9.3, provide the ability to create fast, simple, lightweight copies of datasets. They provide a modifiable view of a regular snapshot, presented at a target directory and accessible by clients across the full range of supported NAS protocols. This allows disaster recovery procedures to be routinely tested on identical, space-efficient, temporary copies of production data.
For more information, see the Data Protection with Dell PowerScale SnapshotIQ white paper.