SmartDedupe considerations include:
- SmartDedupe will not share blocks across files with different protection policies applied.
- OneFS metadata, including the deduplication index, is not deduplicated.
- SmartDedupe will not attempt to deduplicate files smaller than 32 KB in size.
- Dedupe job performance will typically improve significantly on the second and subsequent job runs after the initial index and the bulk of the shadow stores have already been created.
- SmartDedupe will not deduplicate the data stored in a snapshot. However, snapshots can certainly be created of deduplicated data.
- If deduplication is enabled on a cluster that already has a significant amount of data stored in snapshots, it will take time before deduplication affects the snapshot data. Newly created snapshots will contain deduplicated data, but older snapshots will not.
- SmartDedupe deduplicates common blocks within the same file, resulting in even better data efficiency.
For more information, see the OneFS SmartDedupe white paper.