Your Browser is Out of Date

Nytro.ai uses technology that works best in other browsers.
For a full experience use one of the browsers below
High Availability and Data Protection with Dell PowerScale Scale-Out NAS
Executive summary High availability and reliability at the petabyte scale High availability with OneFS OneFS fault tolerance High availability and data protection strategies
Overview
OneFS high availability and data protection suite
Connection load balancing and failover—SmartConnect
Snapshots
Writable snapshots
SmartDedupe
Small File Storage Efficiency
Inline data reduction
Replication
Archiving and data security
CloudPools
SmartLock
Data Encryption at Rest
Audit
OneFS hardening
Secure Remote Services and SupportAssist
File filtering
Nearline, VTL, and tape backup
FC backup accelerator card
Backup from snapshots
Parallel streams
NDMP
Direct NDMP model
Remote NDMP model
Certified backup applications
Summary OneFS glossary

SmartDedupe

Home > Storage > PowerScale (Isilon) > Product Documentation > Data Protection > High Availability and Data Protection with Dell PowerScale Scale-Out NAS > SmartDedupe

SmartDedupe

  • Shadow stores provide the basis for SmartDedupe, which maximizes the storage efficiency of a cluster by decreasing the amount of physical storage required to house an organization’s data. Efficiency is achieved by scanning the on-disk data for identical blocks and then eliminating the duplicates. This means that initial file write or modify performance is not impacted, since no additional computation is required in the write path.

    Illustration of a post-process SmartDedupe job deduplicating a file.

    OneFS storage efficiency with SmartDedupe

    When SmartDedupe runs for the first time, it scans the dataset and selectively samples data blocks from it, creating the fingerprint index. The index is scanned for duplicates. When a match is found, a byte-by-byte comparison of the blocks is performed to verify that they are identical and to ensure that there are no hash collisions. Then, if the blocks are determined to be identical, duplicate blocks are removed from the actual files and replaced with pointers to the shadow stores.

    For more information, see the Dell PowerScale OneFS: Data Reduction and Storage Efficiency white paper.