Your Browser is Out of Date

Nytro.ai uses technology that works best in other browsers.
For a full experience use one of the browsers below
Dell PowerMax 2500 and 8500: Data Reduction
Executive summary Data reduction
Overview
Activity-based reduction
Compression
Deduplication
Deduplication algorithm
Compaction
Extended data compression
CKD compression
Data reduction I/O flow
System resource usage Managing and monitoring system capacity and resources Supported data services Conclusion

Deduplication

Home > Storage > PowerMax and VMAX > Storage Admin > Dell PowerMax 2500 and 8500: Data Reduction > Deduplication

Deduplication

  • Deduplication is a capacity-savings method that identifies identical copies of data and stores a single instance of each copy. There are a few deduplication components that are required to provide efficient capacity savings:

    • Hash ID: The hash ID is a unique identifier for incoming data that is used to determine if a deduplication relationship is needed. The system uses a SHA-256 algorithm to generate the hash ID.
    • Hash ID table: Hash ID tables are an allocation of system memory distributed between the system directors. These tables catalog the hash IDs used by the deduplication process. Entries in the table are used to determine if a deduplication relationship exists, or if a new entry is required and the data can be stored on disk. 
    • Deduplication Management Object (DMO): The DMO is a 64-byte object within system memory that only exists when a deduplication relationship exists. These objects store and manage the pointers between front-end devices and the deduplicated data that consumes backend capacity in the array. DMOs manage the pointers for deduplicated data between front-end devices and the data stored on disk. They also manage what hash ID table the hash IDs are stored in when deduplication relationships exist.

    Deduplication uses the same data reduction hardware as compression, and a unique hash ID is generated when the hardware processes the data. Then, the hash ID is compared to the hash ID table to look for the same ID. When a match is found, the data is not stored on disk, and a deduplication share is created. Pointers are set between the front-end volume and the unique ID in the hash ID table. The pointers link the single instance of data stored on disk to the volume, providing future access to the data. The DMO manages the pointers between the data, the front-end volumes accessing the data, and the hash ID table. When there is no match in the hash ID table, a new entry is added for future hash ID comparisons.