Home > Storage > PowerScale (Isilon) > Product Documentation > Data Efficiency > Dell PowerScale OneFS: Data Reduction and Storage Efficiency > Defragmentation
When containerized files with shadow references are deleted, truncated, or overwritten it can leave unreferenced blocks in shadow stores. These blocks are later freed and can result in holes, which reduce the storage efficiency.
The actual efficiency loss depends on the protection level layout used by the shadow store. Smaller protection group sizes are more susceptible, as are containerized files, since all the blocks in containers have at most one referring file and the packed sizes (file size) are small.
In OneFS 8.2 and later, a shadow store defragmenter is added to reduce fragmentation resulting of overwrites and deletes of files. This defragmenter is integrated into the ShadowStoreDelete job. The defragmentation process works by dividing each containerized file into logical chunks (~32MB each) and assessing each chunk for fragmentation.
If the storage efficiency of a fragmented chunk is below target, that chunk is processed by evacuating the data to another location. The default target efficiency is 90% of the maximum storage efficiency available with the protection level used by the shadow store. Larger protection group sizes can tolerate a higher level of fragmentation before the storage efficiency drops below this threshold.
The ‘isi_sstore list’ CLI command can be run to report fragmentation and efficiency scores. For example:
The fragmentation score is the ratio of holes in the data where FEC is still required, whereas the efficiency value is a ratio of logical data blocks to total physical blocks used by the shadow store. Fully sparse stripes do not need FEC so are not included. The guideline is that lower fragmentation scores and higher efficiency scores are better.
The defragmenter does not require a license to run and is disabled by default in OneFS 8.2 and later. It can be easily activated using the following CLI commands:
# isi_gconfig -t defrag-config defrag_enabled=true
Once enabled, the defragmenter can be started by the job engine’s ShadowStoreDelete job, either from the OneFS WebUI or by the following CLI command:
# isi job jobs start ShadowStoreDelete
The defragmenter can also be run in an assessment mode. This reports on and helps to determine the amount of disk space that will be reclaimed, without moving any actual data. The ShadowStoreDelete job can run the defragmenter in assessment mode but the statistics generated are not reported by the job. The isi_sstore CLI command has a ‘defrag’ option and can be run with the following syntax to generate a defragmentation assessment:
# isi_sstore defrag -d -a -c -p -v
…
Processed 1 of 1 (100.00%) shadow stores, space reclaimed 31M
Summary:
Shadows stores total: 1
Shadows stores processed: 1
Shadows stores skipped: 0
Shadows stores with error: 0
Chunks needing defrag: 4
Estimated space savings: 31M