Your Browser is Out of Date

Nytro.ai uses technology that works best in other browsers.
For a full experience use one of the browsers below
High Availability and Data Protection with Dell PowerScale Scale-Out NAS
Executive summary High availability and reliability at the petabyte scale High availability with OneFS OneFS fault tolerance High availability and data protection strategies
Overview
OneFS high availability and data protection suite
Connection load balancing and failover—SmartConnect
Snapshots
Writable snapshots
SmartDedupe
Small File Storage Efficiency
Inline data reduction
Replication
Archiving and data security
CloudPools
SmartLock
Data Encryption at Rest
Audit
OneFS hardening
Secure Remote Services and SupportAssist
File filtering
Nearline, VTL, and tape backup
FC backup accelerator card
Backup from snapshots
Parallel streams
NDMP
Direct NDMP model
Remote NDMP model
Certified backup applications
Summary OneFS glossary

Small File Storage Efficiency

Home > Storage > PowerScale (Isilon) > Product Documentation > Data Protection > High Availability and Data Protection with Dell PowerScale Scale-Out NAS > Small File Storage Efficiency

Small File Storage Efficiency

  • Another principal consumer of OneFS shadow stores is Small File Storage Efficiency. This feature maximizes the space utilization of a cluster by decreasing the amount of physical storage required to house the small files that often consist of an archive dataset, such as found in healthcare PACS workflows.

    Efficiency is achieved by scanning the on-disk data for small files, which are protected by full copy mirrors, and packing them in shadow stores. These shadow stores are then parity protected, rather than mirrored, and typically provide storage efficiency of 80 percent or greater.

    Graphic illustrating OneFS file packing, where multiple small files are containerized within a shadow store, which is then parity protected, reducing on-disk overhead.

    Small file containerization

    Small File Storage Efficiency trades a small read latency performance penalty for improved storage utilization. The archived files obviously remain writable, but when containerized files with shadow references are deleted, truncated, or overwritten it can leave unreferenced blocks in shadow stores. These blocks are later freed and can result in holes, which reduces the storage efficiency.

    The efficiency loss depends on the protection level layout used by the shadow store. Smaller protection group sizes are more susceptible, as are containerized files, since all the blocks in containers have at most one referring file and the packed sizes (file size) are small.

    A shadow store defragmenter is integrated into the ShadowStoreDelete job to help reduce the fragmentation of files as a result of overwrites and deletes. The defragmentation process works by dividing each containerized file into logical chunks (~32 MB each) and assessing each chunk for fragmentation.

    If the storage efficiency of a fragmented chunk is below target, that chunk is processed by evacuating the data to another location. The default target efficiency is 90 percent of the maximum storage efficiency available with the protection level used by the shadow store. Larger protection group sizes can tolerate a higher level of fragmentation before the storage efficiency drops below this threshold.