The ScoutAM Policy Engine controls the number of data copies, grouping of data, and placement of data on premises and in the cloud.

Different sets of automated policies are configured in advance and then applied to cache data to generate the desired number of copies and orchestrate writes to a composable mix of mass storage destinations. Policy definitions enable sophisticated control over the grouping of data by user, file type, application, file size, group, creation/modification time, access time or any combination of these attributes. Policy definitions also set boundaries on the time that can pass prior to the creation of copies, as well as boundaries on the data set size. The ability to apply differentiated policies to coalesce random incoming data streams into efficient streaming data sets is one of the core functions of the platform.

Data sets are containerized into the open-source PAX Tar format, for read/write efficiency. This format has distinct benefits for small files, which are grouped and then written to mass storage, in package sizes that are optimal for obtaining maximum throughput of the target archive storage devices and media. When retrieving data, the system can recall files or objects individually without reading the entire PAX Tar file. Large files may be divided into segments of configurable size and then efficiently streamed to tape drives in parallel. This capability allows ScoutAM to stripe large files across an arbitrary number of drives simultaneously. ScoutAM allows files that were written in parallel across many drives to be re-read with a different number of drives so that administrators can control drive resources based upon workloads. Non-segmented files are written from all nodes using a round robin algorithm.

Archive resources can include any combination of tape, private cloud, public cloud, and disk storage systems. The archive resources are specified by configurable policies. For example, a customer may specify a configuration for a specific user or file type that would write copy 1 to a private cloud system, copy 2 to a tape library, copy 3 to AWS, and copy 4 to a different tape library at a remote site. Storage resources may be reserved and prioritized, making scenarios such as placing certain types of data on separate devices within the same data collection possible. It is also possible to ensure that multiple file copies written to the same tape library are always written to different pieces of physical media.