Home > Storage > PowerScale (Isilon) > Industry Solutions and Verticals > Analytics > Algorithmic Trading—Scaling Concurrency and Performance with Dell PowerScale > PowerScale all-flash NAS
Dell PowerScale all-flash nodes are designed to drive demanding workloads at the edge, in the data center, or in the cloud with a choice of NVMe or SSD storage types. PowerScale clusters can scale from 3 to 252 PowerScale nodes with capacities ranging from 11 TB to 186 PB per cluster, making PowerScale ideal for demanding storage requirements in high-volume tick data applications.
The embedded storage operating system, PowerScale OneFS, provides a unifying clustered file system with integrated scalable data protection that simplifies storage management and administration. OneFS is a fully symmetric file system with no single point of failure. It takes advantage of clustering not just to scale performance and capacity but also to allow for any-to-any failover and multiple levels of redundancy that go far beyond the capabilities of RAID. Self-encrypting drives can be used to provide enhanced data security.
The PowerScale OneFS operating system supports near-instantaneous, downtime-free, file system expansion. Hardware can be incorporated or removed from the cluster at will and at any time, abstracting the data and applications from the physical hardware. Tick databases that are stored on OneFS can take advantage of this architecture and increase the capacity and performance without system outages or downtime. Data is given infinite longevity, and the cost and pain of data migrations and hardware refreshes are eliminated.
OneFS is a single-file-system, single-namespace architecture distributed across all nodes in the cluster and accessible by clients connecting to any node in the cluster. There is no partitioning, and there is no need for volume creation because all information is shared among nodes. Data can be written to or read from any node, thus optimizing performance when multiple kdb+ users or applications are concurrently reading and writing to the same set of tick data.
The intelligence of OneFS makes it easy to manage at any scale. The storage nodes powered by OneFS are simple to install, manage, and scale. Adding capacity and performance capabilities to a cluster is significantly easier than with other storage systems—requiring only three simple steps for the storage administrator:
The new node provides additional capacity and performance because each node includes CPU, memory, and network. The AutoBalance feature of OneFS automatically moves data across the back-end network in a coherent manner so existing data that resides on the cluster moves onto the new storage node.
A PowerScale cluster is a group of NAS nodes powered by the PowerScale OneFS operating system. A PowerScale node is a rack-mountable, enterprise-class appliance containing storage capacity, memory, CPU, and networking. As such, each node in the distributed cluster has compute as well as storage capabilities.
Several different types of nodes, all of which can be incorporated into a single cluster, provide varying ratios of capacity and performance. Thus, customers can tier data and meet price and performance requirements by using different PowerScale node types within the same cluster.
Tier | Workload profile | Drive type | Node type |
Massive performance | HPC/high capacity | NVMe | F900 |
Massive performance | HPC/density | NVMe | F6001 |
High performance | Demanding business and analytic applications | NVMe | F600 |
High performance | “Hot edge” and small deployments | SSD | F200 |
High performance | Multi-use | SSD | F800 |
High performance | Multi-use | SSD | F810 |
1 The F600 has a higher performance tier option with more memory and CPU performance per node.
As with most data, only a portion of a historical kdb+ instance is accessed frequently. Generally, the most recent 30 days of trade data is queried frequently, with older data being read periodically. For example, one back history strategy might involve scanning the past 30 days of trade data on a daily basis to validate new and existing algorithms; another strategy might involve scanning the past 180 days of trade data on a weekly basis; and so on.
For this reason, it is most cost effective to develop a tiered approach to storing this data. PowerScale provides this functionality inherently within the OneFS file system using OneFS SmartPools, which moves data from tier to tier without the need to change mount points or directory structure. All data, regardless of storage tier, continues to be accessible to clients, without downtime or application reconfiguration.