Home > Storage > PowerScale (Isilon) > Product Documentation > Storage (general) > PowerScale OneFS SmartFlash > Level 1 cache
The level 1 (L1) cache, or front-end cache, is memory that is nearest to the protocol layers (NFS, SMB, HDFS, S3, and so on) used by clients, or initiators, connected to that node. The primary purpose of L1 cache is to prefetch data from remote nodes. Data is prefetched per file, and is optimized in order to reduce the latency associated with the nodes’ InfiniBand (IB) back-end network. Because the back-end interconnect latency is relatively small, the size of L1 cache, and the typical amount of data stored per request, is less than L2 cache.
L1 cache is also known as remote cache because it contains data retrieved from other nodes in the cluster. It is coherent across the cluster but only the node on which it resides uses it. It is not accessible by other nodes. Data in L1 cache on storage nodes is aggressively discarded after it is used. L1 cache uses file-based addressing, in which data is accessed through an offset into a file object.
The L1 cache refers to memory on the same node as the initiator. It is only accessible to the local node, and typically the cache is not the primary copy of the data. It is analogous to the L1 cache on a CPU core, which might be invalidated as other cores write to main memory.
L1 cache coherency is managed through a MESI-like protocol using distributed locks, as described in Introduction to OneFS cache coherency.
OneFS also uses a dedicated inode cache in which recently requested inodes are kept. The inode cache frequently has a large impact on performance. Clients often cache data, and many network I/O activities are primarily requests for file attributes and metadata, which can be quickly returned from the cached inode.
Note: L1 cache is used differently in accelerator nodes, such as the PowerScale P100, which do not contain any disk drives. Instead, the entire read cache is L1 cache because all the data is fetched from other storage nodes. Also, cache aging is based on a Least Recently Used (LRU) eviction policy, as opposed to the drop-behind algorithm typically used in a storage node’s L1 cache. An accelerator’s L1 cache is large, and the data in it is highly likely to be requested again. Data blocks are not immediately removed from cache upon use. However, metadata and update-heavy workloads do not benefit as much, and an accelerator’s cache is only beneficial to clients directly connected to the node.
To support OneFS inline compression, a node’s L1, or client-side, read cache is divided into separate address spaces. The result is that both the on-disk compressed data and the logical uncompressed data can be cached. The address space for the L1 cache is already split for data and FEC blocks, so a similar technique is used to divide it again. Data in the uncompressed L1 cache is fed from data in the compressed L1 cache which, in turn, is fed from disk.
OneFS prefetch caching has also been enhanced to accommodate compressed data. Because reading part of a compressed chunk results in the entire compression chunk being cached, prefetch requests, in effect, are rounded to compression chunk boundaries. Because a prefetch request is not complete until the uncompressed data is available in cache, the callback used for prefetch requests performs the decompression.