Home > Storage > PowerScale (Isilon) > Industry Solutions and Verticals > Media and Entertainment > Dell Technologies PowerScale: OneFS Best Practices for DaVinci Resolve > PowerScale storage configuration
The latest PowerScale storage nodes feature powerful inline deduplication (dedupe) and compression that are enabled by default. These features automatically compress and dedupe data as it is being written to the cluster for storage efficiency.
Video data tends to already be compressed and does not tend to dedupe well. As such, disabling these features frees up more PowerScale cluster resources for reading and writing video.
It is strongly recommended to disable OneFS inline dedupe and compression on clusters being used for Resolve workflows. Disabling these features are global on the PowerScale cluster.
To view inline dedupe and compression settings, run the following commands from the OneFS command line:
isi dedupe inline settings view
isi compression settings view
To disable inline dedupe and compression, run the following commands from the OneFS command line:
isi dedupe inline settings modify --mode disabled
isi compression settings modify --enabled false
DaVinci Resolve supports Windows, macOS, and Linux and as such both SMB and NFS connection protocols need to be considered:
Configuration and optimization of the client SMB and NFS stack is outside of the scope of this document and is covered elsewhere. Care should be taken to review the connection, configuration, and optimization guidelines that are appropriate for the platform being used.
Regardless of connection protocol, for high throughput workflows, and especially for network connections of 10 GbE and greater, use of Jumbo frames (9000mtu) can have a significant positive impact. However, care must be taken that the entire connection from Isilon storage to network switch to workstation are properly configured for the benefit of jumbo frames to be recognized.
Architects should consider four key optimizations for OneFS when deploying in a DaVinci Resolve environment.
File access performance can be optimized by enabling Metadata read/write acceleration. When enabled, this optimization sees OneFS use SSDs for reading and writing file system metadata. On PowerScale nodes with hard drive storage, metadata read/write can improve access time and reduce latency. When all underlying storage units are SSD or NVMe such as PowerScale F-series nodes, this optimization is not meaningful or required.
The default configuration of hybrid PowerScale nodes is to use the SSDs for L3 cache. Converting from L3 mode to metadata read/write acceleration includes changes to the default file pool policies in OneFS. Not all hybrid clusters have enough SSD space to accommodate metadata read/write. Also, converting to metadata read/write can negatively impact workflows that may be relying on L3 cache acceleration (metadata read/write and L3 modes are mutually exclusive).
At a high level the methodology for changing from L3 to metadata read/write entails:
It is recommended to engage with Dell support to make these changes.
OneFS has multiple methodologies for reading and writing data to disk. These strategies can be configured at the file or directory level from the OneFS command line or file pool policy. The good news is that the default file access and layout strategies in OneFS are best for most workflows involving compressed video.
However, when working with uncompressed and/or images sequence-based material, meaningful gains in performance can be had with streaming mode and file name based prefetch.
Streaming performance of OneFS can be optimized by enabling Streaming Mode at the file pool, or at the directory level. Streaming mode can be helpful when playing high throughput media linearly with minimal timeline scrubbing.
Consideration should be given to other workflows hosted in the same pool or directory before enabling Streaming mode. Best performance for all workflows on a single cluster might be achieved by selectively enabling Streaming Mode on a limited number of directories.
Streaming Mode optimizes two behaviors of OneFS to deliver increased streaming performance:
Concurrency is the default file layout and access strategy. This default mode is best for most DaVinci Resolve compressed media workflows. Due to the streaming mode’s aggressive prefetching, there are circumstances when DaVinici Resolve may drop frames due to excessive prefetching for a particular client or when scrubbing a timeline. As such, the default concurrency mode is recommended in most cases.
Streaming mode is optimized for a few clients playing back high throughput material such as ProRes 4444 4K @ 60 fps. Concurrency mode uses an adaptive prefetch algorithm that is more measured in its approach. For clusters with large numbers of users working with relatively low-bandwidth material (such as ProRes 422 1080p @ 30 fps), sticking with concurrency mode is the most appropriate choice.
A strength of OneFS is that both streaming and concurrency modes can co-exist on the same cluster on a directory by directory basis. A common approach is to use concurrency mode generally while selectively setting media directories with high throughput material to streaming mode.
Streaming or Concurrency mode can be enabled as a File Pool Policy at the OneFS Web User Interface or from the OneFS CLI.
From the Web Interface, the file access strategy is configured in:
File System à Storage Pools à File Pool Policies
Edit the default policy and change file access methodology. A SmartPools job then needs to run to change the OneFS flag on each file.
Access and layout strategies can be manually set from the OneFS command line using the isi set command:
isi set -R -l [concurrency | streaming | random] /path/to/directory/
isi set -R -a [default | streaming | random | disabled | custom {1..5}] /path/to/directory
It should be noted that when Filename Pre-Fetch (outlined below) is also required and enabled, there is an interaction between the two optimizations.
Filename Pre-Fetch can be enabled to optimize the streaming performance of OneFS when working with image sequences. It is critical that this optimization only be applied to directories containing image sequences of successively named files. Applying it to directories that do not contain image sequences may result in false fetches (where resources are used prefetching unnecessary files).
If DaVinci Resolve is being used to work with image sequences, then improved performance may be achieved by enabling Filename-based Pre-Fetch. Where image sequences are not used in the environment, Filename based Pre-Fetch should not be enabled.
Filename Pre-Fetch enables a performance optimization by detecting when image sequences are requested. When a sequence is detected, OneFS will prefetch files from the underlying storage units (disc drives or SSDs) before they are requested.
The Filename Pre-Fetch algorithm is optimized to detect image sequences with either decimal or hexadecimal filename numerical increments.
Figure 8 Filename based prefetch
Filename Pre-Fetch can be enabled on individual directories at the command line of OneFS. A detailed overview and configuration guide is available at OneFS Filename based prefetch.
The Filename Based Pre-Fetch guide provides good explanations of setting and examining OneFS file layout and access strategies from the command-line interface. The guide is recommended reading even if the Filename Pre-Fetch is not to being used.