Home > Storage > PowerMax and VMAX > Data Protection > Dell PowerMax and VMAX All Flash: SRDF/Metro Overview and Best Practices > Witness behavior during failures and recovery
This section describes the behavior provided by a witness regarding witness selection, redundancy, and availability decisions.
Activity between a pair of SRDF/Metro groups is known as a SRDF/Metro session. When a session starts, the R1 and R2 arrays negotiate which of the available witness instances to use to protect the session. Thus, an individual array could be using several witness instances simultaneously. In the same way, an individual witness instance may be monitoring several SRDF/Metro sessions simultaneously as described previously.
The SRDF/Metro paired array polls all of the witness instances in its definition list every second. Each witness then sends a reply. This enables the paired array to maintain a list of instances that are available and operational. If an array detects that an instance has not responded for 10 seconds, it checks whether the instance is in use by any SRDF/Metro session. If it is in use, the R1 and R2 arrays negotiate an alternative witness to use in its place. If there are no witnesses available, the session uses bias functionality as a fallback.
If either array detects that an SRDF/Metro session has failed (that is, the array has lost contact with the partner group either due to a failure of the SRDF link or in the partner array), the array will request a lock from the witness instance allocated to the SRDF/ Metro session. On the R1 side, the array sends this lock request to the witness instance for that session immediately. Typically, the R2 array waits 5 seconds before sending a similar lock request to the witness. This allows time for the R1 side to request the lock. In this manner, the R1 array has priority and acquires the lock during this 5 second period. The witness instance grants the lock in response to the first request it receives. The side that gains the lock remains available to the host while the other side becomes unavailable.
In addition to determining which witness instance to use, the arrays in each SRDF/Metro session also negotiate which of them is the preferred winner. In the event of a failure, the preferred winner is the side that has priority when requesting the lock from the witness instance; that is, the preferred winner is the R1 side. When either side runs HYPERMAX OS 5977, SRDF/Metro uses the bias settings for the devices to determine the preferred winner. That is, the devices defined as the being on the bias side, if Device Bias were to be used, become the preferred winners.
As described in this and related witness failure sections, there are a number of possible single, dual, and triple failures scenarios and outcomes covered by a witness in addition to other factors taken into account regarding the ability of a particular array to better service host I/Os. The recovery from a specific scenario may range from performing a simple establish operation, half swap operation, or may require other more detailed recovery steps.
Note: To determine the actions necessary to properly recover SRDF/Metro from a specific failure scenario, see the SRDF/Metro Recovery Knowledge Base (KB) article KB516522 (https://www.dell.com/support/kbdoc/000021764), engage Dell Technologies support directly, or escalate to your local account or support team as the urgency of the situation dictates.