Uplink failure detection (UFD) detects the loss of upstream connectivity from switch uplinks to the next-hop switch. If the switch loses upstream connectivity, the related downstream server-facing interfaces are shut down so the host can use a different path to send data out of the fabric. By default, the attached hosts continue to send traffic to that switch without a direct path to the destination. The downstream devices do not generally receive an indication that the upstream connectivity was lost because connectivity to the local switch is still operational. To solve this issue, use UFD. The VLTi link to the peer switch can temporarily handle traffic during a network outage, but this is not considered a best practice.
An uplink state group is configured on each switch, which creates an association between the uplinks to the upstream devices and the downlink interfaces. In the event that all uplinks fail on a switch, UFD automatically shuts down the downstream interfaces. This propagates to the hosts attached to the switch. Each host then uses its link to the remaining switch to continue sending traffic across the network. An interface in an uplink-state group can be a physical interface or a port channel (LAG) aggregation of physical interfaces.
In SmartFabric mode, UFD is automatically enabled with OME-M 1.10.20. UFD is user-configurable with OME-M 1.20.00 and later. In Full Switch mode, UFD is NOT enabled by default and must be configured at the switch CLI. Enabling UFD is recommended.
For example, in the MX scenario that is mentioned in Scenario 5: Connect MX9116n FSE to Fibre Channel storage - NPIV Proxy Gateway mode, when an uplink is set as FC gateway, UFD associates the set of downstream interfaces which are part of the corresponding FCoE VLAN into a UFD group. In this scenario, the VLANs are VLAN 30 and VLAN 40 on each switch respectively. The downstream interfaces are the ones connected to the MX740c compute sleds.
In SmartFabric mode with an FC uplink failure situation, where all FC uplink ports are down (for example, removing the fibre channel transceiver from the switch), the switch operationally disables the downstream interfaces which belong to that UFD group AND have the FCoE VLAN provisioned to them. A server that does not have an impacted FCoE VLAN is not disturbed. Once the downstream ports are set operationally down, the traffic on these server ports is stopped, giving the operating system the ability to fail traffic over to the other path. In a scenario with MX9116n FSEs, a maximum of eight FC ports can be part of an FC Gateway uplink.
This is resolved by shutting down only the corresponding compute sled downstream ports which provide an alternate path to the compute sleds. Bring up at least one FC port that is part of the FC gateway uplink so that the FCoE traffic can transition through another FC port on the NIC or an IOM in the fabric. Remove FCoE VLANs from Ethernet-only downstream ports to avoid an impact on Ethernet traffic.
To include uplinks into a UFD group in SmartFabric mode, perform the following steps.
Steps
- Access the OME-M console.
- Select Devices > Fabric. Choose created fabric.
- The UFD group can be included in two ways. If uplinks are not created, select Add Uplink. Enter Name, Description, and Uplink type.
- Mark the check box Include in Uplink Failure Detection Group.
- If uplinks are created, choose an uplink, select Edit.
- Under Edit Uplink, mark the check box Include in Uplink Failure Detection Group.
- This enables UFD and includes the uplink into the UFD group.
Verifying UFD configuration
To verify UFD on a switch, run the following CLI commands.
MX9116n-1# show uplink-state-group 1
Uplink State Group: 1, Status: Enabled, Up
MX9116n-1# show uplink-state-group detail
(Up): Interface up (Dwn): Interface down
(Dis): Interface disabled (NA): Not Available
*: VLT Port-channel V: VLT Status P: Peer Operational Status ^: Tracking Status
Uplink State Group : 1 Status : Enabled,up
Upstream Interfaces : Fc 1/1/44:1(Up), Fc 1/1/44:2(Up)
Downstream Interfaces: Eth 1/1/1(Up), Eth 1/1/3(Up), Eth 1/71/2(Up), Eth 1/71/7(Up)