Below are some pre-requisites and important notes for enabling Fabric Notifications with MDS switches and directors:
- Dell supports NX-OS 9.3(2a) or above with Fabric Notifications. Any switch or director capable of running these versions is supported. No additional licenses are required.
- FCIP is not supported.
- E-Ports are not currently supported.
- Notifications for oversubscription-based congestion are not supported in NX-OS 9.3. Cisco has committed to supporting this feature in a future NX-OS release. Please refer to NX-OS release notes for updates.
Please refer to the Fabric Perf Impact Notification (FPIN) Simple Support Matrix within the PowerMax/VMAX All Flash/VMAX3 Features Simple Support Matrix for more information and for Dell supported configurations. Please refer to the NX-OS release notes and the Guidelines and Limitations for Fabric Notifications section in the Cisco MDS 9000 Series Interfaces Configuration Guide, Release 9.x for additional support information and limitations.
Fabric Notifications on MDS are not enabled by default and must be enabled through a procedure. Cisco provides a policy called “fabricmon_edge_policy” that will enable Fabric Notifications and sets the following default thresholds:
- counter link-loss poll-interval 30 delta rising-threshold 5 event 4 falling-threshold 1 event 4 alerts syslog rmon portguard FPIN
- counter sync-loss poll-interval 30 delta rising-threshold 5 event 4 falling-threshold 1 event 4 alerts syslog rmon portguard FPIN
- counter signal-loss poll-interval 30 delta rising-threshold 5 event 4 falling-threshold 1 event 4 alerts syslog rmon portguard FPIN
- counter invalid-words poll-interval 30 delta rising-threshold 1 event 4 falling-threshold 0 event 4 alerts syslog rmon portguard FPIN
- counter invalid-crc poll-interval 30 delta rising-threshold 5 event 4 falling-threshold 1 event 4 alerts syslog rmon portguard FPIN
- counter txwait poll-interval 1 delta rising-threshold 30 event 4 falling-threshold 10 event 4 alerts syslog rmon portguard FPIN
This policy can be activated or modified to the customer’s requirements. Please note that there is no single set of thresholds that will address all customer situations and requirements. The thresholds and procedure below should be considered a starting point that can be adjusted and tuned as desired:
- Minimum NX-OS of 9.3(2a) for system is required along with kickstart code. Use the #show version to display the version of NX-OS running on the switch or director.
- Enable the FPM-feature using the following commands:
- switch #configure
- switch(config) #feature fpm
- Plug into FC-ports/interfaces with FC FPIN-compliant ports and assign a VSAN# membership to these ports.
- To verify if FC FPIN-complaint ports are logged in and registered, run the following command, and verify the output:
#show fpm fpin <vsan#>
This command can be used to view all FC interfaces that are logged in and able to receive Fabric Notifications.
- If a previous policy is active, it is recommended to backup that policy. De-activate the previous policy using the following command:
#no port-monitor activate <policy-name>
- Activate the “fabricmon_edge_policy” policy using the following command:
#port-monitor activate fabricmon_edge_policy
Fabric Notifications are now enabled and the switch is running Cisco’s default thresholds.
- If the customer wants to change the thresholds in the “fabricmon_edge_policy”, use the following commands:
- Deactivate the active policy and enter port monitoring policy configuration mode by entering the following commands:
#no port-monitor activate fabricmon_edge_policy
#config terminal
switch(config)# port-monitor name fabricmon_edge_policy
- Thresholds can be adjusted using the portguard command. Below are examples used at Dell to lower the txwait and invalid-crc thresholds:
#counter txwait poll-interval 1 delta rising-threshold 2 event 2 falling-threshold 0 portguard FPIN
#counter invalid-crc poll-interval 60 delta rising-threshold 3 event 2 falling-threshold 1 portguard FPIN
# exit
- To activate the new port-monitor policy and review settings/parameters, use the following commands:
#show port-monitor active
#port-monitor activate fabricmon_edge_policy
#show port-monitor active
#show running-config | inc FPIN
Additional documentation on Fabric Notifications with MDS can be found in the following documents: