Toggle auto negotiation
Enabling or disabling auto negotiation from the OME-M console can bring up the uplinks connecting to the upstream switches. For example, when deploying the SmartFabric with the Cisco Nexus 3232C (see Scenario 2: SmartFabric connected to Cisco Nexus 3232C switches with Ethernet - No Spanning Tree uplink), disable auto negotiation on uplink ports on the MX switches to bring the link up.
The OME-M console is used to disable/enable auto negotiation ports on MX switches. The following steps illustrate turning disabling auto negotiation on ports 41 and 42 of a MX9116n.
- From switch management page, choose Hardware > Port Information.
- Select the ports on which auto negotiation must be disabled. In this example, ports 1/1/41 and port 1/1/42 are selected.
- Click Toggle AutoNeg > Finish.
Set uplink ports to administratively up
The uplink ports on the switch might be administratively down. Enabling the uplink ports can be carried out from the OME-M console. The uplink ports can be administratively down when a port group breakout happens, especially for FC breakouts.
The OME-M console can be used to disable/enable the ports on MX switches. The following steps illustrate turning setting the administrative state on ports 41 and 42 of an MX9116n.
- From switch management page, choose Hardware > Port Information.
- Select the ports. Note: In this example, ports 1/1/41 and port 1/1/42 are selected.
- Click Toggle Admin State > Finish .
Verify MTU size
Set the same MTU size on the ports that connect the MX switches, the ports on the upstream switches, and the server NICs. To set the MTU size from the OME-M console, see Configure Ethernet ports.
Verify auto negotiation settings on upstream switches
Verify the auto negotiation settings on the upstream switches. In the case where auto negotiation settings are modified, the links might not come up. Change the auto negotiation on upstream switches to resolve the issue.
For example, if the auto negotiation was disabled on the Cisco Nexus upstream switches, the setting can be turned on. To enable the autonegotiation on an Ethernet interface on Cisco Nexus switches, run the following commands:
switch# configure terminal
switch(config)# interface ethernet <interface-number>
switch(config-if)# negotiate auto
The following example shows interface ethernet 1/2
with auto negotiation enabled on the interface: Nexus-3232C-Leaf1(config-if)# do show int eth 1/2
Ethernet1/2 is down (XCVR not inserted)
admin state is down, Dedicated Interface
Hardware: 40000/100000 Ethernet, address: 00fe.c8ca.f367 (bia 00fe.c8ca.f36c)
MTU 1500 bytes, BW 100000000 Kbit, DLY 10 usec
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, medium is broadcast
auto-duplex, auto-speed
Beacon is turned off
Auto-Negotiation is turned on, FEC mode is Auto
---- OUTPUT TRUNCATED -----
Verify LACP
The interface status of the upstream switches can provide valuable information for the link being down. The following example shows interfaces 1 and 3 on upstream Cisco Nexus switches as members of port channel 1:
3232C-Leaf2# show interface status
--------------------------------------------------------------------------------
Port Name Status Vlan Duplex Speed Type
--------------------------------------------------------------------------------
mgmt0 -- connected routed full 1000 --
Eth1/1 To MX Chassis 1 suspended trunk full 100G QSFP-100G-SR4
Eth1/2 -- xcvrAbsen routed auto auto --
Eth1/3 To MX Chassis 2 suspended trunk full 100G QSFP-100G-SR4
---- OUTPUT TRUNCATED -----
Checking interface 1 reveals that the ports are not receiving the LACP PDUs as shown in the following example:
3232C-Leaf2# show int eth 1/1
Ethernet1/1 is down (suspended(no LACP PDUs))
admin state is up, Dedicated Interface
Belongs to Po1
---- OUTPUT TRUNCATED -----
show interface status
command to view the interfaces and associated status information. Use the show interface ethernet interface number
to view the interface details. In the following example, the errors listed above occurred because an uplink was not created on the fabric.
The following image shows the Topology with QSFP28 100 GbE connection on ports 37 and 39 instead of QSFP28-DD connection, an unsupported configuration.
The resolution is to add one or more uplinks and verify that the fabric turns healthy.