PowerEdge MX 1.20.10 adds support for the Broadcom 57504 quad-port Ethernet adapter. For chassis with MX7116n FEMs, the first QSFP28-DD port is used when attaching dual-port NICs. The first and second QSFP28-DD ports of the MX7116n FEM are used when attaching quad-port NICs. When both QSFP28-DD ports are connected, a server with a dual-port NIC will only use the first port on each FEM. With quad-port NICs, both ports are used.
The MX9116n FSE has sixteen 25 GbE server-facing ports, ethernet1/1/1 through ethernet1/1/16, which are used when the PowerEdge MX server sleds are in the same chassis as the MX9116n FSE.
With only dual-port NICs in all server sleds, only the odd-numbered server-facing ports are active. If the server has a quad-port NIC, but the MX7116n FEM has only one port connected to the MX9116n FSE, only half of the NIC ports will be connected and show a link up.
The following table shows the MX server sled to MX9116n FSE interface mapping for dual-port NIC servers which are directly connected to the switch.
Sled number | MX9116n FSE server interface |
Sled 1 | ethernet 1/1/1 |
Sled 2 | ethernet 1/1/3 |
Sled 3 | ethernet 1/1/5 |
Sled 4 | ethernet 1/1/7 |
Sled 5 | ethernet 1/1/9 |
Sled 6 | ethernet 1/1/11 |
Sled 7 | ethernet 1/1/13 |
Sled 8 | ethernet 1/1/15 |
With quad-port NICs in all server sleds, both the odd- and even-numbered server-facing ports will be active. The following table shows the MX server sled to MX9116n FSE interface mapping for quad-port NIC servers which are directly connected to the switch.
Sled number | MX9116n FSE server interface |
Sled 1 | ethernet 1/1/1, ethernet 1/1/2 |
Sled 2 | ethernet 1/1/3, ethernet 1/1/4 |
Sled 3 | ethernet 1/1/5, ethernet 1/1/6 |
Sled 4 | ethernet 1/1/7, ethernet 1/1/8 |
Sled 5 | ethernet 1/1/9, ethernet 1/1/10 |
Sled 6 | ethernet 1/1/11, ethernet 1/1/12 |
Sled 7 | ethernet 1/1/13, ethernet 1/1/14 |
Sled 8 | ethernet 1/1/15, ethernet 1/1/16 |
When using multiple chassis and MX7116n FEMs, virtual slots are used to maintain a continuous mapping between the NIC and physical port. For more information on virtual slots, see Virtual ports and slots.
In a multiple chassis Scalable Fabric, the interface numbers for the first two are mixed, as one NIC connection is to the MX9116n in the same chassis as the server, and the other NIC connection is to the MX7116n. In this example, the following table shows the server interface mapping for Chassis 1 using quad-port adapters.
Chassis 1 sled number | Chassis 1 MX9116n server interface | Chassis 2 MX9116n server interface |
Sled 1 | ethernet 1/1/1, ethernet 1/1/2 | ethernet 1/71/1, ethernet 1/71/9 |
Sled 2 | ethernet 1/1/3, ethernet 1/1/4 | ethernet 1/71/2, ethernet 1/71/10 |
Sled 3 | ethernet 1/1/5, ethernet 1/1/6 | ethernet 1/71/3, ethernet 1/71/11 |
Sled 4 | ethernet 1/1/7, ethernet 1/1/8 | ethernet 1/71/4, ethernet 1/71/12 |
Sled 5 | ethernet 1/1/9, ethernet 1/1/10 | ethernet 1/71/5, ethernet 1/71/13 |
Sled 6 | ethernet 1/1/11, ethernet 1/1/12 | ethernet 1/71/6, ethernet 1/71/14 |
Sled 7 | ethernet 1/1/13, ethernet 1/1/14 | ethernet 1/71/7, ethernet 1/71/15 |
Sled 8 | ethernet 1/1/15, ethernet 1/1/16 | ethernet 1/71/8, ethernet 1/71/16 |
Quad-port NIC restrictions and guidelines
- If the server has a quad-port NIC, but the MX7116n FEM has only one port connected to the MX9116n FSE, only half of the NIC ports will be connected and show a link up.
- Both ports on the MX7116n FEM must be connected to the same MX9116n FSE. Note: Do not connect one MX7116n FEM port to one MX9116n FSE and the other MX7116n FEM port to another MX9116n FSE. This is not supported. The Unsupported configuration for quad-port NICs figure shows the unsupported configuration.
- If a Scalable Fabric has some chassis with quad-port NICs and some with only dual-port NICs, only the chassis with quad-port NICs require the second MX7116n FEM port to be connected, as shown in the Multiple chassis topology with quad-port and dual-port NICs – single fabric figure.
- It is supported to have a dual-port NIC in Fabric A and a quad-port NIC in Fabric B (or the inverse), or have a quad-port NIC in both Fabric A and Fabric B.
- Up to five chassis with quad-port NICs are supported in a single Scalable Fabric.
The following figure shows a single-chassis topology with quad-port NICs. Make sure to connect both ports on the MX7116n FEM to the same MX9116n FSE.
The following figure shows the two-chassis topology with quad-port NICs in each chassis. Only a single fabric is configured. Make sure to connect both ports on the MX7116n FEM to the same MX9116n FSE.
The following figure shows the two-chassis topology with quad-port NICs. Dual fabrics are configured.
The following figure shows the multiple chassis topology with quad-port NICs. Only a single fabric is configured.
The following figure shows the multiple chassis topology with quad-port NICs in two chassis and dual-port NICs in one chassis. Only a single fabric is configured. Make sure to connect both ports on the MX7116n FEM to the same MX9116n FSE with the quad-port card. Do not connect the second port on the MX7116n FEM when configured with a dual-port NIC.
The following figure shows one example of an unsupported topology. The ports on the MX7116n FEMs must never be connected to different MX9116n FSEs.