Home > Integrated Products > VxBlock 1000 and 3-Tier Platform Reference Architectures > Guides > 3-Tier Platform Design Guide > Data plane network layer
The data plane network layer carries all the production network traffic that flows through the 3-Tier Platform.
The Cisco Nexus 9336C-FX2 switch provides 36 x 40 Gbps or 36 x 100 Gbps QSFP28 ports. Each QSFP28 port can be divided into four 10 Gb connections or four 25 Gb connections to facilitate connections to SFP+ or SFP28 based interfaces. There are no restrictions on dividing ports on a Cisco Nexus 9336C-FX2 data plane switch. This switch can be used for the ToR network and for additional switch pairs to support PowerScale connectivity.
Two 100 Gbps direct-attach connections form the vPC peer link with the Cisco Nexus 9336C-FX2 switches.
Uplinks from the Cisco UCS 6454, Cisco UCS 64108, and Cisco UCS 6536 FIs to the Cisco Nexus 9336C-FX2 switch use:
The Cisco UCS 6454 FIs can consume four, eight, or 12 total ports on a pair of Cisco Nexus 9336C-FX2 switches for a standard or enhanced bandwidth uplink topology:
The Cisco UCS 64108 FIs can consume eight, 12, or 16 ports on a pair of Cisco Nexus 9336C-FX2 switches for a standard or enhanced bandwidth uplink topology:
The Cisco UCS 6536 FIs can consume four, eight, 12, or 16 total ports on a pair of Cisco Nexus 9336C-FX2 switches for a standard or enhanced bandwidth uplink topology:
The following table shows the supported connection types to the Cisco Nexus 9336C-FX2 ToR switches for Cisco UCS 4th-generation and 5th-generation FI uplinks:
Table 34. Supported connection types for Cisco UCS 4th and 5th-generation FI uplinks
Bandwidth | Transceiver | Media | Minimum uplinks per switch pair | Maximum uplinks per switch pair | Maximum distance |
100 Gbps | QSFP-100G-SR4-S | OM4 MPO MMF | 4 | 16 | 100 M |
100 Gbps | QSFP-40G/100G-SRBD | OM4 LC MMF | 4 | 16 | 100 M |
100 Gbps | QSFP-100G-AOC | AOC Direct Attach Cable | 4 | 16 | 30 M |
A Cisco QSFP-40G-SR4 or QSFP-100G-SR-S transceiver connects the 10 Gbps, 25 Gbps, or 40 Gbps NAS ports on a storage array to the Cisco Nexus 9336C-FX2 switch. A combination of MPO-to-LC hydra cables, MPO trunks, and MPO-to-LC cassettes provide fiber connectivity to the endpoint. NAS devices are cross-connected to both switches.
The following table shows the transceiver connections for each ToR switch:
Table 35. Transceiver connections per ToR switch
ToR switch | Transceiver | Breakout type | Array ports |
Cisco Nexus 9336C-FX2 | QSFP-100G-SR4-S | A combination of MPO-to-LC hydra cables, MPO trunks, and MPO-to-LC cassettes | 25 GbE |
Cisco Nexus 9336C-FX2 | QSFP-40G-SR4 | A combination of MPO-to-LC hydra cables, MPO trunks, and MPO-to-LC cassettes | 10 GbE |
Cisco Nexus 9336C-FX2 | QSFP-40G-SR4 | A combination of MPO-to-LC hydra cables, MPO trunks, and MPO-to-LC cassettes | 40 GbE |
A combination of MPO-to-LC hydra cables, MPO trunks, and MPO-to-LC cassettes provide four 25 Gbps LC fiber connectivity from each QSFP module to the endpoint. These connections are used for SDNAS and intracluster communication.
The following table shows the transceiver connections for each PowerStore T model appliance:
Table 36. Transceiver connections per PowerStore T model appliance
ToR switch | Transceiver | Breakout type | Mezz card |
Cisco Nexus 9336C- FX2 | QSFP-100G-SR4-S | A combination of MPO-to-LC hydra cables, MPO trunks, and MPO-to-LC cassettes | 25 GbE |
QSFP-40G-SR4 | A combination of MPO-to-LC hydra cables, MPO trunks, and MPO-to-LC cassettes | 10 GbE |
For each node, mezzanine ports 0 and 1 are an LACP vPC pair across the ToR switches. Mezzanine ports connect to the ToR switches as follows:
Starting from PowerStoreOS 3.0, you can update the LACP pair from two ports to four ports. Additional mezzanine ports are cabled as:
The following table shows the transceiver connections for each ToR switch:
Table 37. Transceiver connections per Cisco Nexus 9336C-FX2 ToR switch
Transceiver | Breakout type | PowerScale front end |
QSFP-100G-SR4-S | A combination of MPO-to-LC hydra cables, MPO trunks, and MPO-to-LC cassettes | 25 GbE |
QSFP-40G-SR4 | A combination of MPO-to-LC hydra cables, MPO trunks, and MPO-to-LC cassettes | 10 GbE |
QSFP28 | A combination of MPO-to-LC hydra cables, MPO trunks, and MPO-to-LC cassettes | 25 GbE |
QSFP-40G-SR4 | A combination of MPO-to-LC hydra cables, MPO trunks, and MPO-to-LC cassettes | 40 GbE |
QSFP28 | A combination of MPO Type B crossover cable, multimode Fiber OM4 MPO cable | 100 GbE |
The following table shows connections to the external network for Cisco Nexus uplinks:
Table 38. External network connections for Cisco Nexus uplinks
Bandwidth | Transceiver | Media | Minimum uplinks per switch pair | Maximum uplinks per switch pair | Maximum distance |
100 Gbps | QSFP-100G-SR4-S | OM4 MPO MMF | 4 | 16 | 100 M |
100 Gbps | QSFP-40G-SRBD or QSFP-100G-SRBD | OM4 LC MMF | 4 | 16 | 100 M |
40 Gbps | QSFP-40G-SRBD | OM4 LC MMF | 4 | 16 | 150 M |
40 Gbps | QSFP-40G-SR4 | OM4 MPO MMF | 4 | 16 | 100 M |
10 Gbps | QSFP-10G-SR4 | OM4 MPO MMF terminated with LC | 4 QSFP (16 x 10 Gbps) | 8 QSFP (32 x 10 Gbps) | 150 M |
The Cisco Nexus 9336C-FX2 switch has a tiered licensing model with two options: