Three-layer configuration
Three-layer configuration
-
The three-layer configuration is based on a standard build for PowerFlex rack using PowerFlex Manager as explained in Figure 3. Any modifications required are detailed in the sections to follow. The MX7000 was deployed based on Dell EMC Best Practices. Both the PowerFlex rack and the MX7000 components were deployed following the LACP bonded NIC port design standard.
Due to different Layer 3 subnets being used between the PowerFlex and MX7000 storage networks, gateways were defined on the customer switches. The following table lists the VLAN and SVI (if different subnets were used).
Table 13. VLAN and SVI details
Purpose
MX7000 VLAN, SVI
PowerFlex VLAN, SVI
Flex node management
848
848
Flex storage management
858
858
Flex data1
711, 192.168.152.1
720, 192.168.153.1
Flex data2
712, 192.168.160.1
721, 192.168.161.1
Flex data3
713, 192.168.168.1
722, 192.168.169.1
Flex data4
714, 192.168.176.1
723, 192.168.177.1
Customer switches
The customer switches provided connectivity between the MX7000 chassis and the PowerFlex rack. The storage traffic traversed these pair of switches, negotiating the gateways defined for each Layer 3 subnet.
The sample code below is from the Dell EMC laboratory Cisco 9332 switches, which operated as the customer switches.
Note: Some VLAN names may vary.
vlan 106
name vmotion
vlan 711
name SO-data1
vlan 712
name SO-data2
vlan 713
name SO-data3
vlan 714
name SO-data4
vlan 720
name CO-data1
vlan 721
name CO-data2
vlan 722
name CO-data3
vlan 723
name CO-data4
vlan 848
name ESXi_mgmt
vlan 858
name SIO_mgmt
vlan 711
name 2LN-SO-D1:172.16.11.0/24
vlan 712
name 2LN-SO-D2:172.16.12.0/24
vlan 713
name 2LN-SO-D3:172.16.13.0/24
vlan 714
name 2LN-SO-D4:172.16.14.0/24
interface Vlan711
description SO-Data1:172.16.11.0/24
no shutdown
no ip redirects
ip address 172.16.11.2/24
no ipv6 redirects
hsrp version 2
hsrp 711
authentication text Vce12345
preempt
priority 110
ip 172.16.11.1
interface Vlan712
description SO-Data2:172.16.12.0/24
no shutdown
no ip redirects
ip address 172.16.12.2/24
no ipv6 redirects
hsrp version 2
hsrp 712
authentication text Vce12345
preempt
priority 110
ip 172.16.12.1
interface Vlan713
description SO-Data3:172.16.13.0/24
no shutdown
no ip redirects
ip address 172.16.13.2/24
no ipv6 redirects
hsrp version 2
hsrp 713
authentication text Vce12345preempt
priority 110
ip 172.16.13.1
interface Vlan714
description SO-Data4:172.16.14.0/24
no shutdown
no ip redirects
ip address 172.16.14.2/24
no ipv6 redirects
hsrp version 2
hsrp 714
authentication text Vce12345preempt
priority 110
ip 172.16.14.1
interface Vlan720
description CO-Data1:192.168.153.0/24
no shutdown
no ip redirects
ip address 192.168.153.2/24
no ipv6 redirects
hsrp version 2
hsrp 720
authentication text Vce12345
preempt
priority 110
ip 192.168.153.1
interface Vlan721
description CO-Data2:192.168.161.0/24
no shutdown
no ip redirects
ip address 192.168.161.2/24
no ipv6 redirects
hsrp version 2
hsrp 721
authentication text Vce12345
preempt
priority 110
ip 192.168.161.1
interface Vlan722
description CO-Data3:192.168.169.0/24
no shutdown
no ip redirects
ip address 192.168.169.2/24
no ipv6 redirects
hsrp version 2hsrp 722
authentication text Vce12345
preempt
priority 110
ip 192.168.169.1
interface Vlan723
description CO-Data4:192.168.177.0/24
no shutdown
no ip redirects
ip address 192.168.177.2/24
no ipv6 redirects
hsrp version 2
hsrp 723
authentication text Vce12345
preempt
priority 110
ip 192.168.177.1
interface port-channel20
description "To PowerFlex Rack"
switchport
switchport mode trunk
switchport trunk allowed vlan 106,711,712,713,714,848,858
mtu 9216
no lacp suspend-individual
vpc 20
interface port-channel21
description MX9116n A
switchport
switchport mode trunk
switchport trunk allowed vlan 106,848,858
mtu 9216
no lacp suspend-individual
vpc 21
interface port-channel22
description MX9116 B
switchport
switchport mode trunk
switchport trunk allowed vlan 106,720,721,722,723,848,858
mtu 9216
no lacp suspend-individual
vpc 22
interface port-channel51
description 9332 Peer-link
switchport
switchport mode trunk
spanning-tree port type network
vpc peer-link
interface Ethernet1/19
description MX9116 B
switchport
switchport mode trunk
switchport trunk allowed vlan 106,720,721,722,723,848,858
mtu 9216
channel-group 22 mode active
no shutdown
interface Ethernet1/20
description MX9116 A
switchport
switchport mode trunk
switchport trunk allowed vlan 104,106,848,858
mtu 9216
channel-group 21 mode active
no shutdown
interface Ethernet1/21
description MX9116 B
switchport
switchport mode trunk
switchport trunk allowed vlan 104,106,720,721,722,723,848,858
mtu 9216
channel-group 22 mode active
no shutdown
interface Ethernet1/22
description MX9116 A
switchport
switchport mode trunk
switchport trunk allowed vlan 104,106,848,858
mtu 9216
channel-group 21 mode active
no shutdown
interface Ethernet1/25
description peer-link
switchport
switchport mode trunk
channel-group 51 mode active
no shutdown
interface Ethernet1/26
description peer-link
switchport
switchport mode trunk
channel-group 51 mode active
no shutdown
interface Ethernet1/31
description to PowerFlex Rack
switchport
switchport mode trunk
switchport trunk allowed vlan 104,106,711,712,713,714,848,858
mtu 9216
channel-group 20 mode active
no shutdown
interface Ethernet1/32
description to PowerFlex Rack
switchport
switchport mode trunk
switchport trunk allowed vlan 104,106,711,712,713,714,848,858
mtu 9216
channel-group 20 mode active
no shutdown
Access switches
The Access switches in the PowerFlex rack used in Dell EMC laboratory were Cisco 93180 switches. They were configured as standard except for extending the Data1-Data4 VLANs north.
Table 14. Required modifications for this POC
Switch 1
Switch 2
interface Ethernet1/49
description wolfpack-n9332-l2agg-a
switchport
switchport mode trunk
switchport trunk native vlan 10
switchport trunk allowed vlan 711-714,815,825,999
spanning-tree port type network
spanning-tree guard loop
mtu 9216
channel-group 41 mode active
no shutdown
interface Ethernet1/50
description wolfpack-n9332-l2agg-b
switchport
switchport mode trunk
switchport trunk native vlan 10
switchport trunk allowed vlan 711-714,815,825,999
interface Ethernet1/49
description wolfpack-n9332-l2agg-b
switchport
switchport mode trunk
switchport trunk native vlan 10
switchport trunk allowed vlan 711-714,815,825,999
spanning-tree port type network
spanning-tree guard loop
mtu 9216
channel-group 41 mode active
no shutdown
interface Ethernet1/50
description wolfpack-n9332-l2agg-a
switchport
switchport mode trunk
switchport trunk native vlan 10
switchport trunk allowed vlan 711-714,815,825,999
spanning-tree port type network
spanning-tree guard loop
mtu 9216
channel-group 41 mode active
no shutdown
interface port-channel41
description "wolfpack uplink"
switchport
switchport mode trunk
switchport trunk native vlan 10
switchport trunk allowed vlan 711-714,815,825,999
spanning-tree port type network
mtu 9216
no lacp suspend-individual
vpc 41
spanning-tree port type network
spanning-tree guard loop
mtu 9216
channel-group 41 mode active
no shutdown
interface port-channel41
description “wolfpack uplink”
switchport
switchport mode trunk
switchport trunk native vlan 10
switchport trunk allowed vlan 711-714,815,825,999
spanning-tree port type network
mtu 9216
no lacp suspend-individual
vpc 41
Aggregation switches
The two aggregation switches in the PowerFlex rack used in the Dell EMC laboratory were Cisco 9332 switches. They were configured as standard except for extending the Storage Data network north.
Table 15. Required modification for this POC
9332 Aggregation Switch A
9332 Aggregation Switch B
interface Ethernet1/16
description to Customer SwitchA
switchport
switchport mode trunk
switchport trunk allowed vlan 711-714,848,858
mtu 9216
channel-group 20 mode active
no shutdown
interface Ethernet1/20
description To Customer SwitchB
switchport
switchport mode trunk
switchport trunk allowed vlan 711-714,848,858
mtu 9216
channel-group 20 mode active
no shutdown
interface port-channel20
description To Customer Switches
switchport
switchport mode trunk
switchport trunk allowed vlan 711-714,848,858
mtu 9216
no lacp suspend-individual
vpc 20
interface Ethernet1/16
description to Customer SwitchB
switchport
switchport mode trunk
switchport trunk allowed vlan 711-714,848,858
mtu 9216
channel-group 20 mode active
no shutdown
interface Ethernet1/20
description To Customer SwitchA
switchport
switchport mode trunk
switchport trunk allowed vlan 711-714,848,858
mtu 9216
channel-group 20 mode active
no shutdown
interface port-channel20
description To Customer Switches
switchport
switchport mode trunk
switchport trunk allowed vlan 711-714,848,858
mtu 9216
no lacp suspend-individual
vpc 20
MX7000
This section details the PowerEdge MX7000 changes required for allowing three-layer connectivity to extend through the customer network.
Network I/O Modules
The MX9116n modules provided northbound access to the customer switches. These switches also provided internal connectivity to the compute nodes. The MX7116n Fabric Extenders allowed connectivity to multiple chassis.
Figure 6. MX9116n modules
Note: Management and customer networks were on the A switches, while the storage network was on the B switches.
Management and Customer Traffic - MX9116n FSE
The Dell EMC Networking MX9116n Fabric Switching Engines in Slot A were dedicated to management and customer VLANs.
Table 16. LACP port channel definitions and purposes
Port Channel ID
Purpose
Target
11
ESXi MGMT
CO Node 1
12
ESXi MGMT
CO Node 2
13
ESXi MGMT
CO Node 3
14
ESXi MGMT
CO Node 4
21
Customer Uplink
9332A, 9332B
The following code provides details of the switch configuration as per the information provided in the table.
Table 17. Switch configuration
MX9116n A2
unit-provision 1/71 F0SHS13
!
port-group 1/1/1
mode Eth 25g-8x fabric-expander-mode
!
port-group 1/1/11
mode Eth 100g-2x
!
port-group 1/1/12
mode Eth 100g-2x
!
port-group 1/1/13
mode Eth 40g-1x
!
port-group 1/1/14
mode Eth 40g-1x
!
interface vlan848
description ESXi_mgmt
no shutdown
!
interface port-channel11
description "CO 236 mgmt cust"
no shutdown
switchport mode trunk
switchport access vlan 1
switchport trunk allowed vlan 848
lacp fallback enable
mtu 9216
vlt-port-channel 11
!
interface port-channel12
description "CO 237 mgmt cust"
no shutdown
switchport mode trunk
switchport access vlan 1
switchport trunk allowed vlan 848
lacp fallback enable
mtu 9216
vlt-port-channel 12
spanning-tree port type edge
!
interface port-channel13
description "CO 238 mgmt cust"
no shutdown
switchport mode trunk
switchport access vlan 1
switchport trunk allowed vlan 848
lacp fallback enable
mtu 9216
vlt-port-channel 13
spanning-tree port type edge
!
interface port-channel14
description "CO 239 mgmt cust"
no shutdown
switchport mode trunk
switchport access vlan 1
switchport trunk allowed vlan 848
lacp fallback enable
mtu 9216
vlt-port-channel 14
!
interface port-channel21
description 9332
no shutdown
switchport mode trunk
switchport access vlan 1
switchport trunk allowed vlan 848
lacp fallback enable
mtu 9216
vlt-port-channel 21
!
interface ethernet1/1/1
description "M740-236 - vmnic0"
no shutdown
channel-group 11 mode active
fec off
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/3
description "M740-237 - vmnic0"
no shutdown
channel-group 12 mode active
fec off
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/37
description "Peer-link 9116-A2"
no shutdown
no switchport
flowcontrol receive off
!
interface ethernet1/1/38
description "Peer-link 9116-A2"
no shutdown
no switchport
flowcontrol receive off
!
interface ethernet1/1/39
description "Peer-link 9116-A2"
no shutdown
no switchport
flowcontrol receive off
!
interface ethernet1/1/40
description "Peer-link 9116-A2"
no shutdown
no switchport
flowcontrol receive off
!
interface ethernet1/1/41:1
description customer-9332
no shutdown
channel-group 21 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/42:1
description customer-9332
no shutdown
channel-group 21 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/71/1
description "M740-238 - vmnic0"
no shutdown
channel-group 13 mode active
fec off
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/71/2
description "M740-239 - vmnic0"
no shutdown
channel-group 14 mode active
fec off
no switchport
mtu 9216
flowcontrol receive off
unit-provision 1/71 F0TJH13
!
port-group 1/1/1
mode Eth 25g-8x fabric-expander-mode
!
port-group 1/1/11
mode Eth 100g-2x
!
port-group 1/1/12
mode Eth 100g-2x
!
port-group 1/1/13
mode Eth 40g-1x
!
port-group 1/1/14
mode Eth 40g-1x
!
interface vlan848
description ESXi_mgmt
no shutdown
!
interface port-channel11
description "CO 236 mgmt cust"
no shutdown
switchport mode trunk
switchport access vlan 1
switchport trunk allowed vlan 848
lacp fallback enable
mtu 9216
vlt-port-channel 11
!
interface port-channel12
description "CO 237 mgmt cust"
no shutdown
switchport mode trunk
switchport access vlan 1
switchport trunk allowed vlan 848
lacp fallback enable
mtu 9216
vlt-port-channel 12
spanning-tree port type edge
!
interface port-channel13
description "CO 238 mgmt cust"
no shutdown
switchport mode trunk
switchport access vlan 1
switchport trunk allowed vlan 848
lacp fallback enable
mtu 9216
vlt-port-channel 13
spanning-tree port type edge
!
interface port-channel14
description "CO 239 mgmt cust"
no shutdown
switchport mode trunk
switchport access vlan 1
switchport trunk allowed vlan 848
lacp fallback enable
mtu 9216
vlt-port-channel 14
!
interface port-channel21
description 9332
no shutdown
switchport mode trunk
switchport access vlan 1
switchport trunk allowed vlan 848
lacp fallback enable
mtu 9216
vlt-port-channel 21
!
interface ethernet1/1/1
description "M740-238 - vmnic1"
no shutdown
channel-group 11 mode active
fec off
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/3
description "M740-239 - vmnic1"
no shutdown
channel-group 12 mode active
fec off
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/37
description "Peer-link 9116-A1"
no shutdown
no switchport
flowcontrol receive off
!
interface ethernet1/1/38
description "Peer-link 9116-A1"
no shutdown
no switchport
flowcontrol receive off
!
interface ethernet1/1/39
description "Peer-link 9116-A1"
no shutdown
no switchport
flowcontrol receive off
!
interface ethernet1/1/40
description "Peer-link 9116-A1"
no shutdown
no switchport
flowcontrol receive off
!
interface ethernet1/1/41:1
description customer-9332
no shutdown
channel-group 21 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/42:1
description customer-9332
no shutdown
channel-group 21 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/71/1
description "M740-236- vmnic1"
no shutdown
channel-group 13 mode active
fec off
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/71/2
description "M740-237 - vmnic1"
no shutdown
channel-group 14 mode active
fec off
no switchport
mtu 9216
flowcontrol receive off
A few clarifications on the preceding code:
- VLAN 848, designated for ESXi management, was also used as the customer VLAN.
- The “show unit-provision” command displayed the unit ID and service tag of the Fabric Expanders attached to a Fabric Switching Engine.
- MX9116n port-group settings were determined based on the type of wire used for the physical port.
Storage traffic - MX9116n FSE
The Dell EMC Networking MX9116n Fabric Switching Engines in Slot B were dedicated to storage traffic and allowed all VLANs northbound.
Table 18. LACP port channel definitions and purposes
Port Channel ID
Purpose
Target
15
Data1-Data4, vMotion
CO Node 1
16
Data1-Data4, vMotion
CO Node 2
17
Data1-Data4, vMotion
CO Node 3
18
Data1-Data4, vMotion
CO Node 4
22
Customer Uplink
9332A, 9332B
Table 19. VLAN details
VLAN ID
Purpose
720
Storage Data 1
721
Storage Data 2
722
Storage Data 3
723
Storage Data 4
104
PXE
106
vMotion
Table 20. Switch configuration
MX9116n B1
MX9116n A2
unit-provision 1/71 F0SGS13
!
port-group 1/1/1
mode Eth 25g-8x fabric-expander-mode
!
port-group 1/1/11
mode Eth 100g-2x
!
port-group 1/1/12
mode Eth 100g-2x
!
port-group 1/1/13
mode Eth 40g-1x
!
port-group 1/1/14
mode Eth 40g-1x
!
interface vlan104
description pxe
no shutdown
!
interface vlan106
description vmotion
no shutdown
!
interface vlan720
description data1
no shutdown
mtu 9216
!
interface vlan721
description data2
no shutdown
mtu 9216
!
interface vlan722
description data3
no shutdown
mtu 9216
!
interface vlan723
description data4
no shutdown
mtu 9216
!
interface vlan858
description fos_mgmt
no shutdown
!
interface port-channel15
description "CO 236 vmotion d1-d4"
no shutdown
switchport mode trunk
switchport access vlan 1
switchport trunk allowed vlan 106,720-723
lacp fallback enable
mtu 9216
vlt-port-channel 15
!
interface port-channel16
description "CO 237 vmotion d1-d4"
no shutdown
switchport mode trunk
switchport access vlan 1
switchport trunk allowed vlan 106, 720-723
lacp fallback enable
mtu 9216
vlt-port-channel 16
spanning-tree port type edge
!
interface port-channel17
description "CO 238 vmotion d1-d4"
no shutdown
switchport mode trunk
switchport access vlan 1
switchport trunk allowed vlan 106, 720-723
lacp fallback enable
mtu 9216
vlt-port-channel 17
!
interface port-channel18
description "CO 239 vmotion d1-d4"
no shutdown
switchport mode trunk
switchport access vlan 1
switchport trunk allowed vlan 106,720-723
lacp fallback enable
mtu 9216
vlt-port-channel 18
!
interface port-channel22
description 9332
no shutdown
switchport mode trunk
switchport access vlan 1
switchport trunk allowed vlan 104,106,720-723,858
lacp fallback enable
mtu 9216
vlt-port-channel 22
!
interface ethernet1/1/1
description "M740-236 - vmnic2"
no shutdown
channel-group 15 mode active
fec off
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/3
description "M740-237 - vmnic2"
no shutdown
channel-group 16 mode active
fec off
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/37
description "Peer-link 9116-B1"
no shutdown
no switchport
flowcontrol receive off
!
interface ethernet1/1/38
description "Peer-link 9116-B1"
no shutdown
no switchport
flowcontrol receive off
!
interface ethernet1/1/39
description "Peer-link 9116-B1"
no shutdown
no switchport
flowcontrol receive off
!
interface ethernet1/1/40
description "Peer-link 9116-B1"
no shutdown
no switchport
flowcontrol receive off
!
interface ethernet1/1/41:1
description customer-9332
no shutdown
channel-group 22 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/42:1
description customer-9332
no shutdown
channel-group 22 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/71/1
description "M740-238 - vmnic2"
no shutdown
channel-group 17 mode active
fec off
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/71/2
description "M740-239 - vmnic2"
no shutdown
channel-group 18 mode active
fec off
no switchport
mtu 9216
flowcontrol receive off
unit-provision 1/71 F0THS13
!
port-group 1/1/1
mode Eth 25g-8x fabric-expander-mode
!
port-group 1/1/11
mode Eth 100g-2x
!
port-group 1/1/12
mode Eth 100g-2x
!
port-group 1/1/13
mode Eth 40g-1x
!
port-group 1/1/14
mode Eth 40g-1x
!
interface vlan104
description pxe
no shutdown
!
interface vlan106
description vmotion
no shutdown
!
interface vlan152
description data1
no shutdown
mtu 9216
!
interface vlan160
description data2
no shutdown
mtu 9216
!
interface vlan168
description data3
no shutdown
mtu 9216
!
interface vlan176
description data4
no shutdown
mtu 9216
!
interface vlan858
description fos_mgmt
no shutdown
!
interface port-channel15
description "CO 236 vmotion d1-d4"
no shutdown
switchport mode trunk
switchport access vlan 1
switchport trunk allowed vlan 106,720-723
lacp fallback enable
mtu 9216
vlt-port-channel 15
!
interface port-channel16
description "CO 237 vmotion d1-d4"
no shutdown
switchport mode trunk
switchport access vlan 1
switchport trunk allowed vlan 106,720-723
lacp fallback enable
mtu 9216
vlt-port-channel 16
spanning-tree port type edge
!
interface port-channel17
description "CO 238 vmotion d1-d4"
no shutdown
switchport mode trunk
switchport access vlan 1
switchport trunk allowed vlan 106,720-723
lacp fallback enable
mtu 9216
vlt-port-channel 17
!
interface port-channel18
description "CO 239 vmotion d1-d4"
no shutdown
switchport mode trunk
switchport access vlan 1
switchport trunk allowed vlan 106,720-723
lacp fallback enable
mtu 9216
vlt-port-channel 18
!
interface port-channel22
description 9332
no shutdown
switchport mode trunk
switchport access vlan 1
switchport trunk allowed vlan 104,106,720-723,858
lacp fallback enable
mtu 9216
vlt-port-channel 22
!
interface ethernet1/1/1
description "M740-238 - vmnic3"
no shutdown
channel-group 15 mode active
fec off
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/3
description "M740-239 - vmnic3"
no shutdown
channel-group 16 mode active
fec off
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/37
description "Peer-link 9116-B2"
no shutdown
no switchport
flowcontrol receive off
!
interface ethernet1/1/38
description "Peer-link 9116-B2"
no shutdown
no switchport
flowcontrol receive off
!
interface ethernet1/1/39
description "Peer-link 9116-B2"
no shutdown
no switchport
flowcontrol receive off
!
interface ethernet1/1/40
description "Peer-link 9116-B2"
no shutdown
no switchport
flowcontrol receive off
!
interface ethernet1/1/41:1
description customer-9332
no shutdown
channel-group 22 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/1/42:1
description customer-9332
no shutdown
channel-group 22 mode active
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/71/1
description "M740-236 - vmnic4"
no shutdown
channel-group 17 mode active
fec off
no switchport
mtu 9216
flowcontrol receive off
!
interface ethernet1/71/2
description "M740-237 - vmnic4"
no shutdown
channel-group 18 mode active
fec off
no switchport
mtu 9216
flowcontrol receive off
MX740c Compute sleds
The MX7000 chassis slots 1 and 2 were occupied by two MX740c compute sleds in each chassis and were designed to function as compute-only-nodes.
Figure 7. MX740 sleds slot placement
VMware ESXi
The ESXi installation followed best practices, using the version detailed in Validated components section.
Prerequisite
Ensure that the network configuration on the MX9116n I/O modules is completed.
Procedure
The following steps provide high-level configuration details required to get the compute-only network operational:
- Install ESXi, following VMware best practices.
- Connect the newly created compute-only nodes to the PowerFlex management vCenter.
The following screenshot is an example that shows MX740 nodes added to the PowerFlex Management node:
- Create vSphere Distributed Switches following PowerFlex standards.
The following example shows CO-DSwitch1 with an MTU of 9000:
- Add the hosts to the VDS, following VMware best practices.
The following table shows the correct data network assignments for the specific sleds used in this POC:
Vmnic
Network
Vmnic2, vmnic3
CO Data 1
Vmnic2, vmnic3
CO Data 2
Vmnic2, vmnic3
CO Data 3
Vmnic2, vmnic3
CO Data 4
Vmnic0, vmnic1
CO vMotion
Vmnic0, vmnic1
CO Management
- Enable LACP.
The following example shows the final LAG group configuration:
- Create VMkernel adapters as per PowerFlex standards.
Results
The VMkernels point to the gateways created on the customer switches.
Check the following examples:
- VDS Topology Overview for the Management/Data vDS.
- VDS Topology Overview for the Storage vDS.
- Ensure that Jumbo Frames are enabled and operational.
Example - From inside ESXi terminal: ping Data1 gateway, when Data1 = vmk2 and gateway = 10.1.153.1
vmkping -I vmk2 -d -s 8972 10.1.153.1
SDC Install and Configuration
Install the SDC with the appropriate parameters to connect with an existing PowerFlex system. This procedure is relevant both for adding more SDCs to an existing system, and for adding SDCs to a two-layer system during initial deployment activities.
Prerequisites
Ensure that you have:
- The virtual IP address or MDM IP address of the existing system. If an MDM virtual IP address is not in use, obtain the IP addresses of all the MDM managers.
- Login credentials for the intended SDC host.
- Download the required installation software package for the SDC operating system from the Technical Support site.
- A GUID (Global Unique Identifier) string for the SDC. These strings can be generated by tools that are freely available online. The GUID must conform to OSF DCE 1.1. The expected format is xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx where each x can be a digit (0–9) or a letter (a–f).
Procedure
The following procedure explains how to manually install an external SDC on an ESXi server using esxcli in command line.
Note: This procedure requires two server reboots.
- Install the SDC on an ESXi 7.0.x server:
- Copy the SDC file to the tmp directory on the local datastore on host
- SSH to the host and run the following command:
esxcli software vib install -d /tmp/sdc-3.5.1100.101-esx7.x.zip --no-sig-check
- Reboot the PowerFlex node.
- Run the following command to set the IP address of the MDM:
esxcli system module parameters set -m scini -p "IoctlIniGuidStr=<XXXXXX> IoctlMdmIPStr=<LIST_VIP_MDM_IPS> IoctMdmPasswordStr=<ip>-<password> bBlkDevIsPdlActive=1 blkDevPdlTimeoutMillis=60000"
where “IoctlIniGuidStr” is the unique value defined earlier.
- Perform another reboot.
- Ensure that the driver is loaded:
vmkload_mod -l | grep scini
Output similar to the following indicates that the driver is loaded:
scini 2 808
The SDC component is installed on the ESXi host.
- Verify the SDC is added in the PowerFlex Presentation UI,
The image below shows a successfully connected SDC.
- Volumes can now be created and mapped to the compute nodes.