Home > AI Solutions > Gen AI > Guides > Dell PowerScale Storage Deployment Guide for NVIDIA DGX SuperPOD > Networking configuration
There are four types of networks on a PowerScale cluster as described in Figure 2:
A Dell PowerScale cluster will require several IPs in many different subnets. Table 2 shows the different IP needed for 1 SU.
Network Interface Type | Comment |
Backend | 16 IP for int-a (10.10.1.1/24 - 10.10.1.254/24). |
Frontend | The recommended configuration is one subnet with two pools. Each pool will contain half of the IP range for 100gige-1 ports and the other pool is for 100gige-2 ports. In total, thirty-two 100 GbE ports are used in the PowerScale cluster. 33 consecutive IP within the same subnet (100.127.98.0/25).
|
Management | From 1 to 16 IP (1 IP per node). It’s not mandatory to have all management ports configured. A minimum of 2 is recommended. |
iDrac | 16 IP (1 IP per node). |
Open a secure shell (SSH) connection to any node in the cluster and log in. Perform the following commands:
isi network subnets modify groupnet0.subnet0 --mtu=9000
isi network pools modify groupnet0.subnet127.100-gig1 --nfsv3-rroce-only=true
isi network pools modify groupnet0.subnet127.100-gig1 --alloc-method=dynamic
isi network pools modify groupnet0.subnet127.100-gig2 --nfsv3-rroce-only=true
isi network pools modify groupnet0.subnet127.100-gig2 --alloc-method=dynamic
isi network subnets view groupnet0.subnet127
ID: groupnet0.subnet127
Name: subnet127
Groupnet: groupnet0
Pools: 100-gig1, 100-gig2
Addr Family: ipv4
Base Addr: 100.127.98.0
CIDR: 100.127.98.0/25
Description:
Firewall Policy: default_subnets_policy
Gateway: 100.127.98.1
Gateway Priority: 20
MTU: 9000
Prefixlen: 25
Netmask: 255.255.255.128
SC Service Addrs: 100.127.98.10-100.127.98.10
SC Service Name:
VLAN Enabled: False
VLAN ID: -
isi network pools view groupnet0.subnet127.100-gig1
ID: groupnet0.subnet127.100-gig1
Groupnet: groupnet0
Subnet: subnet127
Name: 100-gig1
Rules: -
Access Zone: System
Allocation Method: dynamic
Aggregation Mode: lacp
Description:
Firewall Policy: default_pools_policy
Ifaces: 1:100gige-1, 2:100gige-1, 3:100gige-1, 4:100gige-1, 5:100gige-1, 6:100gige-1, 7:100gige-1, 8:100gige-1, 9:100gige-1, 10:100gige-1, 11:100gige-1, 12:100gige-1, 13:100gige-1, 14:100gige-1, 15:100gige-1, 16:100gige-1
IP Ranges: 100.127.98.11-100.127.98.26
IPv6 Perform DAD: No
Rebalance Policy: auto
SC Failover Policy: round_robin
Static Routes: -
NFS RDMA RRoCE only: Yes
isi network pools view groupnet0.subnet127.100-gig2
ID: groupnet0.subnet127.100-gig2
Groupnet: groupnet0
Subnet: subnet127
Name: 100-gig2
Rules: -
Access Zone: System
Allocation Method: dynamic
Aggregation Mode: lacp
Description:
Firewall Policy: default_pools_policy
Ifaces: 1:100gige-2, 2:100gige-2, 3:100gige-2, 4:100gige-2, 5:100gige-2, 6:100gige-2, 7:100gige-2, 8:100gige-2, 9:100gige-2, 10:100gige-2, 11:100gige-2, 12:100gige-2, 13:100gige-2, 14:100gige-2, 15:100gige-2, 16:100gige-2
IP Ranges: 100.127.98.27-100.127.98.42
IPv6 Perform DAD: No
Rebalance Policy: auto
SC Failover Policy: round_robin
Static Routes: -
NFS RDMA RRoCE only: Yes
mkdir /ifs/superpod
chmod a+rwx /ifs/superpod
isi nfs exports create /ifs/superpod
isi nfs settings global modify --nfsv3-rdma-enabled=true
isi services nfs enable
isi compression settings modify --enabled=0
isi dedupe inline settings modify --mode=disabled
isi_sysctl_cluster isi.access.default.prefetch.offload=1
isi_gconfig registry.Services.scheduler.Parameters.FHAMaxConcurrency=24
isi_gconfig registry.Services.scheduler.Parameters.FHAFileBytes=$((24*1024*1024))
# This command will restart nfs service on every PowerScale nodes. Use carefully !
isi_for_array /usr/likewise/bin/lwsm shutdown
# OneFS 9.9 Specific
# This feature will more aggressively reclaim sub-file data-locks that have been dropped due to inactivity.
isi_sysctl_cluster efs.bam.datalock.coordinator.expand_ranges=1
# This feature includes performance improvements for workloads that do heavy concurrent access to a single file.
isi_sysctl_cluster efs.bam.l1_shard_enabled=1
# flush IFS L1 and L2 caches
isi_for_array isi_flush