The data network is primarily composed of the ToR and the aggregation switches. The following component blocks make up this network:
- Server nodes
- Leaf Switches
- Spine switches
- Core
- Layer-2 and Layer-3 Switching
- VLANs
- Management network services
- Dell EMC OpenSwitch solution
Server nodes
In order to create a highly-available solution, the network must be resilient to loss of a single network switch, network interface card (NIC) or bad cable. To achieve this, the network configuration uses bonding across the servers and switches.
There are several types (or modes) of bonding, but only one is recommended for the Solution. The OpenStack Controller, Compute nodes, Red Hat Ceph storage nodes, and Solution Admin Host can use:
802.3ad or LACP (mode = 4)
balance-rr
(mode=0),
balance-xor
(mode=2),
broadcast
(mode=3),
balance-tlb
(mode=5), and
balance-alb
(mode=6), are not supported. Please check with your technician for the current support status of
active-backup
(mode = 1).
All nodes' endpoints are terminated to switch ports that have been configured for LACP bonding mode across two S5232F-ON ToR switches for 25GbE/100GbE configured with a Virtual Link Trunking interconnect (VLTi) across them.
Please contact your Dell EMC sales representative for other viable options.
Bonding type | |
Node type | 802.3ad (LACP mode 4) |
Solution Admin Host | Yes (solution default) |
OpenStack Controller nodes | Yes (solution default) |
OpenStack Compute nodes | Yes (solution default) |
Red Hat Ceph storage nodes | Yes (solution default) |
OpenStack Hyper-Converged Infrastructure nodes | Yes (solution default) |
A single port is an option when bonding is not required. However, it is neither used nor validated in the Dell EMC Ready Architecture for Red Hat OpenStack Platform. The need to eliminate single points of failure is taken into consideration as part of the design, and this option has been eliminated wherever possible.
Please contact your Dell EMC sales representative for other configurations.
Leaf Switches
Dell EMC’s recommended architecture uses VLT for HA between the two Leaf switches, which enables the servers to terminate their Link Aggregation Group (LAG) interfaces (or bonds) into two different switches instead of one. This configuration enables active-active bandwidth utilization and provides redundancy within the rack if one Leaf switch fails or requires maintenance. Dell EMC recommended Leaf switch is:
- 10/25/100GbE connectivity – S5232F-ON
The Leaf switches are responsible for providing the different network connections such as tenant networks, storage networks etc, between the different nodes of the OpenStack deployment such as compute, controller and storage nodes.
Spine switches
Layer-2 and Layer-3 Switching
The layer-2 and layer-3 boundaries are separated at the aggregation layer.
The Architecture Guide uses layer-2 as the reference up to the Leaf layer, which is why VLT is used on the Leaf switches.
The three network links - Provisioning, Storage, and Management - can have uplinks to a gateway device. The Provisioning network can use the Red Hat OpenStack Director as a proxy for pulling packages from a subscription server, or a gateway can be added. The Red Hat Ceph storage, Dell EMC Unity, SC series storage or the Dell EMC PowerMax storage arrays on the Storage network may need access:
- From metrics and monitoring tools
- To enable management and updates
There are many tools for Out of Band management, (OOB) for the iDRAC, which you can use after first adding the gateway to the network, and then updating the iDRAC.
The OpenStack Controllers are connected to a gateway device, usually a router or firewall. This device will handle routing for all networks external to the cluster. The required networks are:
- The floating IP range used by virtual machines
- A network for all external Public API and Graphical User Interface access
VLANs
This Architecture Guide implements at a minimum eight (8) separate Layer 2 VLANs:
- External Network VLAN for Tenants—Sets up a network that will support the floating IPs and default external gateway for tenants and virtual machines. This connection is through a router external to the cluster.
- Internal Networks VLAN for Tenants—Sets up the backend networks for Nova and the VMs to use.
- Management/Out of Band (OOB) Network—iDRAC connections can be routed to an external network. All OpenStack HA Controllers need direct access to this network for IPMI operations.
- Private API Network VLAN—Used for communication between OpenStack Controllers, the Red Hat OpenStack Director, and Compute nodes for Private API and cluster communications.
- Provisioning Network VLAN—Connects a NIC from all nodes into the fabric, used for setup and provisioning of the OpenStack servers and access to the Red Hat Ceph Storage Dashboard.
- Public API Network VLAN—Sets up the network connection to a router that is external to the cluster. The network is used by the front-end network for routable traffic to individual VMs, access to the OpenStack API, RADOS Gateway, and the Horizon GUI. Depending upon the network configuration these networks may be either shared or routed, as needed. The Red Hat OpenStack Director requires access to the Public API Network.
- Storage Clustering Network VLAN—Used by all Storage nodes for replication and data checks (Red Hat Ceph storage clustering).
- Storage Network VLAN—Used by all nodes for the data plane reads/writes to communicate to OpenStack Storage; setup, and provisioning of the Red Hat Ceph storage cluster; and when included, the Dell EMC Unity storage or SC series storage arrays.
- Tenant Tunnel Network VLAN—Used by Tenants for encapsulated networks such as, GRE or VXLAN tunnels, in place of the Internal Networks VLAN for Tenants.
Management network services
The management network and the provisioning network for all the servers and switches aggregate into a Dell EMC Networking S3048-ON switch.
The Management network services is used for several functions:
- The highly available software uses it to reboot and partition servers.
- An uplink to a router and an iDRAC configure a gateway monitoring the servers and gathering metrics.
Dell EMC OpenSwitch solution
In addition to the Dell EMC switch-based Architecture Guide, Dell EMC provides an open standard that enables you to choose other brands and configurations of switches for your OpenStack environment.
The following list of requirements will enable other brands of switches to properly operate with Dell EMC's required tools and configurations:
- Support for IEEE 802.1Q VLAN traffic and port tagging
- Support for using one untagged, and multiple tagged VLANs, on the same port
- Support for using bonded interfaces as a single interface for TFTP/DHCP booting
- Ability to provide a minimum of 96 x 25 GbEs Ethernet ports in a non-blocking configuration within the Provisioning VLAN
- Configuration can be a single switch or a combination of stacked switches to meet the additional requirements
- The ability to create LAGs with a minimum of two physical links in each LAG
- If multiple switches are stacked:
- The ability to create a LAG across stacked switches
- Full-bisection bandwidth
- Support for VLANs to be available across all switches in the stack
- 250,000 packets-per-second capability per switch
- A managed switch that supports SSH and serial line configuration
- SNMP v3 support