Below is some networking glossary of terms used for ECS networking:
- Front-end switches - public or Top of Rack (ToR) switches that connect to the customer’s network. This includes the default public network and any defined separated networks such as management, replication or data. The customary name for the two physical components in this switch complex is rabbit and hare. The node is configured to bond the two NICs into a single LACP bonding interface.
- Public network - the default network of the appliance that consists of two bonded interfaces with connections to the public (front-end) switch. By default, all types of public traffic will use the public network unless explicitly defined.
- Management network - an optional separated VLAN network dedicated to hosting the ECS web portal, all common infrastructure services such as NTP, DNS, DHCP and Dell’s secure remote services (SRS).
- Replication network - an optional separated network dedicated to replicating objects between virtual data centers.
- Back-end switches - private switches used for internal maintenance including the ECS private network and the private.4 network (also known as Nile Area Network (NAN)). The customary names for these physical switches are hound and fox. All server nodes in an ECS intra-rack have two connections going to a back-end switch. Both connections are bonded into a single LACP bonding interface.
- Private network - a rack only network used for service operations such as install, reinstall, and expansion.
- Private.4 network - a network which interconnects all co-located ECS racks through their private switches onto a single VLAN, which is VLAN 4 by default. Also referred to as the Nile area network (NAN).
Note: For ECS D- and U-series hardware models, the private switch is referred to as the turtle.
The EX300, EX500, and EX3000 appliances all use the Dell S5148F for the front-end pair of switches and for the pair of back-end switches. Customers have the option of using their own front-end switches instead of the Dell switches.
Error! Reference source not found. below provides a visual representation of how ports are intended in front end network switch to enable ECS node traffic and customer uplink ports. This is standard across all implementations.
Figure 3. Front-end network switch port of S5148F
Note: For an EX300 appliance, the 25GbE ports run at 10GbE.
The diagram below as Error! Reference source not found. provides a visual representation of how ports are intended to be used to enable ECS management traffic and diagnostic ports.
Figure 4. Back-end network switch port of S5148F
The EXF900 is the first release of a high performant ECS object storage appliance offering. The ECS software uses an NVMe engine to configure local NVMe targets and establish NVMe over Fabric (NVMe-oF) connections. High performing and reliable connections to remote storage nodes are accomplished by having a dedicated high available network using Ethernet-based RDMA as the transport fabric.
The EXF900 appliance uses the Dell S5248F for the front-end pair of switches and for the pair of back-end switches and S5232F for the aggregation back-end switch. Note that customers have the option of using their own front-end switches instead of the Dell switches.
Error! Reference source not found. shows a visual representation of how ports in front end network switch to be used to enable ECS node traffic and customer uplink ports.
Figure 5. Front-end network switch port of S5248F
Error! Reference source not found. provides a visual representation of how ports are intended to be used in back-end network switch. These port allocations are standard across all implementations.
Figure 6. Back-end network switch port of S5248F
Dell provides two 100GbE S5232F back-end aggregation switches (AGG1 and AGG2) with four 100GbE VLT cables. These switches are referred to as the Falcon and Eagle switches as shown in Error! Reference source not found.. The number of uplinks between each rack and the aggregation switches ensures all the EXF900 nodes have line rate performance to any node in any rack. This setup allows for low latency and high throughput across the entire cluster.
Figure 7. Aggregation switch port of S5232F
The following documents should be consulted for ease in switch, switch port, and overall network planning:
- ECS Designer (Available from Dell Sales) - Absolutely critical document in the design and provisioning process, especially around switches and their related configuration, and guides users through important questions.
- ECS EX Series Hardware Guide - Provides information about supported hardware configurations, upgrade paths, and rack cabling requirements.
- ECS Networking and Best Practices - A white paper that describes details of ECS networking and specifics on ECS network hardware, network configurations, and network separation.
Table 3. Networking best practice highlights
- Use the ECS Designer throughout the design and deployment process. Record customer provided switch manufacturers, models, and firmware versions.
- Record ECS rack uplink information along with switch and port identifiers and cabling descriptions.
- Reserve the necessary number of ports on the customer’s switch infrastructure.
- Understand the options for port channel configuration.
- See the ECS Networking and Best Practices white paper.