In this deployment, all spine switches are in one autonomous system. This BGP EVPN topology shows a leaf-spine data center network in which eBGP peer sessions between leaf and spine switches advertise both underlay IP routes and EVPN routes. To advertise underlay IP and EVPN routes, eBGP unnumbered peer sessions are established between the leaf, and spine nodes using an interface IPv6 link local address.
Follow the procedures described in Dell leaf switches to update your user credentials and password. You will be prompted to change your password at initial login.
Configure the spine hostname.
- Connect to the Management Framework CLI (MF-CLI) from Legacy SONiC CLI using the sonic-cli command.
- Enter configuration mode using the configure terminal command.
- Enter interface-naming standard in configuration mode. Throughout this deployment guide, we have used Standard Interface Naming mode. You will be prompted to exit the session and then reenter the MF-CLI.
- Exit the MF-CLI session to the Linux shell to activate the interface naming standard mode.
Sonic-cli configure terminal interface-naming standard end write memory exit | Sonic-cli configure terminal interface-naming standard end write memory exit |
- Reenter the MF-CLI using the sonic-cli command for the Standard Interface naming mode. Enter configuration mode using the configure terminal command.
- Set the spine hostname.
- Create ip management vrf.
- Enable ssh-server for management vrf.
- Save the action configuration to the flash using the write memory command.
sonic-cli configure terminal hostname Spine1 ip vrf mgmt ssh-server vrf mgmt end write memory exit sonic-cli | sonic-cli configure terminal hostname Spine2 ip vrf mgmt ssh-server vrf mgmt end write memory exit sonic-cli |
Note: You must exit the MF-CLI session to the Linux shell to activate the interface naming mode and reenter the MF-CLI. The following message is displayed during the SONiC session.
“Broadcast message: Interface naming mode has changed. Users running 'sonic-cli' are required to restart your session.”
“Broadcast message: Hostname has been changed from sonic to Spine1 or Spine2. Users running 'sonic-cli' are suggested to restart your session.”
Each spine connects to each leaf switch. The configuration provides the underlay portion from the spine switches. After you enable IPv6 on an interface that is connected to a BGP neighbor, an IPv6 link-local address is automatically created. BGP uses the link-local address to set up a BGP session with the neighbor. Unnumbered interfaces use IPv6 router advertisements (RAs) to identify the address of a BGP neighbor.
- As a best practice, provide a description for each interface.
- Set MTU to 9216. This is the default on this switch for this deployment.
- Set the speed to 100 Gbps.
- Bring up the physical interface by using the no shutdown command.
- Use ipv6 enable to enable the link-local address only on the interface, allowing users to forward Layer 3 traffic without configuring an address. The ipv6 enable command creates the routing interface based on the autogenerated IPv6 link-local address.
configure terminal interface Eth1/1 description Leaf1A mtu 9216 speed 100000 no shutdown ipv6 enable ! interface Eth1/2 description Leaf1B mtu 9216 speed 100000 no shutdown ipv6 enable ! interface Eth1/3 description Leaf2A mtu 9216 speed 100000 no shutdown ipv6 enable ! interface Eth1/4 description Leaf2B mtu 9216 speed 100000 no shutdown ipv6 enable ! end write memory | configure terminal interface Eth1/1 description Leaf1A mtu 9216 speed 100000 no shutdown ipv6 enable ! interface Eth1/2 description Leaf1B mtu 9216 speed 100000 no shutdown ipv6 enable ! interface Eth1/3 description Leaf2A mtu 9216 speed 100000 no shutdown ipv6 enable ! interface Eth1/4 description Leaf2B mtu 9216 speed 100000 no shutdown ipv6 enable ! end write memory |
Spine Switch Router ID
Configure the spine switch router ids on the loopback interfaces by using the code in the following table:
configure terminal interface Loopback 0 description Router-ID ip address 1.1.1.1/32 exit end write memory | configure terminal interface Loopback 0 description Router-ID ip address 1.1.1.2/32 exit end write memory |
Configure the unnumbered BGP configuration.
- Start the BGP configuration by using the router bgp <AS_number> command.
- Specify the router-ID with the router-id <ip_address> command, Loopback 0.
- Use the bestpath as-path multipath relax command to relax the as-path consideration and allow ECMP.
- Set the BGP Keepalive and Hold-down timers by using the timers <x> <y> command.
- Configure and activate the Layer 3 ipv4 unicast by using address-family ipv4 unicast.
- Set the system to redistribute connected routes.
- Set maximum-paths to 256.
- Set maximum-paths ibgp to 1.
- Configure and activate the Layer 2 VPN EVPN address family for VXLAN host-based routing to a BGP neighbor using the address-family l2vpn evpn command.
- Set the system to advertise all VNIs.
configure terminal router bgp 65100 router-id 1.1.1.1 log-neighbor-changes bestpath as-path multipath-relax timers 60 180 ! address-family ipv4 unicast redistribute connected maximum-paths 256 maximum-paths ibgp 1 ! address-family l2vpn evpn advertise-all-vni | configure terminal router bgp 65100 router-id 1.1.1.2 log-neighbor-changes bestpath as-path multipath-relax timers 60 180 ! address-family ipv4 unicast redistribute connected maximum-paths 256 maximum-paths ibgp 1 ! address-family l2vpn evpn advertise-all-vni |
- Define Peer Group LEAF using the neighbor LEAF peer-group command to improve scaling by generating the same update information to all leaf neighbors.
- Use the remote-as external command to only create a peer when an ASN is specified. If the ASN is the same, the connection is denied.
- Set the BGP Keepalive and Hold-down timers using the timers <x> <y> command.
- To maintain a stable routing table, set the Advertisement Interval using the advertisement-interval <x> command.
- Enable Bi-directional Forwarding Detection by using the bfd command.
- Use the capability extended-nexthop command to allow BGP to negotiate the extended-nexthop capability with its peer.
- Configure and activate the Layer 3 ipv4-unicast by running address-family ipv4 unicast.
- Configure and activate the Layer 2 VPN EVPN address family for VXLAN host-based routing to a BGP neighbor by using the address-family l2vpn evpn command.
- Add the interfaces Eth1/1, Eth1/2, Eth1/3, and Eth1/4 to the Peer-Group Leaf.
! peer-group leaf remote-as external timers 3 9 timers connect 30 advertisement-interval 0 bfd capability extended-nexthop ! address-family ipv4 unicast activate send-community both ! address-family l2vpn evpn activate ! neighbor interface Eth1/1 peer-group leaf ! neighbor interface Eth1/2 peer-group leaf ! neighbor interface Eth1/3 peer-group leaf ! neighbor interface Eth1/4 peer-group leaf ! end write memory | ! peer-group leaf remote-as external timers 3 9 timers connect 30 advertisement-interval 0 bfd capability extended-nexthop ! address-family ipv4 unicast activate send-community both ! address-family l2vpn evpn activate ! neighbor interface Eth1/1 peer-group leaf ! neighbor interface Eth1/2 peer-group leaf ! neighbor interface Eth1/3 peer-group leaf ! neighbor interface Eth1/4 peer-group leaf ! end write memory |
The configuration of the Leaf-Spine fabric with two racks, two leaf-pairs, and two spines is complete. The VxRail nodes can be deployed across the two racks and are now able to communicate across the leaf-spine fabric. Create a VM on each node for validation.
Note: To add additional VxRail Clusters, repeat the planning processes and configuration procedures described in this chapter. For any additional clusters, refer to the appropriate VxRail Cluster Procedures and requirements on the VxRail Solve website (account required).