The following diagram highlights the switches configured as border leafs. BLeaf25 and BLeaf26 are configured as an MCLAG pair and are also supporting hosts.
- Enter Management Framework CLI (MF-CLI) and then configuration mode.
- Change the interface naming mode to Standard, add hostname, save configuration and exit back to Linux shell to activate the changes.
BLeaf25 BLeaf26 sonic-cli configure terminal interface-naming standard hostname BLeaf25 end write memory exit
sonic-cli configure terminal interface-naming standard hostname BLeaf26 end write memory exit
- Re-enter the MF-CLI and configuration mode.
- Assign a unique router ID on each leaf.
- Then enable IPv6 on spine uplinks to support unnumbered BGP point-to-point links and enable the interfaces.
BLeaf25 BLeaf26 sonic-cli configure terminal interface loopback 0 description router-id ip address 10.0.2.25/32 exit interface Eth 1/53 description Spine1 ipv6 enable no shutdown interface Eth 1/56 description Spine4 ipv6 enable no shutdown exit
sonic-cli configure terminal interface loopback 0 description router-id ip address 10.0.2.26/32 exit interface Eth 1/53 description Spine1 ipv6 enable no shutdown interface Eth 1/56 description Spine4 ipv6 enable no shutdown exit
- Assign a VRF for each tenant. This POC only has one tenant, but will support multiple tenants with additional VRFs.
- Configure VLANs 80 and 140 for the hosts, bind to the tenant’s VRF and assign the same anycast-address to each host VLAN.
- Assign a dedicated VLAN (Vlan60) for a L3 VNI and bind to the tenant’s VRF.
Note: The VLAN assigned to the L3 VNI does not need an IP address.
BLeaf25 BLeaf26 ip vrf Vrf-tenant1 interface Vlan 80 ip vrf forwarding Vrf-tenant1 ip anycast-address 192.168.80.254/24 neigh-suppress exit interface Vlan 140 ip vrf forwarding Vrf-tenant1 ip anycast-address 192.168.140.254/24 neigh-suppress exit interface Vlan 60 ip vrf forwarding Vrf-tenant1 exit
ip vrf Vrf-tenant1 interface Vlan 80 ip vrf forwarding Vrf-tenant1 ip anycast-address 192.168.80.254/24 neigh-suppress exit interface Vlan 140 ip vrf forwarding Vrf-tenant1 ip anycast-address 192.168.140.254/24 neigh-suppress exit interface Vlan 60 ip vrf forwarding Vrf-tenant1 exit
- Configure the MCLAG peer-link and port channel members.
- Configure all VLANs on the MCLAG’s peer-link port channel.
BLeaf25 BLeaf26 interface PortChannel 100 description MCLAG-Peer-Link switchport trunk allowed vlan add 60,80,140 exit interface Eth 1/49 description MCLAG-Peer-Link channel-group 100 no shutdown exit interface Eth 1/51 description MCLAG-Peer-Link channel-group 100 no shutdown exit
interface PortChannel 100 description MCLAG-Peer-Link switchport trunk allowed vlan add 60,80,140 exit interface Eth 1/49 description MCLAG-Peer-Link channel-group 100 no shutdown exit interface Eth 1/51 description MCLAG-Peer-Link channel-group 100 no shutdown exit
CAUTION: When copying and pasting the content from the table above, ensure that the underlined monospace text remains on one line. Failure to keep the underlined content on one line results in a line error on the switch. - Configure the MCLAG domain.
- The loopback IP address used for the router-id is also used as the MCLAG source and peer IP address.
- Assign the host’s port channel interfaces and assign to the MCLAG domain.
Note: By default, port channels are LACP. Use the interface PortChannel <num> mode on command for static LAGs. If changing LAG modes, you must completely remove and reassign the port channel.
Note: The spine switches are used for the MCLAG peer's keep-alive link communication.
BLeaf25 BLeaf26 mclag domain 1 source-ip 10.0.2.25 peer-ip 10.0.2.26 peer-link PortChannel 100 delay-restore 90 exit interface PortChannel 202 description Host1_PortChannel switchport trunk allowed vlan add 80,140 mclag 1 exit interface Eth 1/3 description Host1_PortChannel channel-group 202 no shutdown exit
mclag domain 1 source-ip 10.0.2.26 peer-ip 10.0.2.25 peer-link PortChannel 100 delay-restore 90 exit interface PortChannel 202 description Host1_PortChannel switchport trunk allowed vlan add 80,140 mclag 1 exit interface Eth 1/3 description Host1_PortChannel channel-group 202 no shutdown exit
- Enable static anycast-address and assign a unique MAC to be used across all leaf switches.
- Create a loopback interface for the VTEP IP address. This address must be the same for each MCLAG peer.
- Create a VXLAN interface and assign the same VTEP source IP address to each peer.
- Assign a unique primary-ip to optimize routing for orphan ports and active-standby hosts. The router-id can be used for this purpose.
- Map the L2 VNI to the host VLANs that are stretched across the fabric.
- Create a L3 VNI by mapping the dedicated VLAN (Vlan60) to the non-default VRF. The Network Virtualization Overlay (NVO) is automatically assigned to the VXLAN interface.
Note: The Network Virtualization Overlay (NVO) is automatically assigned to the VXLAN interface
BLeaf25 BLeaf26 ip anycast-address enable ip anycast-mac-address 00:00:00:00:01:02 interface Loopback 1 description LogicalVTEP ip address 10.10.10.25/32 exit interface vxlan vtep25 source-ip 10.10.10.25 primary-ip 10.0.2.25 map vni 1400 vlan 140 map vni 600 vlan 60 map vni 600 vrf Vrf-tenant1 exit
ip anycast-address enable ip anycast-mac-address 00:00:00:00:01:02 interface Loopback 1 description LogicalVTEP ip address 10.10.10.25/32 exit interface vxlan vtep25 source-ip 10.10.10.25 primary-ip 10.0.2.26 map vni 1400 vlan 140 map vni 600 vlan 60 map vni 600 vrf Vrf-tenant1 exit
- Configure the BGP router with ECMP enabled.
- Configure the underlay’s address-family and redistribute the connected networks.
- Add VNI advertisements to the EVPN address family and assign the MCLAG peer’s primary-ip.
BLeaf25 BLeaf26 router bgp 65025 router-id 10.0.2.25 bestpath as-path multipath-relax address-family ipv4 unicast redistribute connected maximum-paths 2 exit address-family l2vpn evpn advertise-all-vni advertise-pip peer-ip 10.0.2.26 exit
router bgp 65025 router-id 10.0.2.26 bestpath as-path multipath-relax address-family ipv4 unicast redistribute connected maximum-paths 2 exit address-family l2vpn evpn advertise-all-vni advertise-pip peer-ip 10.0.2.25 exit
- Configure the BGP spine peer-group.
- Set timers and enable unnumbered BGP and BFD on the peer-group.
- Activate both IPv4 unicast and EVPN address families for the spine peer group.
- Set allowas-in to support MCLAG peer’s ASN to be received.
BLeaf25 BLeaf26 peer-group SPINE advertisement-interval 5 timers 3 9 remote-as external capability extended-nexthop bfd address-family ipv4 unicast activate allowas-in 1 exit address-family l2vpn evpn activate exit exit
peer-group SPINE advertisement-interval 5 timers 3 9 remote-as external capability extended-nexthop bfd address-family ipv4 unicast activate allowas-in 1 exit address-family l2vpn evpn activate exit exit
- Assign the spine neighbor interfaces to the peer-group.
- Create the BGP router for each tenant’s VRF.
- Save the configuration.
BLeaf25 BLeaf26 neighbor interface Eth 1/53 peer-group SPINE exit neighbor interface Eth 1/56 peer-group SPINE exit exit router bgp 65025 vrf Vrf-tenant1 address-family ipv4 unicast redistribute connected exit address-family l2vpn evpn advertise ipv4 unicast exit exit end write memory
neighbor interface Eth 1/53 peer-group SPINE exit neighbor interface Eth 1/56 peer-group SPINE exit exit router bgp 65025 vrf Vrf-tenant1 address-family ipv4 unicast redistribute connected exit address-family l2vpn evpn advertise ipv4 unicast exit exit end write memory