Appendix A: Cloud Foundation on VxRail checklist
Use Cases | - Determination of use cases planned for VCF on VxRail integrated platform
- Determination of application availability requirements for VCF on VxRail integrated platform
|
Workload Planning | - Captured performance metrics from applications targeted for VCF on VxRail integrated platform
- Completed sizing exercise with Dell Technologies VCF on VxRail sizing tool
- Convert sizing report into top-level architecture for VCF on VxRail integrated platform
|
Data Center Requirements |
Rack Space | - Calculated data center rack space and power requirements for VCF on VxRail integrated platform
- Enable redundant power sources for each data center rack.
|
Data Center Infrastructure | - Ethernet switch ports compatible with VxRail node ports
- Sufficient open ports for VxRail nodes
- Jumbo frames enabled on data center network
- Ethernet switches supporting VCF on VxRail integrated platform support Unicast and, if applicable, Multicast
- Ethernet switches supporting VCF on VxRail integrated platform support Border Gateway Protocol
- Ethernet switches supporting VCF on VxRail integrated platform support hardware-based VTEP, if desired
- If planning to use FC storage to support VI workload, Fibre Channel infrastructure compatible with VxRail
|
Data Center Services | - Domain Name Services (DNS) deployed in data center planned for VCF on VxRail integrated platform
- Network Time Protocol (NTP) services deployed in data center planned for VCF on VxRail integrated platform
- Active Directory (A-D) configured in data center planned for VCF on VxRail integrated platform (required for certain use cases)
- Dynamic Host Configuration Protocol (DHCP) services configured in data center planned for VCF on VxRail integrated platform. This is not required if using static IP addresses for the host overlay networks.
- SFTP server for backups for NSX-T and SDDC Manager instances configured in data center planned for VCF on VxRail integrated platform
- Certificate generation utility (required for certain use cases)
- Syslog server (optional)
|
WAN | - Minimum 10 Mb bandwidth between the central management site and any planned remote workload sites
- Maximum 50-millisecond latency between the central management site and any planned remote workload sites
- WAN redundancy enabled between the central management site and any planned remote workload sites
- Geneve-compatible network between VCF on VxRail management domain instances across regions if NSX-T Federation is planned.
|
Licensing |
Licenses | - vCenter Server Standard
- ESXi Enterprise Plus (Management and VI Workload Domains)
- ESXi Enterprise Plus for Kubernetes (VI Workload Domains for Kubernetes)
- vSAN Advanced or higher
- NSX-T Data Center
- vRealize Suite (minimum 2019)
|
Credentials |
| - Login credentials for Dell-Technologies support site
- Login credentials for VMware support site
|
Reserve VLANs | - One external management VLAN for VxRail Manager, vCenter Servers, NSX-T management components, and SDDC Manager deployed in management workload domain requiring external access
- One internal management VLAN with IPv6 multicast for VxRail node auto-discovery and device management. The default is 3939.
- One VLAN with IPv4 unicast for vSAN traffic, unless planning for FC storage
- One VLAN for vSphere vMotion
- One VLAN for NSX-T Host Overlay network
- One VLAN for the first NSX-T edge uplink (for NSX-T edge services)
- One VLAN for the second NSX-T edge uplink (for NSX-T edge services)
- One VLAN for the NSX-T edge overlay network (for NSX-T edge services)
- One VLAN for IDRAC management of the VxRail nodes
|
| - Determine default gateway and subnet mask.
- Reserve IP addresses for VxRail nodes for each VxRail cluster.
- Reserve one IP address for vCenter Server.
- Reserve one IP address for VxRail Manager.
- Decide whether you want to use the default TCP-IP stack for vMotion, or a separate IP addressing scheme for the dedicated vMotion TCP-IP stack.
- Reserve IP addresses and a subnet mask for vSphere vMotion.
- Select the gateway for either the default TCP-IP stack, or the dedicated vMotion TCP-IP stack.
- Reserve IP addresses and a subnet mask for vSAN, unless using external storage for VI workload.
- Reserve IP address for SDDC Manager.
- Reserve IP addresses for NSX-T Management VIP and appliance nodes.
- Reserve IP addresses for the first NSX-T edge uplink (for NSX-T edge services).
- Reserve IP addresses for the second NSX-T edge uplink (for NSX-T edge services).
- Reserve IP addresses for the NSX-T edge overlay network (for NSX-T edge services).
- Reserve IP addresses for the NSX-T host overlay network (unless using DHCP).
- If witness is required for stretched cluster, reserve one IP address for the management network and one IP address for the vSAN network.
- If NSX-T Federation is a requirement, reserve IP addresses for the remote TEPs on the edge gateways in each region.
|
Reserve Hostnames | - Determine parent and child DNS domains.
- Decide on your VxRail host naming scheme. The naming scheme is applied to all VxRail hosts.
- Reserve hostname for vCenter Server
- Reserve hostname for VxRail Manager
- Reserve hostname for SDDC Manger
- Reserve hostnames for NSX-T Management VIP and NSX-T appliance nodes.
|
Passwords | - Determine password structure following VMware password policy.
- Select passwords for VxRail management components.
- Select passwords for NSX-T Data Center.
- Select passwords for SDDC Manager.
|
Prepare Data Center Services |
Prepare DNS | - Configure forward and reverse DNS records for VxRail Manager.
- Configure forward and reverse DNS records for vCenter Server.
- Configure forward and reverse DNS records for all VxRail nodes.
- Configure forward and reverse DNS records for SDDC Manager.
- Configure forward and reverse DNS records for NSX-T Management Cluster.
|
Prepare DHCP | - Configure IP address scope for NSX-T host overlay network (unless using static IP addresses).
|
Prepare Active Directory | - If a use case for Cloud Foundation on VxRail include vRealize Suite to support a future VI workload domain, Active Directory must be deployed in the data center to support this requirement.
|
Prepare Leaf Switches | - Configure at least 1600 MTU (9000 preferred).
- Configure the required VLANs on the top-of-rack switches.
- Configure Layer 3 settings on VxRail external management network VLAN.
- Configure Layer 3 settings on NSX-T host overlay network.
- Configure Layer 3 settings on NSX-T edge overlay network (for NSX-T edge services).
- Configure the switch ports to be directly connected to the VxRail nodes as Layer 2 trunk ports.
- Configure unicast on the vSAN network.
- Configure multicast on the VxRail internal management network.
- Configure MLD snooping and MLD querier on the VxRail internal management network, unless manually assigning VxRail management IP addresses.
- Configure Spanning Tree on the switch ports supporting VxRail nodes as edge ports, or in ‘portfast’ mode.
- Configure inter-switch links on switches below the Layer 2/3 boundary.
|
Prepare Routing Services | - Configure Border Gateway Protocol at the Layer 2/3 network boundary.
- Configure BGP peering with NSX-T Tier-0 Gateway (for NSX-T edge services)
|
External Storage (if applicable) | - Configure LUN or LUNs on FC storage array
- Perform zoning and masking to present LUNs to VxRail nodes
|
Appendix B: Cloud Foundation on VxRail footprints for sizing
Use these tables to obtain footprint estimates of the resources for Cloud Foundation on VxRail.
Base Virtual Machines for every Cloud Foundation Management workload domain
Management | SDDC Manager | 4 | 19 | 800 |
Management | vCenter | 4 | 19 | 694 |
Management | NSX-T Manager 1 | 6 | 24 | 300 |
Management | NSX-T Manager 2 | 6 | 24 | 300 |
Management | NSX-T Manager 3 | 6 | 24 | 300 |
- A ‘Small’ vCenter server is adequate for the management domain, which will not scale in comparison to VI workload domains.
- NSX-T Manager and NSX-T edge devices can be deployed in three sizes: Small, Medium, Large. Cloud Builder deploys the ‘Medium’ sized NSX-T Manager virtual appliances into the management workload domain.
vCenter instance deployed in Cloud Foundation Management workload domain for each VxRail cluster deployed to support a Cloud Foundation VI workload domain. Default size is ‘Medium’ can manage up to 64 nodes.
Management | vCenter | 8 | 28 | 908 |
NSX-T edge gateways deployed in Cloud Foundation VI Workload Domain to support NSX-T networking. The default size is ‘Medium’.
Workload | NSX-T Edge 1 | 4 | 8 | 200 |
Workload | NSX-T Edge 2 | 4 | 8 | 200 |
Recommend using ‘Large’ size if load balancing is a requirement, or if NSX-T edge services will be shared with many other VI workload domains.
Workload | NSX-T Edge 1 | 8 | 32 | 200 |
Workload | NSX-T Edge 2 | 8 | 32 | 200 |
NSX-T Managers deployed in Cloud Foundation Management workload domain for each Cloud Foundation VI workload domain which does not use a shared NSX-T management instance. The default size is ‘Large’. This size will support more than 64 nodes.
Management | NSX-T Manager 1 | 12 | 48 | 300 |
Management | NSX-T Manager 2 | 12 | 48 | 300 |
Management | NSX-T Manager 3 | 12 | 48 | 300 |
For VI workload domains for Kubernetes, the same ‘Large’ size is deployed in Cloud Foundation Management workload domain for each Cloud Foundation VI workload domain which does not use a shared NSX‑T management instance.
Management | NSX-T Manager 1 | 12 | 48 | 300 |
Management | NSX-T Manager 2 | 12 | 48 | 300 |
Management | NSX-T Manager 3 | 12 | 48 | 300 |
A size of ‘Medium’ can be considered for deployments of fewer than 64 nodes.
Management | NSX-T Manager 1 | 6 | 24 | 300 |
Management | NSX-T Manager 2 | 6 | 24 | 300 |
Management | NSX-T Manager 3 | 6 | 24 | 300 |
NSX-T Global Managers deployed in one of the Cloud Foundation Management workload domains to support NSX-T Federation across regions. The size of the virtual machines depends on the size of the federation under management, either Medium or Large.
Management | NSX-T Global Manager 1 | 6/12 | 24/48 | 300 |
Management | NSX-T Global Manager 2 | 6/12 | 24/48 | 300 |
Management | NSX-T Global Manager 3 | 6/12 | 24/48 | 300 |
The following table lists the sizing to prepare for a vRealize Suite Lifecycle Manager download and deployment.
Note: Cloud Foundation on VxRail does not automate the deployment or the life cycle management of the other vRealize Suite components. vRealize Suite Lifecycle Manager is used to deploy and manage those components.
Management | vRealize Suite Lifecycle Manager | 2 | 6 | 78* 100** |
* Initial deployment
** Added for product binaries
The following table lists the sizing to prepare for the deployment of vSphere with Tanzu workload domain.
Workload | Supervisor Cluster control plane | 12 | 48 | 200 |
Workload | Registry Service | 7 | 7 | 200 |
Workload | NSX-T Edge 1 | 16 | 64 | 400 |
Workload | NSX-T Edge 2 | 16 | 64 | 400 |
Workload | Tanzu Kubernetes Cluster control plane | 6 | 12 | 48 |
Workload | Tanzu Kubernetes Cluster worker nodes | 6 | 12 | 48 |
Appendix C: Cloud Foundation on VxRail networks
Following are the core VLANs that must be configured on the data center switches supporting the Cloud Foundation on VxRail platform.
VxRail | External Management | VxRail and Cloud Foundation components | Yes | 1 per management component |
Internal Management | VxRail device discovery | No | N/A |
vMotion | Virtual machine mobility | Yes if routed vMotion is required | 1 per host |
vSAN | vSphere storage | Yes for stretched cluster | 1 per host |
NSX-T | Host Overlay | NSX-T host overlay network | Yes. Must route to Edge Overlay. | 2 per host |
NSX-T Edge | NSX-T Uplinks | 2 uplink VLANs for BGP peering of edge & physical switch | Enables BGP peering of edge gateway and physical switch | 2 per edge node |
NSX-T Edge Overlay | NSX-T edge overlay network | Yes. Must route to Host Overlay | 1 per edge node |
Node | iDRAC | PowerEdge out-of-band management | Yes | 1 per host |
Appendix D: VxRail network configuration
The following table lists the configuration settings required by VxRail Manager to deploy a VxRail cluster.
VxRail | Management Network | VxRail and Cloud Foundation management network subnet. Must be large enough for all VxRail and Cloud Foundation management components. |
External Management | VLAN ID for the management network that passes upstream from the top-of-rack switches |
Internal Management | VLAN ID for VxRail device discovery. This network stays isolated on the top-of-rack switches. The default VLAN ID is 3939. |
System | Global settings | Time zone |
NTP server(s) IP Address |
DNS server(s) IP Address |
Syslog Server |
Management | System-Generated ESXi hostnames | ESXi hostname prefix |
Separator |
Iterator |
Offset |
Suffix |
Domain |
| Customer-supplied Hostnames | Hostname 1 |
Hostname 2 |
Hostname 3 |
Hostname 4 |
ESXi IP Addresses | Can be individual IP addresses or in sequence |
| vCenter Server
| VxRail vCenter Server hostname |
VxRail vCenter Server IP address |
VxRail Manager | VxRail hostname |
VxRail IP address |
Networking | Subnet mask |
Gateway |
vMotion | | IP address pool for vMotion |
Gateway |
Subnet mask |
VLAN ID |
vSAN | | IP address pool for vSAN |
Subnet mask |
VLAN ID |
Dell Node | iDRAC | IP address for iDRAC port on each VxRail node |
The following table applies to stretched clusters only.
Witness | Management | Hostname |
IP Address |
Subnet Mask |
Gateway |
VSAN | IP Address |
Subnet Mask |
Gateway |
Witness Site | vSphere Host | IP Address |
Network | Witness Traffic Separation | Optional VLAN ID to manage traffic between two sites hosting VxRail nodes and witness site |
Second Site | vMotion | IP address pool for vMotion |
Subnet mask |
VLAN ID |
vSAN | IP address pool for vSAN |
Subnet mask |
VLAN ID |
VxLAN | IP address pool for VxLAN |
Subnet mask |
VLAN ID |
Appendix E: Cloud Builder and management VI workload configuration
This table lists the configuration settings that are required by Cloud Builder to deploy the Cloud Foundation management workload domain on the VxRail cluster platform.
Cloud Builder | IP Address | Temporary for Cloud Builder virtual appliance |
Global | NTP | IP Address |
DNS | IP Address |
SSO Site Name | Must be the same site name as used for the VxRail cluster. |
SSO Domain | |
DNS Zone Name | |
SDDC | Manager | Hostname |
IP Address |
Domain Name |
NSX-T | Manager (VIP) | Hostname |
IP Address |
Manager Node 1
| Hostname |
IP Address |
Manager Node 2 | Hostname |
IP Address |
Manager Node 3 | Hostname |
IP Address |
Appliance Size | (Small, Medium, Large) |
NSX-T Host Overlay Network | Static IP Assignment Method | Name of static IP address pool |
IP address range in CIDR format |
Starting IP address to be assigned for host overlay network |
Ending IP address to be assigned for host overlay network |
Gateway |
Dynamic IP Assignment Method | IP address of DHCP server to assign IP addresses to VTEP tunnel endpoints for host overlay network |
Range of IP addresses in DHCP server to be assigned to VTEP tunnel endpoints for host overlay network |
vSphere Objects | Data Center Name | Must match value in VxRail cluster |
Cluster Name | Must match value in VxRail cluster |
VxRail Distributed Switch Name(s) | Must match values used in VxRail cluster |
NSX-T Distributed Switch Name | If deploying separate VDS for NSX-T networking. VDS name must be unique in VxRail cluster. |
vSAN Datastore Name | Must match value used in VxRail cluster |
vSphere Resource Pools | SDDC Management | Required for consolidated architecture |
SDDC Edge | Required for consolidated architecture |
User Edge | Required for consolidated architecture |
User VM | Required for consolidated architecture |
Appendix F: VI workload domain configuration settings
This table lists the configuration settings required to support the configuration of a standard VI workload domain by SDDC Manager.
Global | Domain Name | |
Datacenter Name | Name of vSphere data center to be configured in vCenter instance supporting VI workload domain |
NTP | IP Address or pool URL |
DNS | IP Address |
SSO Site Name | |
SSO Domain | Can join management domain or a new domain |
NSX-T Host Overlay Network | Static IP Assignment Method | Name of static IP address pool |
IP address range in CIDR format |
Starting IP address to be assigned for host overlay network |
Ending IP address to be assigned for host overlay network |
Gateway |
Dynamic IP Assignment Method | IP address of DHCP server to assign IP addresses to VTEP tunnel endpoints for host overlay network |
Range of IP addresses in DHCP server to be assigned to VTEP tunnel endpoints for host overlay network |
A workload domain can either join an existing NSX-T instance or configure a new NSX-T instance. Use this table only if a new NSX-T management instance is deployed as part of the VI workload domain.
NSX-T ASN | Autonomous System Number | ASN for the workload domain edge cluster |
NSX-T Manager | NSX-T management cluster | IP address of NSX-T Manager VIP |
IP address for first NSX-T Manager |
IP address for second NSX-T Manager |
IP address for third NSX-T Manager |
Subnet Mask |
Default Gateway |
NSX-T Edge Node 1 | Name | Hostname of virtual appliance |
Management IP Address | Must be within workload domain management network subnet |
Uplink 1 IP Address | IP address for BGP peering on first NSX-T edge uplink VLAN for this workload domain |
Uplink 2 IP Address | IP address for BGP peering on second NSX-T edge uplink VLAN for this workload domain |
Overlay IP Address | IP address for overlay network between edge nodes in this workload domain |
NSX-T Edge Node 2 | Name | Hostname of virtual appliance |
Management IP Address | Must be within workload domain management network subnet |
Uplink 1 IP Address | IP address for BGP peering on first NSX-T edge uplink VLAN for this workload domain |
Uplink 2 IP Address | IP address for BGP peering on second NSX-T edge uplink VLAN for this workload domain |
Overlay IP Address | IP address for overlay network between edge nodes in this workload domain |
NSX-T Edge Uplink 1 | VLAN | Used for BGP peering with upstream routing services for this workload domain |
NSX-T Edge Uplink 2 | VLAN | Used for BGP peering with upstream routing services for this workload domain |
NSX-T Edge Overlay Network | VLAN | Used for edge overlay network connecting NSX-T edge nodes in this workload domain |
If a new edge cluster is deployed, use this table to capture the BGP neighbors for the NSX-T edge gateway.
External ASN | ASN Value | Autonomous System Number for external routers |
External Router 1 | IP Address | IP Address for peering with NSX-T edge gateway on first NSX-T edge uplink VLAN |
Password | Neighbor password for BGP peering |
External Router 2 | IP Address | IP Address for peering with NSX-T edge gateway on second NSX-T edge uplink VLAN |
Password | Neighbor password for BGP peering |
Use this table only if a vSphere for Tanzu supervisor cluster will be configured on the VI workload domain.
Pod CIDRs | Internal | Used by Kubernetes pods that run in the cluster |
Service CIDRs | Internal | Used by Kubernetes applications that need a service IP address |
Ingress CIDRs | External | Used for load balancing |
Egress CIDRs | External | Used for NAT endpoint use |
Appendix G: Edge Gateway configuration
This table lists the configuration settings required to support deployment of the NSX-T edge gateways
External ASN | ASN Value | Autonomous System Number for external routers |
External Router 1 | IP Address | IP Address for peering with NSX-T edge gateway on first NSX-T edge uplink VLAN |
Password | Neighbor password for BGP peering |
External Router 2 | IP Address | IP Address for peering with NSX-T edge gateway on second NSX-T edge uplink VLAN |
Password | Neighbor password for BGP peering |
NSX-T Edge Gateways |
Cluster | Name | Name of NSX-T Edge Cluster |
Internal ASN | ASN Value | BGP Autonomous System Number for NSX-T Edge Gateways |
Edge Node 1 | Name | Hostname of virtual appliance |
Management IP Address | Must be within management network subnet range |
Uplink 1 IP Address | IP address for BGP peering on first NSX-T edge uplink VLAN |
Uplink 2 IP Address | IP address for BGP peering on second NSX-T edge uplink VLAN |
Overlay IP Address | IP address for overlay network between edge nodes |
Edge Node 2 | Name | Hostname of virtual appliance |
Management IP Address | Must be within management network subnet range |
Uplink 1 IP Address | IP address for BGP peering on first NSX-T edge uplink VLAN |
Uplink 2 IP Address | IP address for BGP peering on second NSX-T edge uplink VLAN |
Overlay IP Address | IP address for overlay network between edge nodes |
Edge Gateway VLANs | Uplink 1 VLAN | First NSX-T edge uplink |
Uplink 2 VLAN | Second NSX-T edge uplink |
Edge Overlay VLAN | Used for edge overlay network connecting NSX-T edge nodes |
Second Site - External Routers |
External ASN | ASN Value | Autonomous System Number for external routers |
External Router 1 | IP Address | IP Address for peering with NSX-T edge gateway on first NSX-T edge uplink VLAN |
Password | Neighbor password for BGP peering |
External Router 2 | IP Address | IP Address for peering with NSX-T edge gateway on second NSX-T edge uplink VLAN |
Password | Neighbor password for BGP peering |
Appendix H: Application Virtual Network configuration
This table lists the configuration settings required to support deployment of the optional Application Virtual Network.
Region A (Local Instance) | Logical Segment | Name of Region A logical segment |
VLAN | VLAN for Region A network |
IP Addresses | IP address range for Region A network |
xRegion (Cross-Instance) | Logical Segment | Name of xRegion logical segment |
VLAN | VLAN for xRegion network |
IP Address | IP address range for xRegion network |
Appendix I: Sample switch configuration settings
This set of sample syntax is for providing basic guidance on the settings that must be performed on the top-of-rack switches to configure VLANs and a switch port for a Cloud Foundation on VxRail deployment, and configuring a switch support BGP peering for the Application Virtual Network (AVN). The actual code that is required on the top-of-rack switches depends on the existing data center network infrastructure, switch operating system and routing standards.
The sample syntax highlights the following required items:
- VLAN for VxRail external management
- VLAN fir VxRail internal management – node discovery
- VLAN for NSX-T host overlay network – with DHCP helper
- Switch port configuration for VxRail node
interface vlan <VxRail External Management>
no shutdown
ip address <gateway>/24
vrrp-group <id>
priority <priority>
virtual-address <virtual gateway>
interface vlan <VxRail Internal Management>
no shutdown
ipv6 mld snooping querier
interface <Host Overlay>
description
no shutdown
mtu 9216
ip address <gateway>/24
ip helper-address <DHCP server IP Address>
vrrp-group <id>
priority <priority>
virtual-address <virtual gateway>
interface ethernet <port>
no shutdown
switchport mode trunk
switchport access vlan <native vlan>
The sample syntax highlights the following required items:
- VLANs for AVN uplinks with assigned IP addresses
- Prefix list to permit route filtering for routes from the AVN
- BGP External ASN setting
- BGP peering with the pair of AVN Edge Service Gateways
interface vlan <VLAN for AVN Uplink>
no shutdown
mtu 9216
ip address <Gateway IP address for AVN uplink>
ip prefix-list <Router-ESGs route map name> permit <IP address range parameters>
router bgp <External ASN>
maximum-paths ebgp 4
router-id <External router ID>
address-family ipv4 unicast
redistribute connected route-map <Router-ESG route map name>
template external-router-to-ESG
advertisement-interval <value>
password <password saved to Edge Gateways>
timers 4 12
neighbor <IP address assigned to first Edge Gateway>
inherit template external-router-to-ESG
remote-as <ASN assigned to Edge Gateways>
no shutdown
neighbor <IP address assigned to second Edge Gateway>
inherit template external-router-to-ESG
remote-as <ASN assigned to Edge Gateways>
no shutdown
