Virtual Database Environment Setup

Prerequisite

Before you set up the ESXi database server, ensure that the vCenter Server Appliance VM is installed and set up on the management server. This setup is required to connect to the vSphere Web Client to perform the tasks in this section.

Introduction

To set up the ESXi database server, follow the procedures in this section to:

1. Install and configure ESXi.
2. Create a vSphere data center.
3. Add the ESXi database host to the vSphere data center.

4. Configure a vSphere Distributed Switch.
5. Configure datastores.
6. Create VMs for the databases.
7. Set up networking for ESXi hosts and VMs.
8. Prepare storage disks for databases.

Installing and configuring ESXi on the database server

On the R940-based virtual database server, install ESXi 6.7 U3 by repeating the procedures in Installing ESXi and deploying vCenter Server Appliance on the management server. However, disregard the section about deploying vCenter Server Appliance because it has already been installed. Installing vCenter Server Appliance again on the virtual database server is unnecessary.

Additional best practices in an XtremIO environment

When ESXi 6.7 U3 was installed on the database server, we applied the following applicable best practices for our virtual environment.

Note: In our test environment, we applied best practices in accordance with the information in the Dell EMC XtremIO Storage Array Host Configuration Guide. See that guide for the complete list of best practices for XtremIO in a VMware ESXi host environment and for details about how to apply them.

HBA queue depth settings

The following table lists the default and recommended HBA queue depth settings for ESXi 6.7 U3 hosts connecting to XtremIO X2 storage arrays:

Table 29.        HBA queue depth settings in ESXi based hosts

We set the LUN queue depth to 256, the recommended value for the QLogic HBAs that we used in our virtualized database server. This value ensures that the XtremIO X2 storage arrays handle an optimal number of SCSI commands (including I/O requests).

Note: vSphere no longer reads the QLogic HBA Queue Depth setting so the setting is not relevant when you are configuring a vSphere host with QLogic HBAs.

Multipath configuration

Configure the virtual database ESXi host by using vSphere Native Multipathing (NMP) and the following parameter values on the host for optimal performance with the XtremIO X2 array:

• Set the native path selection policy on the XtremIO volumes that are presented to the ESXi hosts to round-robin.
• Set the vSphere NMP round-robin path switching frequency to XtremIO volumes from the default value (1,000 I/O packets) to 1.

Note: In ESXi 6.7 U3, the default path selection policy is round-robin and the default path switching frequency is 1. Therefore, no change was needed in our virtualized database server.

Host parameter settings

Configure the following ESXi host parameters:

• Disk.SchedNumReqOutstanding—Determines the maximum number of active storage commands (I/O) that are allowed at any given time at the VMkernel. For each XtremIO volume that is presented to the ESXi host, set this parameter to the recommended value of 256 by running the following CLI commands on the ESXi 6.7 U3 host:
  1. To get the list of all XtremIO volumes:

$> esxcli storage nmp path list | grep XtremIO -B1 | grep "\ naa" | sort | uniq

2. To set the value for each volume:

$> esxcli storage core device set -d <naa.xxx> -O 256

where <naa.xxx> is the XtremIO volume that is obtained from the preceding command.

• Disk.SchedQuantum—Determines the maximum consecutive sequential I/O that is allowed before switching from one VM to another VM. Change this value to 64.

Creating a vSphere data center

To create the data center and cluster containers inside vSphere:

1. Open a web browser and connect to the vSphere Web Client as follows, using the vCenter Server Appliance IP address:

https://<vCenter Server Administrator FQDN or IP>/vsphere-client

2. Log in with an account that has administrator privileges.
3. Go to Home > Inventory > Hosts and Clusters.
4. On the navigator menu, right-click the top-level vCenter Server Administrator object and click New Datacenter.
5. Enter the name for the data center and click OK.

Adding hosts to vCenter

After you create a vSphere data center, add the hosts to vCenter by performing the following steps on each of the servers that will be part of the data center:

1. Open a web browser, and open the vSphere Web Client:

https://<vCenter Server Administrator FQDN or IP>/vsphere-client

2. Log in with an account that has administrator privileges.
3. Go to Home > Inventory > Hosts and Clusters.
4. Right-click the data center object and select Add host.
5. Enter the DNS name or IP address that is assigned to the virtual database server and complete the remainder of the wizard.

Configuring vSphere Distributed Switch

VDS provides centralized VM network administration. In VDS, virtual switches of each ESXi host are abstracted into a large pool and spread across multiple ESXi hosts within the data center.

Note: Typically, for single ESXi host implementations, standard switches and port groups are sufficient. In this solution, even though we used only one ESXi host, we created distributed switches and port groups to provide the means to easily expand the environment by using multiple ESXi hosts, if needed.

To configure a VDS:

1. Create the distributed switch.

                  Create corresponding port groups.

The following table describes the distributed switches and port groups for the basic virtual network infrastructure for the databases:

Table 30.        Distributed switches and port groups

* A vMotion network is used for a solution with multiple ESXi hosts and is not required for this solution. If you add more ESXi hosts later, then you can use vMotion to migrate the VMs among the hosts.

Note: The uplink port group is created by default when you create the distributed switch.

Creating distributed switches

Create a VDS on VMware vSphere 6.7 U3 or later as follows:

1. In the vSphere Web Client navigator, right-click the data center name and select Distributed Switch > New Distributed Switch.

2. At Name and location, enter a name for the VDS (DBPub) and click Next.
3. At Select version, select Distributed Switch 6.7.0 and click Next.
4. At Edit settings:
   1. Select 4 for Number Of Uplinks.

Uplink ports connect VDS to the physical NICs on associated hosts.

2. Set Network I/O Control to Enabled.

Network I/O Control monitors the load over the network and dynamically allocates resources.

3. Select Create a default port group, and then click Next.
4. At the Ready to complete page, review the settings and click Finish.

Creating port groups

After you create distributed switches, create port groups:

1. Right-click the VDS (DBPub), and then select New Distributed Port Group.
   1. At Select name and location, enter a name for the port group (DBPublic) and click Next.
   2. At Configure settings:

i             For VLAN type, select VLAN.

ii            For VLAN ID, select the ID.

Note: We set VLAN ID 16 for DBPublic (database public and backup and recovery traffic) and VLAN ID 99 for vMotion distributed port groups. The two uplink ports that are shared by DBpublic and vMotion distributed port groups were also tagged with the same VLAN ID 16 and 99 on the two ToR S5248F-ON switches to which they connect.

iii          Under Advanced, select Customize default policies configuration.

iv          Click Next.

3. At Ready to complete, review the summary and click Finish.

2. Repeat the preceding step to create the following additional port group:

vMotion on DBPub

Configuring datastores

VMFS datastores are repositories for VMs. You can set up VMFS datastores for iSCSI-based storage, FC-based storage, and local storage.

Note: First install and configure any required adapters and rescan the adapters to discover newly added storage devices.

Create datastores in vSphere 6.7 U3:

1. In the vSphere Web Client, go to Home > Storage > Datastores.

2. Click the Create a new datastore icon, as shown in the following figure, and then create datastore VM-OS:

Figure 16.     Creating a datastore

1. At Type, select VMFS.
2. At Name and device selection, for Datastore name type VM-OS, scroll to select the XtremIO volume that you created for VM-OS, and then click Next.
3. Review the summary, and then click Finish to create the datastore.

3. Repeat the preceding steps to create additional datastores.[3]

The following table provides the details of the datastore design in the virtual database environment:

Table 31.        vSphere VM datastore design of XtremIO X2 storage volumes for the virtualized databases

Creating VMs for databases

Create the two database VMs:

1. In vSphere Web Client, create a VM (VM1) on your ESXi host, as shown in the following figure:

Figure 17.     Creating a VM

2. Select your ISO image, and configure the VM.

The following table shows the specific guest operating system and VM settings for both VMs. We left all values set to the default except the values that are specified in the table.

Table 32.        VM settings for virtual databases

Setting up networking for ESXi hosts and VMs

ESXi networking provides communication between VMs on the same host and on different hosts, and between other virtual and physical hosts. It also manages ESXi hosts and communicates between VMkernel services and the physical network.

To set up networking:

1. Add the cluster hosts to VDS.

2. Add network adapters to VMs.

Adding cluster hosts to VDS

1. On the vSphere Web Client home page, click Networking.
2. Right-click DBPub VDS and click Add and Manage Hosts.
   1. Select Add hosts, and then click Next.
   2. Click New hosts and select the hosts to add.
   3. Click OK and Next.
   4. Ensure that Manage physical adapters and Manage VMkernel adapters are selected, and then click Next.
   5. On the ESXi host, select the 10 GbE vmnic that is reserved for public and vMotion traffic and assign it to uplink 1, as shown in the following figure, and click OK:

Figure 18.     Assigning an uplink to an adapter

6. Repeat the preceding step to assign the other 10 GbE vmnic that is reserved for public and vMotion traffic to uplink2.
7. Right-click a host and click New Adapter to add a VMkernel port for vMotion.

i             Select Select an existing network and click Browse to select the vMotion port group, as shown in the following figure:

Figure 19.     Selecting the port group

ii            Click OK, and then click Next.

iii          At Port group properties, select VMotion traffic and click Next.

iv          Select Use Static IPV4 settings, provide the details, and then click Finish.

Adding network adapters to VMs

After adding cluster hosts to VDS, add network adapters to VMs:

1. In the vSphere Web Client, right-click the VM and select Edit Settings.

2. From the New device list box, select Network and click Add.

The new network adapter appears at the bottom of the list.

3. Expand the new network, and change the following settings:
   1. For the Status setting:

• Connected—Select this option when the VM is running to connect or disconnect the VM network adapter. This option is not available when the VM is not running.
• Connect at power on—Select this option to connect the virtual network adapter to the network when the VM is on.
  2. From the Adapter Type list box, select the VMXNET 3 option.
  3. Optionally, select how to assign MAC addresses—Automatic or Manual. Automatic is preferred.
  4. From the New Network type list box, select the appropriate standard or distributed port group that is based on the usage of this adapter.

Choose the DBPublic distributed port group that was created for the database public traffic.

4. Click OK to add the network adapters.

Preparing storage disks for stand-alone databases

Set up volumes and hard disks for VM1:

1. In the vSphere Web Client, go to the host, right-click the VM, and select Edit settings.
2. Add a SCSI controller 1 with the following settings:

Figure 20.     New SCSI controller settings

3. Create two additional SCSI controllers (2 and 3) with the same controller settings that are shown in the preceding step.
4. Add hard disks for VM1:
   1. Select New Hard Disk.
   2. Select Add.
   3. Expand New hard disk and browse to the datastore location.
   4. Set the disk size to Max allowed.
   5. Set disk provisioning to Thick Provision eager Zeroed.
   6. Set sharing to No Sharing.
   7. Select the SCSI controller depending on the purpose of the hard disk, as shown in the following table:

Table 33.    SCSI controller and hard disk mappings in each virtual database VM

8. Repeat steps a through g to add more hard disks for this VM, with the sizes that are specified in Table 31.

Set up volumes and hard disks for VM2:

1. Repeat steps 1 through 3 in the previous section to create SCSI controllers for your VM by using the same design as provided in Table 33.
2. Add hard disks for VM2[4]:
   1. Select New Hard Disk.
   2. Select Add.
   3. Expand New Hard disk and browse to the Datastore location (the same datastore as used for VM1).
   4. Set disk size to Max allowed.
   5. Set disk provisioning to Thick Provision eager Zeroed.
   6. Set sharing to No Sharing.
   7. Select the SCSI controller depending on the purpose of the hard disk, as shown in Table 33.
   8. Repeat steps a through g to add more hard disks for this VM, with the sizes that are specified in Table 31.