Host bus adapter (HBA) ports (initiators) and storage ports (targets) are connected to an FC or Ethernet switch based on the connectivity requirements. FC connectivity requires that you create zones on the switch and define initiator and target relationships. The zones create an I/O path between the host and storage. Zoning and paths strongly affect performance aspects of the host and database.
FC connectivity best practices
Best practices for FC connectivity include:
- When zoning host initiators to storage target ports, ensure that each pair is on the same switch. Performance bottlenecks are often created when I/Os must travel through ISL (paths between switches), which are shared and limited.
- Use at least two HBAs for each database server to enable better availability and scale. Use multipathing software such as Dell EMC PowerPath or Microsoft Windows MPIO to balance loads and automatically failover or recover paths. Use commands such as powermt display paths or multipath -l to ensure that all paths are visible and active.
- Consider port speed and count when planning bandwidth requirements. Each 8 Gb FC port can deliver up to about 800 MB/sec. Therefore, a server with four ports cannot deliver more than about 3 GB/sec. Also consider that among the host initiator, storage port, and switch, the lowest speed supported by any of these components is negotiated and used for that path.
- Consider the number of paths that are available to the database storage devices. Each path between the host and storage ports creates additional SCSI representation for the database devices on the host.
- While more paths add I/O queues and the potential for more concurrency and performance, consider that server boot time is affected by the number of discovered SCSI devices (one for each path combination per device, plus a pseudo device). In addition, after connectivity needs are satisfied for performance and availability, additional paths do not add more value and only add more SCSI representations to the host.
- In most cases, for availability and performance, it is sufficient to have each HBA port zoned/masked to two or four PowerMax ports, preferably on different engines and directors.
iSCSI connectivity best practices
- Use VLANs dedicated to iSCSI setup. VLANs allow logical grouping of network endpoints, which minimize network bandwidth contention for iSCSI traffic and eliminate impact on iSCSI traffic due to noisy neighbors.
- If all network devices in the iSCSI communication paths support jumbo frames, using jumbo frames on Ethernet improves iSCSI performance.
- To minimize host CPU impact due to network traffic, ensure that Transmission Control Protocol (TCP) offloading is enabled on a host network interface card (NIC), which offloads processing of the TCP stack to the NIC and eases impact on the CPU.
- As with FC connectivity, using PowerPath software or native multipathing for Windows helps with load balancing and eases queuing issues for iSCSI traffic through the host NICs.
Number and size of host devices
- PowerMax systems use thin devices exclusively. Therefore, on creation, devices do not fully allocate their capacity in the SRP, and they only consume as much capacity as the application actually writes to them. For example, by default, a 1 TB device that has not been written to does not consume any storage capacity. This approach enables capacity savings because storage is consumed based on demand and not during device creation. However, if certain applications require a guarantee for their capacity, their devices can be fully created and allocated in the SRP. New devices can be created easily using Unisphere for PowerMax or by using CLI, as in the following example:
PS > symdev create -tdev -cap 500 -captype gb -N 4 -v # create 4 x 500GB thin devices
- PowerMax host devices are natively striped at 128 KB across the data pools in the SRP. Although you can create only a few very large host devices, consider these factors:
- As discussed previously, each path to a device creates a SCSI representation on the host. Each such representation provides a host I/O queue for that path. Each such queue can service a tunable, but limited (often 32), number of I/Os simultaneously. Provide enough database devices for concurrency (multiple I/O queues), but not so many that management overhead would be increased.
- Another benefit of using multiple host devices is that internally the storage array can use more parallelism for operations such as data movement and local or remote replications. By performing more copy operations simultaneously, the overall operation takes less time.
- While the size and number of host devices can vary, we recommend finding a reasonable, low number that offers enough concurrency, provides an adequate building block for capacity increments when additional storage is needed, and does not become too large to manage. For example, four or eight devices, sized at 250 GB to 1 TB (choose one size) can be a good design starting point for databases from 1 to 8 TB, if enough connectivity/concurrency exists.
Physical host connectivity best practices
For physical host connectivity, consider the number and speed of the HBA ports (initiators) and the number and size of host devices:
- HBA ports—Each HBA port (initiator) creates a path for I/Os between the host and the SAN switch and then to the PowerMax storage. If a host uses a single HBA port, it has a single I/O path that must serve all I/Os. Such a design is not advisable because a single path does not provide HA and risks a potential bottleneck during high I/O activity due to the unavailability of additional ports for load balancing.
For a better design, provide each database server with at least two HBA ports, preferably on two separate HBAs. The additional ports provide more connectivity and enable multipathing software such as Dell EMC PowerPath or Microsoft Multipath I/O (MPIO) to balance loads and to fail over across HBA paths.
Each path between the host and storage device creates a SCSI device representation on the host. For example, two HBA ports connected to two PowerMax front-end adapter ports with a 1:1 relationship create three presentations for each host device. One port is used for each path, and the multipathing software uses the other port to create a Dell EMC Symmetrix Multi-Path Disk Device (PowerPath System Devices). If each HBA port was zoned and masked to both FA ports (1: many relationship) there would be five SCSI device representations for each host device (one for each path combination plus pseudo device).
While modern operating systems can manage hundreds of devices, it is not advisable or necessary, and it burdens the user with complex tracking and storage provisioning management overhead. We recommend that you:
- Establish enough HBA ports to support workload concurrency, availability, and throughput
- Use 1:1 relationship for storage front-end ports
- Do not zone or mask each HBA port to all PowerMax front-end ports
Following these suggestions will provide enough connectivity, availability, and concurrency, while reducing unnecessary complexity.
- Number and size of host devices—PowerMax can create host devices with capacity ranging from a few megabytes to multiple terabytes. With the native striping across the data pools that PowerMax provides, the DBA might be tempted to create only a few very large host devices. For example, a 1 TB Microsoft SQL Server database can reside on one 1 TB host device, or perhaps on ten 100 GB host devices; while either option satisfies the capacity requirement, you should use a reasonable number of host devices of appropriate size. In this example, if the database capacity was to rise above 1 TB, the DBA might want to add another device of the same capacity, even if 2 TB was not currently needed. Therefore, large host devices create very large building blocks when additional storage is needed.
Each host device also creates its own host I/O queue for the host operating system. Each such queue can service a tunable, but limited, number of I/Os that can be transmitted simultaneously. If, for example, the host has four HBA ports and a single 1 TB LUN with multipathing software, it will have only four paths available to queue I/Os. A high level of database activity generates more I/Os than the queues can service, resulting in artificially elongated latencies. In this example, two or more host devices are advisable to alleviate such an artificial bottleneck. Host software such as Dell EMC PowerPath or Windows PerfMon can help in monitoring host I/O queues to ensure that the number of devices and paths is adequate for the workload.
Another benefit of using multiple host devices is that, internally, the storage array can use more parallelism when operations such as FAST data movement or local and remote replications take place. By performing more copy operations simultaneously, the overall operation takes less time.
While there is no one magic number for the size and number of host devices, we recommend finding a reasonably low number that offers enough concurrency, provides an adequate building block for capacity when additional storage is needed, and does not become too large to manage.
Host I/O limits and multi-tenancy
The host I/O limits QoS feature was introduced in VMAX arrays, and it continues to provide the option to place specific IOPS or bandwidth limits on any SG. For example, assigning a specific host I/O limit for IOPS to an SG with low performance requirements can ensure that a spike in I/O demand will not saturate or overload the storage, affecting performance of more critical applications. As shown by test cases later in this paper, even though PowerMax systems can maintain high performance at low latency, using a host I/O limit may limit the impact of noisy neighbors on mission-critical applications.