Oracle ASM for Oracle single instance or RAC

Oracle ASM is the preferred storage management solution for either single instance database or RAC. Oracle ASM takes place of the traditional Linux volume manager and file system. When proper ownership and permission are set on the disks, ASM takes over the management of the disks and create disk groups where data files reside and are managed automatically. Using Oracle ASM also improves overall performance as it stripes all files across all disks evenly. Another advantage of ASM is that disks can be added or removed dynamically without any downtime, and files are restriped across the remaining disks automatically to ensure optimal performance. ASM also has many advantages over the Logical Volume Manager (LVM) file-system solution (discussed in File system for Oracle single instance database) and it should be considered as the preferred choice whenever possible.

Some of the benefits of ASM include:

• Automatic file management
• Online data files rebalance across disk volumes
• Online addition and removal of disk volumes
• Single solution for both volume and file management

The most common way to consume XC Series storage is to mount it as a datastore in the VMware hypervisor through NFS. Virtual disks are then created through the regular vSphere client and added to the Linux guest OS virtual machines. Oracle ASM can take control of these virtual disks just like any physical disks. Thus, no extra configuration steps are required to make XC Series storage work with ASM.

The following Oracle ASM best practices are recommended:

• Use Linux device management facility, udev, to assign ownership and access permissions persistently on disk volumes. Without proper permission, ASM cannot manage and control the disk volumes. For example, the udev rule specifies a disk volume with the device id/WWN, 36000c294a883a308becef0cd8448725a, creates a symlink in /dev/asmdatadg1, and assigns grid:oinstall and 0660 permission to the device file.

/etc/udev/rules.d/99-oracle-asm.rules:

KERNEL=="sd*", PROGRAM="/lib/udev/scsi_id --page=0x83 --whitelisted --device=/dev/%k",RESULT=="36000c294a883a308becef0cd8448725a", SYMLINK+="asmdatadg1", OWNER:="grid", GROUP:="oinstall",MODE="0660"

To find out how to query the disk id/WWN, refer to Appendix A: How to identify and query the disk ids/WWNs.

• Oracle supports asynchronous I/O by default. The disk_asynch_io database parameter is set to true. Oracle recommends using the deadline I/O scheduler for Oracle ASM. Oracle UEK kernels have the deadline I/O scheduler as default. The Nutanix recommendation is to use the deadline I/O scheduler for hybrid configurations consisting of SSDs and HDDs. If the default disk I/O scheduler is not deadline, a udev rule file can be created to set it. The following shows an example of the rule.

60-disk-scheduler.rules:

ACTION=="add|change", KERNEL=="sd*", RUN+="/bin/sh -c '/bin/echo deadline > /sys$env{DEVPATH}/queue/scheduler'"

For all-flash configurations, Nutanix recommends the NOOP I/O scheduler because it can reduce I/O latency, improve throughput, and help reduce CPU cycles. The following shows an example of the udev rule to set IO scheduler as NOOP.

60-disk-scheduler.rules:

ACTION=="add|change", KERNEL=="sd*", RUN+="/bin/sh -c '/bin/echo noop > /sys$env{DEVPATH}/queue/scheduler'

• Although ASM supports volumes of different sizes in the same disk group, to ensure best storage performance, all volumes should have the same space capacity and performance characteristics.
• A minimal of two disk groups should be created: one for data files and one for a fast recovery area.
• If Oracle Grid Infrastructure will be installed for Oracle RAC, create a separate disk group to hold the crs/voting disks.
• Oracle ASM offers several levels of data redundancy to increase data availability. In High Redundancy, ASM creates two mirror copies, and in Normal Redundancy mode, ASM creates one mirror copy. In External Redundancy mode, there is only one copy of the data. Since XC Series storage already offers data protection and HA by creating two replicas of the data on the storage level by default, it is sufficient to choose External Redundancy for the ASM disk groups. However, customers can choose other redundancy options to increase the level of protection, but must make sure to factor in the additional storage required by using one of these redundancy modes.
• It is recommended to keep the ASM allocation unit size as 1MB which is the default value for ASM disk groups.

Table 3 shows an example of ASM disk group configuration:

Table 3.              Example ASM disk group configuration

File system for Oracle single instance database

Oracle also supports storing data files on modern file systems such as ext3/ext4. xfs is a high-performance file system many Linux distributions include as default option. However, as of this time, Oracle has not certified or tested Oracle databases running on xfs. Therefore, it should not be used for storing data and log files. For performance reasons, Linux LVM should be considered for any file systems that hold the database files. LVM provides the capability to distribute files across multiple disks similar to Oracle ASM. The use of file system with LVM is recommended for a single-instance configuration only. If shared disks are required such as Oracle RAC, use Oracle ASM instead. The following guidelines should be followed:

• Nutanix recommends that a volume group consist of at least four to six volumes; more volumes can be added later based on the capacity requirements.
• All volumes should have the same space capacity and performance characteristics. The maximum size available to the database is restrained by the smallest volume multiplied by the number of volumes in the volume group.
• Create two volume groups and one logical volume in each volume group: one for database files and one for archived logs.
• Striping the logical volumes can significantly improve the I/O performance.
• Create an ext4 file system on the logical volumes. Use of ext4 file system is recommended at this time. For a complete list of supported file systems, refer to Oracle KB article 1601759.1.
• Oracle supports asynchronous I/O by default. The disk_asynch_io database parameter is set to true.
• Set the database parameter filesystemio_options = setall to instruct Oracle to use asynchronous I/O and direct I/O when the underlying file system supports this.
• Using Oracle Managed Files is recommended to simply the administration and management of database files on file systems.
• The Oracle software binaries should be installed into a separate file system, separated from the root file system and the database file systems. 

Table 4 shows an example of LVM configuration.

Table 4.              Example LVM configuration