This section discusses common storage practices for Oracle RAC database.

Oracle ASM for RAC database

Oracle Automatic Storage Management (ASM) is the recommended volume manager which can be used for both Oracle RAC and Single Instance Oracle Databases. Oracle ASM simplifies storage management through the principle of “stripe and mirror everything” (SAME). Intelligent mirroring capabilities enable administrators to define 2- or 3-way mirrors to protect vital data. But this is not needed when using enterprise storage solutions like PowerFlex. ASM takes over the management of the disks and creates disk groups where datafile reside and are managed automatically. ASM is highly integrated with, and highly optimized for, the Oracle Database.

Some of the benefits of ASM include:

• Automatic file management
• Stripes files rather than logical volumes
• Enables online disk reconfiguration and dynamic rebalancing
• Provides adjustable rebalancing speed
• Choice of external (array-based) data protection, two-way, and three-way mirror protection

The most common way to consume PowerFlex storage is to mount as a VMFS datastore in the ESXi hypervisor. Create multiple VMFS datastores for different types of Oracle database files (datafiles, redo logfiles, archive logs, OCR files) using VMware vSphere web client. Create Shared VMDKs using multiwriter option and attach the same type of virtual drives to Oracle RAC database VMs. Oracle ASM takes control of these virtual drives like any raw devices. Thus, no extra configuration steps are required to make PowerFlex storage work with ASM.

Oracle ASM best practices:

• Configure ASM disk groups for external redundancy. In External Redundancy mode, there is only one copy of the data. PowerFlex maintains the user data in a RAID-1 mesh mirrored layout. Each piece of data is stored on two different servers, it is enough to choose External Redundancy for the ASM disk groups.
• Use a 1 MB ASM allocation unit (AU) size for ASM disk groups. ASM disks are divided into Allocation Units. Data within a file residing on ASM is broken up into extents. As the AU size increases, the extent count decreases, reducing the number of physical IOs required when performing IO operations. For OLTP, 1- 4 MB is enough.
• Use different ASM disk groups for OCR, database files, redo log, and archive log storage
• ASM disk groups should contain uniform-sized disks, so the amount of data on each disk is approximately the same. This uniform-sized disk ensures in reducing I/O bottlenecks within a disk group and rebalance is faster whenever the storage configuration of a disk group changes such as when a disk is added.
• Oracle supports asynchronous I/O by default. The disk_asynch_io database parameter is set to true and filesystemio_options parameter be set to setall. For all-flash configurations, Dell EMC recommends the NOOP I/O scheduler because it can reduce I/O latency, improve throughput, and help reduce CPU cycles. The following shows an example of the udev rule to set IO scheduler as NOOP.

echo noop > /sys/block/$device/queue/scheduler

• Set the Linux maximum I/O size (max_sectors_kb) to match ASM AU size in rc.local or using UDEV.

echo 1024 > /sys/block/$device/queue/max_sectors_kb

• Use Linux device management facility, udev, to assign ownership and access permissions persistently on disk volumes. Without proper permission, ASM cannot manage and control the disk volumes. For example, the udev rule specifies a disk volume with the device id/WWN, creates a symlink and assigns grid:oinstall and 0660 permission to the device file.

/etc/udev/rules.d/99-oracle-asm.rules:

KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM="/lib/udev/scsi_id --page=0x83 --whitelisted --device=/dev/%k", RESULT=="36000c29f253d0fb44c9c59b18dc95a94",

SYMLINK+="oracleasm/oradata01", OWNER="grid", GROUP="asmadmin", MODE="0660"

Table 4.              ASM disk group configuration