Home > Storage > PowerScale (Isilon) > Product Documentation > Protocols > Dell Multipath Client Driver: Installation and Configuration Guide > Specific Linux version build instructions
While the previous procedure describes the general driver acquisition, build, and install process, there are some idiosyncrasies and variations to be aware of for the different Linux distributions. The following section covers the specific caveats and additional steps required for each Linux version:
Ubuntu 18.04 or greater can use the "dellnfs-ctl" script to reload the NFS client modules as follows:
# dellnfs-ctl reload
Ubuntu 22.x clients should also have the ‘nfs-kernel-server’ package installed. For example, on an Ubuntu system:
# sudo apt install nfs-kernel-server
# sudo apt-get install ./dist/dellnfs-modules_*-generic_amd64.deb
# sudo update-initramfs -u -k `uname -r`
# dpkg -l | grep dellnfs-modules
dellnfs-modules 2:4.0.22-dell.kver.5.4.0-150-generic amd64 NFS RDMA kernel modules
# dpkg -S /lib/modules/`uname -r`/updates/bundle/net/sunrpc/xprtrdma/rpcrdma.ko
dellnfs-modules: /lib/modules/5.4.0-150-generic/updates/bundle/net/sunrpc/xprtrdma/rpcrdma.ko
# reboot
# dellnfs-ctl status
version: 4.0.22-dell
kernel modules: sunrpc rpcrdma compat_nfs_ssc lockd nfs_acl nfs nfsv3
services: rpcbind.socket rpcbind rpc-gssdrpc_pipefs: /run/rpc_pipefs
The Mellanox OpenFabrics Enterprise Driver for Linux (MLNX_OFED) is a single Virtual Protocol Interconnect (VPI) software stack that operates across the network adapters in DGX systems. MLNX_OFED is an NVIDIA-tested and packaged version of OFED which supports Ethernet, as well as Infiniband, using RDMA and kernel bypass APIs (OFED verbs).
To build the multipath driver on an NVIDA DGX platform or alongside a DKMS, install of the Mellanox OpenFabrics Enterprise Distribution (MLNX_OFED) driver package. There are a few extra steps required beyond the package manager install itself.
If a DKMS install is required by your system (typically NVDIA DGX platforms), the package will be formatted with the term ‘multikernel’ in the package name. For example:
# ls
dellnfs-dkms_4.5-OFED.4.5.1.0.1.1.gb4fdfac.multikel_all.deb
This indicates the package is built for DKMS and therefore must be installed by DKMS. After installing with your package manager, the following files will be present under the /usr/src directory:
# ls /usr/src
dellnfs-99.4.5 kernel-mft-dkms-4.22.1 linux-hwe-5.19-headers-5.19.0-45 mlnx-ofed-kernel-5.8 srp-5.8
iser-5.8 knem-1.1.4.90mlnx3 mlnx-nfsrdma-5.8 ofa_kernel
isert-5.8 linux-headers-5.19.0-45-generic mlnx-nvme-5.8 ofa_kernel-5.8
Next, the following CLI command will install the driver:
# dkms install -m dellnfs -v 99.4.5
Creating symlink /var/lib/dkms/dellnfs/99.4.5/source ->
/usr/src/dellnfs-99.4.5
Kernel preparation unnecessary for this kernel. Skipping...
Building module:
cleaning build area...
./_dkms-run.sh -j8 KVER=5.19.0-45-generic
K_BUILD=/lib/modules/5.19.0-45-generic/build......................................
Once the DKMS installation is complete, either reload the driver with the ‘dellnfs-ctl’ utility, or reboot the client:
# dellnfs-ctl reload Or: # reboot |
Building and installing the multipath driver on a physical (bare metal) client (true install) is a straightforward process.
# zypper in ./dist/dellnfs-4.0.22-kernel_5.14.21_150400.24.97_default.x86_64.rpm
Loading repository data...
Reading installed packages...
Resolving package dependencies...
The following NEW package is going to be installed:
dellnfs
1 new package to install.
# rpm -qa | grep dell
dellnfs-4.0.22-kernel_5.14.21_150400.24.100_default.x86_64
# reboot
# systemctl start portmap
# systemctl start nfs
# systemctl status nfs
nfs.service - Alias for NFS client
Loaded: loaded (/usr/lib/systemd/system/nfs.service; disabled; vendor preset: disabled)
Active: active (exited) since Wed 2023-12-13 15:11:09 PST; 2s ago
Process: 15577 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
Main PID: 15577 (code=exited, status=0/SUCCESS)
Dec 13 15:11:09 CLI22 systemd[1]: Starting Alias for NFS client...
Dec 13 15:11:09 CLI22 systemd[1]: Finished Alias for NFS client.
# dellnfs-ctl status
version: 4.0.22
kernel modules: sunrpc
services: rpcbind.socket rpcbind
rpc_pipefs: /var/lib/nfs/rpc_pipefs
However, when building and installing on an OpenSUSE virtual instance (VM), additional steps are required. OpenSUSE does not reliably install a kernel-devel kit that matches the running kernel version, so this must be forced to happen.
# zypper install rpmbuild tar gzip git
The recommended way to install kernel-devel package according to OpenSUSE documentation is to use:
# zypper install kernel-default-devel
Warning: The ‘zypper install kernel-default-devel’ command occasionally fails to install the correct kernel-devel package.
This can be verified by looking at the following paths:
# ls /lib/modules/
5.14.21-150500.55.39-default
# uname -a
Linux 6f8edb8b881a 5.15.0-91-generic #101~20.04.1-Ubuntu SMP Thu Nov 16 14:22:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Note that the contents of /lib/modules above does not match the ‘uname’ command output:
‘5.14.21-150500.55.39-default’ vs. ‘5.15.0-91-generic’
Another issue with installing with ‘kernel-devel’ is that sometimes the /lib/modules/$(uname -r) directory will not include the /build subdirectory.
If this occurs, the client side driver will fail with the following error:
# ls -alh /lib/modules/$(uname -r)/build
ls: cannot access '/lib/modules/5.14.21-150400.24.63-default/build': No such file or directory
....
Kernel root not found
The recommendation is to install the specific kernel-devel package for the client’s Linux version. For example:
# ls -alh /lib/modules/$(uname -r)
total 5.4M
drwxr-xr-x 1 root root 488 Dec 13 19:37 .
drwxr-xr-x 1 root root 164 Dec 13 19:42 ..
drwxr-xr-x 1 root root 94 May 3 2023 kernel
drwxr-xr-x 1 root root 60 Dec 13 19:37 mfe_aac
-rw-r--r-- 1 root root 1.2M May 9 2023 modules.alias
-rw-r--r-- 1 root root 1.2M May 9 2023 modules.alias.bin
-rw-r--r-- 1 root root 6.4K May 3 2023 modules.builtin
-rw-r--r-- 1 root root 17K May 9 2023 modules.builtin.alias.bin
-rw-r--r-- 1 root root 8.2K May 9 2023 modules.builtin.bin
-rw-r--r-- 1 root root 49K May 3 2023 modules.builtin.modinfo
-rw-r--r-- 1 root root 610K May 9 2023 modules.dep
-rw-r--r-- 1 root root 809K May 9 2023 modules.dep.bin
-rw-r--r-- 1 root root 455 May 9 2023 modules.devname
-rw-r--r-- 1 root root 802 May 3 2023 modules.fips
-rw-r--r-- 1 root root 181K May 3 2023 modules.order
-rw-r--r-- 1 root root 1.2K May 9 2023 modules.softdep
-rw-r--r-- 1 root root 610K May 9 2023 modules.symbols
-rw-r--r-- 1 root root 740K May 9 2023 modules.symbols.bin
drwxr-xr-x 1 root root 36 May 9 2023 vdso
# rpm -qf /lib/modules/$(uname -r)/
kernel-default-5.14.21-150400.24.63.1.x86_64 <---------------------
kernel-default-extra-5.14.21-150400.24.63.1.x86_64
kernel-default-optional-5.14.21-150400.24.63.1.x86_64
Take the package name and prefix it with ‘kernel-default-devel’:
====================================================================
# zypper install kernel-default-devel-5.14.21-150400.24.63.1.x86_64
Loading repository data...
Reading installed packages...
The selected package 'kernel-default-devel-5.14.21-150400.24.63.1.x86_64' from repository 'Update repository with updates from SUSE Linux Enterprise 15' has lower version than the installed one. Use 'zypper install --oldpackage kernel-default-devel-5.14.21-150400.24.63.1.x86_64' to force installation of the package.
Resolving package dependencies...
Nothing to do.
# zypper install --oldpackage kernel-default-devel-5.14.21-150400.24.63.1.x86_64
Loading repository data...
Reading installed packages...
Resolving package dependencies...
The following 2 NEW packages are going to be installed:
kernel-default-devel-5.14.21-150400.24.63.1 kernel-devel-5.14.21-150400.24.63.1
2 new packages to install.
Now the build directory exists:
# ls -alh /lib/modules/$(uname -r)
total 5.4M
drwxr-xr-x 1 root root 510 Dec 13 19:46 .
drwxr-xr-x 1 root root 164 Dec 13 19:42 ..
lrwxrwxrwx 1 root root 54 May 3 2023 build -> /usr/src/linux-5.14.21-150400.24.63-obj/x86_64/default
drwxr-xr-x 1 root root 94 May 3 2023 kernel
drwxr-xr-x 1 root root 60 Dec 13 19:37 mfe_aac
It is less likely you will run into this if you run ‘zypper update’ first. Be aware, however, that this can take more than fifteen minutes to complete.
Next, reboot the Linux client and then run:
# zypper install kernel-default-devel