14 Set Up DirectNFS for Example Database

References

Overview

One of the OCM 11G objectives to use DirectNFS driver. I plan to practice to set up a data file on NFS.

Procedure

Create NFS Volume and Share

Using the FreeNAS Web Gui at http://freenas.yaocm.id.au, I created a ZFS dataset called NFSDATA which is 1GB in size, and is owned by asm:asmdba.

I also created a NFS Share for /mnt/PENRITH/NFSDATA for access only by the 192.168.2.0/24 network and having a Maproot User of root.

Create Mount Point on PENRITH1

Create the /u08 directory on PENRITH1:

sudo su - mkdir /u08 chmod 775 /u08 chown oracle:dba /u08

Update the /etc/fstab to allow the mounting of the ZFS Dataset (/mnt/PENRITH/NFSDATA) as /u08.

sudo su - cat >>/etc/fstab <<DONE freenas-nas:/mnt/PENRITH/NFSDATA /u08 nfs (rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,noac,actimeo=0,nfsvers=3,timeo=600,addr=192.168.2.1) DONE mount /u08

Enabling Direct NFS Client Oracle Disk Manager Control of NFS

Following the procedure in Enabling Direct NFS Client Oracle Disk Manager Control of NFS, on PENRITH1:

[oracle@penrith1 dbs]$ cd $ORACLE_HOME/lib [oracle@penrith1 lib]$ cp libodm11.so libodm11.so_stub [oracle@penrith1 lib]$ ln -s libnfsodm11.so libodm11.so ln: creating symbolic link `libodm11.so' to `libnfsodm11.so': File exists [oracle@penrith1 lib]$ ls -l libodm11.so lrwxrwxrwx 1 oracle oinstall 12 Jan 2 15:37 libodm11.so -> libodmd11.so [oracle@penrith1 dbs]$ cd /u01/app/oracle/product/11.1.0/db_2/dbs [oracle@penrith1 dbs]$ cat >oranfstab <<DONE > server: freenas-nas > path: penrith1-nas > export: /mnt/PENRITH/NFSDATA /u08 > DONE

Get EXAMPLE Database to use DirectNFS

p>I created a tablespace called NFSTEST with a datafile on /u08, and there are still no rows in the V$DNFS_* views.

Review the Documentation

When stuck, RTFM.

Step by Step - Configure Direct NFS Client (DNFS) on Linux (Doc ID 762374.1) says that:

WARNING: The DNFS Guide says to enable the init.ora param filesystem_io to enable direct I/O support.

Doing this, all DB access to all files will be via DIO.

There is no way to only enable direct IO for certain files and exclude others.

So ensure ALL the places where are located your database files (including pfile/spfile, logs,....) are configured to accept Direct IO.

The actual parameter is FILESYSTEMIO_OPTIONS.

Set ASM up for Direct IO

The following is added to the pfile, $ORACE_HOME/dbs/init+ASM1.ora,:

FILESYSTEMIO_OPTIONS = directIO

I also corrected the $ORACLE_HOME/dbs/oranfstab as follows:

server: freenas path: 192.168.2.1 local: 192.168.2.2 export: /mnt/PENRITH/ASMDATA mount: /u07

DirectNFS is Active on ASM

After restarting ASM, I see the following message in the +ASM1 alert log:

Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 2.0

Later on, there is:

Direct NFS: mount complete dir /mnt/PENRITH/ASMDATA on freenas path 192.168.2.1 mntport 623 nfsport 2049

v$dnfs_servers shows:

ID SVRNAME DIRNAME MNTPORT NFSPORT WTMAX RTMAX ---------- -------- -------------------- ---------- ---------- ---------- ---------- 1 freenas /mnt/PENRITH/ASMDATA 623 2049 32768 32768

v$dnfs_files shows:

FILENAME FILESIZE PNUM SVR_ID -------------------- ---------- ---------- ---------- /u07/data.dbf 3255435264 16 1 /u07/recovery.dbf 4259840000 16 1

v$dnfs_channels shows:

PNUM SVRNAME PATH CH_ID SVR_ID SENDS RECVS PINGS ---------- -------- --------------- ---------- ---------- ---------- ---------- ---------- 12 freenas 192.168.2.1 0 1 0 0 0 12 freenas 192.168.2.1 1 1 0 0 0 13 freenas 192.168.2.1 0 1 0 0 1 13 freenas 192.168.2.1 1 1 0 0 1 16 freenas 192.168.2.1 0 1 9 25 0 16 freenas 192.168.2.1 1 1 1 2 0 17 freenas 192.168.2.1 0 1 11 22 0 17 freenas 192.168.2.1 1 1 1 2 0 8 rows selected.

EXAMPLE Database to Use DirectNFS

The $ORACLE_HOME/dbs/oranfstab has been changed to:

server: freenas path: 192.168.2.1 local: 192.168.2.2 export: /mnt/PENRITH/NFSDATA mount: /u08

The initialization parameter was changed as follow:

alter system set filesystemio_options="directIO" scope=spfile;

The EXAMPLE database now successfully uses DirectNFS as shown by the alert log, and the contents of v$dnfs_servers:

ID SVRNAME DIRNAME MNTPORT NFSPORT WTMAX RTMAX ---------- ------------ -------------------- ---------- ---------- ---------- ---------- 1 freenas-nas /mnt/PENRITH/ASMDATA 623 2049 32768 32768 2 freenas /mnt/PENRITH/NFSDATA 623 2049 32768 32768

This is interesting because the EXAMPLE1 instance can see NFS files mounted by ASM.

13 Create Database Successful

15 Deploy EM Agent on PENRITH Cluster