NFS (Network File System)

NFS allows a system to share directories and files with others over a network. By using NFS, users and programs can access files on remote systems almost as if they were local files.

Some of the most notable benefits that NFS can provide are:

• Local workstations use less disk space because commonly used data can be stored on a single machine and still remain accessible to others over the network.
• There is no need for users to have separate home directories on every network machine. Home directories could be set up on the NFS server and made available throughout the network.
• Storage devices such as floppy disks, CDROM drives, and USB Thumb drives can be used by other machines on the network. This may reduce the number of removable media drives throughout the network.

Firewall Ports for NFS

If you have a firewall you need to make sure ports 32771, 111 and 2049 are open

Setup NFS Server

At a terminal prompt enter the following command to install the NFS Server on Ubuntu:

bash$ sudo apt-get install nfs-kernel-server

Configuration Files

There are three main configuration files you will need to edit to set up an NFS server: /etc/exports, /etc/hosts.allow, and /etc/hosts.deny.

The /etc/exports file contains a list of entries; each entry indicates a volume that is shared and how it is shared.

directory machine1(option11,option12) machine2(option21,option22)

• directory - the directory that you want to share. It may be an entire volume though it need not be. If you share a directory, then all directories under it within the same file system will be shared as well.
• machine - client machines that will have access to the directory. The machines may be listed by:
  • DNS - machine.company.com
  • IP address - 192.168.4.2
  • Network - 192.168.4.0/255.255.255.0
  • Wildcard - *, *.company.com
• options - listing for each machine will describe what kind of access that machine will have:
  • ro - read only, this is a default
  • rw - read and write access
  • no_root_squash - By default, any file request made by user root on the client machine is treated as if it is made by user nobody on the server. (Exactly which UID the request is mapped to depends on the UID of user "nobody" on the server, not the client.) If no_root_squash is selected, then root on the client machine will have the same level of access to the files on the system as root on the server. This can have serious security implications, although it may be necessary if you want to perform any administrative work on the client machine that involves the exported directories. You should not specify this option without a good reason.
  • no_subtree_check - If only part of a volume is exported, a routine called subtree checking verifies that a file that is requested from the client is in the appropriate part of the volume. If the entire volume is exported, disabling this check will speed up transfers.
  • sync - By default, all but the most recent version (version 1.11) of the exportfs command will use async behavior, telling a client machine that a file write is complete - that is, has been written to stable storage - when NFS has finished handing the write over to the filesystem.

For example, sharing /home directory to all the server in the 192.168.4.0 network with read and write access:

/home 192.168.4.0/255.255.255.0(rw)

Some precautionary with NFS sharing:

• If a directory is exported, its parent and child directories cannot be exported if they are in the same filesystem.
• it is a poor idea to export a FAT or VFAT (i.e., MS-DOS or Windows 95/98) filesystem with NFS. FAT is not designed for use on a multi-user machine, and as a result, operations that depend on permissions will not work well.

The /etc/hosts.allow and /etc/hosts.deny specify which computers on the network can use services on your machine. Each line of the file contains a single entry listing a service and a set of machines. When the server gets a request from a machine, it does the following:

• It first checks hosts.allow to see if the machine matches a rule listed here. If it does, then the machine is allowed access.
• If the machine does not match an entry in hosts.allow the server then checks hosts.deny to see if the client matches a rule listed there. If it does then the machine is denied access.
• If the client matches no listings in either file, then it is allowed access.

In general it is a good idea with NFS (as with most internet services) to explicitly deny access to IP addresses that you don't need to allow access to.

The first step in doing this is to add the followng entry to /etc/hosts.deny:

portmap:ALL

lockd:ALL

mountd:ALL

rquotad:ALL

statd:ALL

Furthermore, adding an entry ALL:ALL in the file /etc/hosts.deny causes any service that looks at these files to deny access to all hosts unless it is explicitly allowed.

Next, we need to add an entry to /etc/hosts.allow to give any hosts access that we want to have access. Entries in hosts.allow follow the format:

service: [host | network/netmask] , [host | network/netmask]

Startup Services

Check that these services are available on the system before starting the NFS services:

• Need to have TCP/IP networking functioning correctly on your machine. If you can use telnet, FTP, and so on, then chances are your TCP networking is fine.
• NFS depends on the portmapper daemon, either called portmap or rpc.portmap. It will need to be started first. It should be located in /sbin but is sometimes in /usr/sbin.

To start the NFS server on Ubuntu, you can run the following command at a terminal prompt:

bash$ sudo /etc/init.d/nfs-kernel-server start

The Daemons

NFS serving is taken care of by five daemons:

• rpc.nfsd - which does most of the work
• rpc.lockd and rpc.stat - handle file locking
• rpc.mountd - handles the initial mount requests
• rpc.rquotad - handles user file quotas on exported volumes.

The daemons are all part of the nfs-utils package, and may be either in the /sbin directory or the /usr/sbin directory.

Verifying NFS

To do this, query the portmapper to find out what services it is providing by using this command:

bash$ rpcinfo quota

You should get something like this:

program  vers  proto  port

100000     2    tcp   111   portmapper

100000     2    udp   111   portmapper

100011     1    udp   749   rquotad

100011     2    udp   749   rquotad

100005     1    udp   759   mountd

100005     1    tcp   761   mountd

100005     2    udp   764   mountd

100005     2    tcp   766   mountd

100005     3    udp   769   mountd

100005     3    tcp   771   mountd

100003     2    udp   2049  nfs

100003     3    udp   2049  nfs

300019     1    tcp   830   amd

300019     1    udp   831   amd

100024     1    udp   944   status

100024     1    tcp   946   status

100021     1    udp   1042  nlockmgr

100021     3    udp   1042  nlockmgr

100021     4    udp   1042  nlockmgr

100021     1    tcp   1629  nlockmgr

100021     3    tcp   1629  nlockmgr

100021     4    tcp   1629  nlockmgr

Making Changes to /etc/exports

Modification to /etc/exports file may not take effect immediately. Run the command below to force nfsd to re-read the /etc/exports file.

bash$ exportfs -ra

If you can't find the exportfs command, then you can kill nfsd with the -HUP flag:

bash$ kill nfsd -HUP

NFS Client

In general, check that the client machine supports NFS mounting by looking into the /proc/filesystems and make sure there is a line containing nfs. If not, type in this command below to insert the nfs module:

bash$ insmod nfs

To begin using machine as an NFS client, you will need the portmapper running on that machine, and to use NFS file locking, you will also need rpc.statd and rpc.lockd running on both the client and the server. Most recent distributions start those services by default at boot time.

For Ubuntu, make sure the nfs-common package is installed on your client. To install nfs-common enter the following command at the terminal prompt:

bash$ sudo apt-get install nfs-common

Mounting

Use the mount command to mount a shared NFS directory from another machine, by typing a command line similar to the following at a terminal prompt:

bash$ sudo mount -f nfs server.name.com:/export/home /mnt/home

Note: The mount point directory /mnt/home must exist. There should be no files or subdirectories in the /mnt/home directory.

To unmount the NFS, simply type in this command:

bash$ umount /mnt/home

To check what is mounted on your client, type in this command:

bash$ mount

Mounting at Boot Time

An alternate way to mount an NFS share from another machine is to add a line to the /etc/fstab file. The line must state the hostname of the NFS server, the directory on the server being exported, and the directory on the local machine where the NFS share is to be mounted.

The general syntax for the line in /etc/fstab file is as follows:

device                  mountpoint    fs-type     options      dump  fsckorder

....

server.name.com:/home    /mnt/home      nfs     rw,hard,intr    0        0

....

Options:

• soft - If a file request fails, the NFS client will report an error to the process on the client machine requesting the file access. Some programs can handle this with composure, most won't. We do not recommend using this setting; it is a recipe for corrupted files and lost data. You should especially not use this for mail disks.
• hard - The program accessing a file on a NFS mounted file system will hang when the server crashes. The process cannot be interrupted or killed (except by a "sure kill") unless you also specify intr. When the NFS server is back online the program will continue undisturbed from where it was. We recommend using hard,intr on all NFS mounted file systems.
• rsize - read size of the chunks of data that the client and server pass back and forth to each other.
• wsize - write size of the chunks of data that the client and server pass back and forth to each other.

Optimizing NFS

Setting block Size to Optimize Transfer Speeds

The mount command options rsize and wsize specify the size of the chunks of data that the client and server pass back and forth to each other. Here are some pointer for optimizing the transfer speed by modifying the option rsize and wsize:

• The most common default value is 4K (4096 bytes) if no rsize and wsize options are specified.
• NFS V2 protocol is 8K
• NFS V3 protocol limit is specific to the server. The maximum block size is defined by the value of the kernel constant NFSSVC_MAXBLKSIZE found in the Linux kernel source file ./include/linux/nfsd/const.h.
• NFS V3 can support up to 64K of block size if permitted.
• Current maximux block size for Linux kernel 2.4.17 is 8K (8192 bytes)
• All 2.4 clients currently support up to 32K block tranfer sizes, allowing the standard 32K block transfers across NFS mounts from other servers, such as Solaris, without client modification.

You will want to experiment and find an rsize and wsize that works and is as fast as possible. You can try to transfer 16384 blocks of 16k each from the special file /dev/zero (which if you read it just spits out zeros really fast) to the mounted partition. Time to see how long it will take to do this operation:

bash$ time dd if=/dev/zero of=/mnt/home/testfile bs=16k count=16384

This creates a 256Mb file of zeroed bytes. In general, you should create a file that's at least twice as large as the system RAM on the server, but make sure you have enough disk space. Then read back the file into the great black hole on the client machine (/dev/null) by typing the following:

bash$ time dd if=/mnt/home/testfile of=/dev/null bs=16k

Repeat this a few times and average how long it takes. Be sure to unmount and remount the filesystem each time (both on the client and, if you are zealous, locally on the server as well), which should clear out any caches. The block size should be a power of two since most of the parameters that would constrain it (such as file system block sizes and network packet size) are also powers of two. However, some users have reported better successes with block sizes that are not powers of two but are still multiples of the file system block size and the network packet size.

Remember to edit /etc/fstab to reflect the rsize/wsize you found to be the most desirable.

Here are some benchmarks that may prove useful:

• Bonnie: http://www.textuality.com/bonnie/
• Bonnie++: http://www.coker.com.au/bonnie++
• IOZone File System Benchmark:http://www.iozone.org
• The official NFS benchmark, SPECsfs97: http://www.spec.org/osg/sfs97/

Packet Size and Network Drivers

Here are some ways for experimenting with your network card to find out how it can best handle traffic:

• Pinging back and forth between the two machines with large packets using the -f and -s options with ping and see if a lot of packets get dropped, or if they take a long time for a reply
• nfsstat command to look at nfs transactions, client and server statistics, network statistics, and so forth. The -o net option will show you the number of dropped packets in relation to the total number of transactions.
• Using tracepath to see the MTU reported at the bottom. Then set the MTU on your network card equal to the path MTU, by using the MTU option to ifconfig, and see if fewer packets get dropped. See the ifconfig man pages for details on how to reset the MTU.
• netstat -s will give the statistics collected for traffic across all supported protocols.
• Also look at /proc/net/snmp for information about current network behavior.