Filesystem Monitoring

(Taken from the wikipedia for inotify http://en.wikipedia.org/wiki/Inotify)

"........

inotify is a Linux kernel subsystem that provides file system event notification. It was written by John McCutchan with help from Robert Love and later Amy Griffis to replace dnotify. It was included in the mainline kernel from release 2.6.13 (2005-06-18), and could be compiled into 2.6.12 and possibly earlier releases by use of a patch. Its function is essentially an extension to filesystems to notice changes to the filesystem, and report those changes to applications.

Its major use is therefore arguably in desktop search utilities like Beagle, where its functionality permits reindexing of changed files without scanning the filesystem for changes every few minutes, which would be very inefficient. By being told that a file has changed directly by the kernel, rather than actively looking, Beagle and such utilities can achieve change-to-reindexing times of only about a second, with very small performance hits (inotify therefore enables the use of such programs in a sensible manner; daemons are generally not accepted by distributors if they drain system performance noticeably to provide userland functionality.[citation needed])

It can also be used to automatically update directory views, reload configuration files, log changes, backup, synchronize, and upload.

............"

There are many little jobs which people tend to schedule, via cron, which do nothing unless particular files have appeared. These busy-wait style scripts may easily be replaced if you have the ability to execute commands when files are created, or filesystem events happen.

Rather than running a job every minute to see if there is a new file in a spool directory to process it makes more sense to begin the execution of a script whenever a file has just been created. Recent Linux kernels have included support for a feature called inotify. The inotify API provides a mechanism for monitoring file system events. Inotify can be used to monitor individual files, or to monitor directories. When a directory is monitored, inotify will return events for the directory itself, and for files inside the directory. To determine if your kernel has support for inotify you may do the following:

    • On RedHat systems:

      • [root@ldap2-lnx boot]# grep INOTIFY /boot/config-$(uname -r)

      • CONFIG_INOTIFY=y

      • CONFIG_INOTIFY_USER=y

      • [root@ldap2-lnx boot]#

    • On OpenSuSE and Debian systems:

      • sandholm@beast2:~> zgrep INOTIFY /proc/config.gz

      • CONFIG_INOTIFY=y

      • CONFIG_INOTIFY_USER=y

      • sandholm@beast2:~>

There is a companion package to the inotify API called incron. The incron utility operates just like your typical cron service. The only difference is that we put rules into the incrontab that specify a directory or file to watch, the condition to trigger on and the pathname of the script to execute when the trigger occurs. The incron utility is packaged and available for Debian systems. For RPM based systems I found a rhel5 and centos RPM package here: http://packages.sw.be/incron/. (NOTE: there's a typo in the /etc/init.d/incrond script, look for "prog" and replace with "proc". This appears to be fixed in the Debian Lenny distribution) You may also pull down the source code and compile it yourself from here: http://inotify.aiken.cz/?section=incron&page=download&lang=en.

Once you've verified your kernel was compiled for INOTIFY support and you've installed the package, you may start the incrond service with:

# /etc/init.d/incrond start

You will need to list your authorized user accounts in the /etc/incron.allow file. I've only authorized my "root" user. The contents of my /etc/incron.allow file:

root

You may now use the "incrontab -e" command to setup a rule for directory or file monitoring. I chose to monitor a directory in /tmp called /tmp/tom. I wanted the program in /usr/local/bin/new_file to be executed whenever a new file was created in /tmp/tom. Here's the /usr/local/bin/new_file program:

#! /bin/bash

echo "> New file found on $(date) at $*" >> /tmp/new_file.log

Here's what I put in my "incrontab -e" session:

/tmp/tom IN_CLOSE_WRITE /usr/local/bin/new_file $@/$#

Use "incrontab -d" each time you update your incrontab to reload the table.

This says "Watch /tmp/tom, and when an IN_CLOSE_WRITE event occurs run /usr/local/bin/new_file with the name of the file that was created".

There are two sets of magic flags here - the first is the IN_CLOSE_WRITE, and the second is $@/$#. The second set of flags is simpler to explain as there are fewer of them:

    • $$ - a dollar sign

    • $@ - the watched filesystem path (ie. the path you're watching)

    • $# - the event-related file name

    • $% - the event flags (textually)

    • $& - the event flags (numerically)

The actual event flags include IN_CLOSE_WRITE, which means that a file was closed for writing. The full list of supported flags include:

    • IN_ACCESS File was accessed (read)

    • IN_ATTRIB Metadata changed (permissions, timestamps, extended attributes, etc.)

    • IN_CLOSE: Watch both IN_CLOSE_WRITE and IN_CLOSE_NOWRITE

    • IN_CLOSE_WRITE File opened for writing was closed

    • IN_CLOSE_NOWRITE File not opened for writing was closed

    • IN_CREATE File/directory created in watched directory

    • IN_DELETE File/directory deleted from watched directory

    • IN_DELETE_SELF Watched file/directory was itself deleted

    • IN_MODIFY File was modified

    • IN_MOVE: Watch both IN_MOVED_FROM and IN_MOVED_TO

    • IN_MOVE_SELF Watched file/directory was itself moved

    • IN_MOVED_FROM File moved out of watched directory

    • IN_MOVED_TO File moved into watched directory

    • IN_OPEN File was opened

    • IN_ALL_EVENTS: Watch all listed events

    • IN_DONT_FOLLOW: Don't follow sym link

    • IN_ONLYDIR: Watch path only if it's a directory

Multiple flags may be separated via commas.

Now when a new file is created in /tmp/tom the script is executed. For example:

[root@ldap2-lnx ~]# echo testing > /tmp/tom/any1

[root@ldap2-lnx init.d]# cat /tmp/new_file.log

> New file found on Mon Sep 8 11:55:41 EDT 2008 at /tmp/tom/any2

> New file found on Mon Sep 8 11:56:03 EDT 2008 at /tmp/tom/any3

> New file found on Mon Sep 8 11:56:05 EDT 2008 at /tmp/tom/any4

> New file found on Mon Sep 8 12:02:23 EDT 2008 at /tmp/tom/any5

> New file found on Mon Sep 8 12:27:21 EDT 2008 at /tmp/tom,any3

> New file found on Mon Sep 8 12:53:28 EDT 2008 at /tmp/tom,any1

[root@ldap2-lnx init.d]#

It's worth noting that the with the IN_CLOSE_WRITE trigger, the script is executed once the file has been closed for writes.

incrontab usage:

incrontab - inotify cron table manipulator

(c) Lukas Jelinek, 2006, 2007, 208

usage: incrontab [<options>] <operation>

incrontab [<options>] <FILE-TO-IMPORT>

<operation> may be one of the following:

-?, --about gives short information about program

-h, --help prints this help text

-l, --list lists user table

-r, --remove removes user table

-e, --edit provides editting user table

-t, --types list supported event types

-d, --reload request incrond to reload user table

-V, --version prints program version

These options may be used:

-u <USER>, --user=<USER> overrides current user (requires root privileges)

-f <FILE>, --config=<FILE> overrides default configuration file (requires r

oot privileges)

For reporting bugs please use http://bts.aiken.cz