rrdtool

What is RRDtool?

RRDtool is a high performance data logging and graphing system for time series data. [1] RRD stands for round-robin database. [2]

A glance to it.

Windows binary

A windows version can be found at rrdtool compiled by Christian Krausse.

Reproducing the output of mrtg-rrd.cgi with custom input data

Due to a special request, I need to plot the graph generated by mrtg-rrd.cgi with a set of given data in Excel using the human form date and transfer rate. Before fully understanding how does mrtg and rrdtool work, the most precise way of reproducing what they do is by inspecting the invocation parameters for rrdtool.

Methodologies:

• 1. Run snmpd on a Tomato router for mrtg to work.

  2. Run mrtg to peek the parameters to run rrdtool create and update.

  3. Run mrtg-rrd.cgi to peek the parameters to run rrdtool graph.

Run snmpd on a Tomato router for mrtg to work.

To see how mrtg use rrdtool, the most precise way is to run it with the option of using rrdtool. mrtg needs at least a host that is running snmpd. Google "Tomato snmp" gives the link http://systembash.com/content/how-to-install-snmp-on-tomato-router-firmware-and-graph-traffic-with-cacti/.

Run mrtg to peek the parameters to run rrdtool create and update.

• 1. Follow these guides: mrtg guide for windows and How to use RRDtool with MRTG.

  2. Since it is quite hard for me to find a working rrdtool perl binding for windows. I've stubbed the binding methods instead. See the rrds.pm attached in this page.

  3. perl mrtg mrtg.cfg

  4. Capture the usage of rrdtool create and update.

This is the final command to create a round-robin database that suits my use after reading the rrdtool reference:

rrdtool create test.rrd -b 123 DS:ds0:GAUGE:86400:0:100000000 DS:ds1:GAUGE:86400:0:100000000 -s 300 RRA:AVERAGE:0.5:24:5800 RRA:AVERAGE:0.5:288:800        RRA:MAX:0.5:24:5800 RRA:MAX:0.5:288:800

If the rate of the available input data is fewer than the heartbeat specified in the Data Source parameter, the missing data will be set by rrdtool as unknown. No matter what your graph won't see anything. To prevent this, use a heartbeat value of at least as large as the available input data rate. In this case it is 86400, which is equivalent to a day.

If the input data is already in rate, use Data Type "GAUGE" instead of "COUNTER".

Run mrtg-rrd.cgi to peek the parameters to run rrdtool graph.

• 1. Patch mrtg-rrd.cgi to use your own mrtg.cfg.

  2. Patch mrtg-rrd.cgi to run on command line. See the snippet below.

  3. Note that the stat line should use a value that exists in mrtg.cfg. Most preferably print the %targets variable to see all valid values.

  4. Capture the usage of rrdtool graph.

The final command after reading the rrdtool reference:

rrdtool graph test.png -s 201112 -e start+1y -c FONT#000000 -c MGRID#000000 -c FRAME#000000 -g -l 0 -c BACK#f5f5f5 -c ARROW#000000 -b 1000 -w 400 -h 100 DEF:in=test.rrd:ds0:MAX -v "Bits per second" AREA:in#00cc00:In HRULE:10000000#cc0000 -m 3 -x WEEK:1:MONTH:1:MONTH:1:2592000:%b

"-m 3" is a useful feature to zoom in or out the graph. It maintains the image quality by using larger font and thicker grids. You may discover that the CDEF is removed. This is because the input data is in bits already so there is no need to multiply it by 8 again.

Snippet of mrtg-rrd.cgi:

sub handler ($)

{

my ($q) = @_;

try_read_config($q->url());

my $path = $q->path_info();

$path =~ s/^\///;

$path =~ s/\/$//;

if (defined $directories{$path}) {

if ($q->path_info() =~ /\/$/) {

print_dir($path);

} else {

print "Location: ", $q->url(-path_info=>1), "/\n\n";

}

# return;

}

my ($dir, $stat, $ext) = ($q->path_info() =~

/^(.*)\/([^\/]+)(\.html|-(day|week|month|year)\.$imagetype)$/);

$dir = 'test';

$stat = '192.168.1.1_2';

$ext = '-week.png';

To combine them all

A Python program is written to convert the input data into RRD and two batch files with a shell script are written to help generating the graphs.

sum.py - combine the input data raw.txt and raw_2h.txt to test.rrd.

graph.bat - generate a graph in year or a month.

graph_all.bat - generate all desired graphs.

mwk.sh - return the first and last day of the same start and end week of a month.

They're zipped in rrdtool_prog.zip.

Problems and workaround

During the process, I've met some problems. Hope sharing them could speed up your progress.

1. rrdtool graph crashes on Windows

So far a known condition is when generating a monthly graph using the default x-axis label or %V, rrdtool will crash. A workaround is to use %U instead.

2. rrdtool is displaying garbage on the x-axis label on Windows for yearly graphs

I believe the rrdtool binary I'm using is using Windows' locale to generate the month names. Therefore, instead of changing the LANG variable for POSIX, change your locale from the regional and language setting's format. I'd suggest to use English as the compiled binary may not support non-English characters.

References:

[1] http://oss.oetiker.ch/rrdtool/

[2] http://oss.oetiker.ch/rrdtool/doc/rrdtool.en.html