sarg

[root@serverX ~]# yum install epel-release

On RedHat/CentOS/Fedora

# yum install –y gcc gd gd-devel make perl-GD wget httpd

# wget http://liquidtelecom.dl.sourceforge.net/project/sarg/sarg/sarg-2.3.10/sarg-2.3.10.tar.gz

# tar -xvzf sarg-2.3.10.tar.gz

# cd sarg-2.3.10

# ./configure

# make

# make install

yum -y install libstdc++-devel pcre-devel gd-devel

[root@serverX ~]# cd /etc/yum.repos.d/

[root@serverX yum.repos.d]# ls

CentOS-Base.repo CentOS-Sources.repo epel.repo

CentOS-Debuginfo.repo CentOS-Vault.repo epel-testing.repo

[root@serverX yum.repos.d]# cd /opt

[root@serverX opt]# wget http://sourceforge.net/projects/sarg/files/sarg/sarg-2.3.7/sarg-2.3.7.tar.gz

[root@serverX opt]# ls

[root@serverX opt]# tar -zxvf sarg-2.3.7.tar.gz

[root@serverX opt]# ls

sarg-2.3.7 sarg-2.3.7.tar.gz

[root@serverX opt]# cd sarg-2.3.7

[root@serverX sarg-2.3.7]# ls

[root@serverX sarg-2.3.7]# ./configure

[root@serverX sarg-2.3.7]# make

[root@serverX sarg-2.3.7]# make install

[root@serverX sarg-2.3.7]# vim /usr/local/etc/sarg.conf

7 access_log /var/log/squid/access.log ;very path

25 title "WorldCM Squid access report"

45 font_size 12px

50 header_font_size 12px

55 title_font_size 35px

120 output_dir /var/www/html/squid-reports

216 date_format e

257 overwrite_report yes

[root@opt sarg-2.3.7]# sarg -x

[root@opt sarg-2.3.7]# service httpd restart

[root@serverX squid]# firewall-cmd --permanent --add-port=80/tcp

[root@serverX squid]# firewall-cmd --reload

[root@serverX squid]# setenforece 0

Move DesktopX:

-------------

Browse: http://x.x.x.x/squid-reports

To Generate the logreport, simply run the command sarg

# mkdir -p /var/www/html/squid-reports

7) To generate the log report by the cron job you can make entry in “/etc/crontab” file

00 23 * * * root /usr/bin/sarg

-------------htp password----------

# [root@linuxbox ~]# vi /etc/httpd/conf.d/sarg.conf

---------------------------------------------

Alias /sarg /var/www/html/squid-reports

DirectoryIndex index.html

Order deny,allow

# Deny from all

# Allow from 127.0.0.1

Allow from all

</Directory>

AuthUserFile /var/www/html/squid-reports/sarg-htpasswd

AuthName "sarg"

AuthType Basic

require valid-user

ExpiresActive On

ExpiresDefault M310

</directory>

------------------------------------------------

# htpasswd -cm /var/www/html/squid-reports/sarg-htpasswd admin

######################################################################################################

SARG on CentOS 6

Usually, it's pretty hard to analyze information from the squid log file. For example, I don't know how to analyze date or number of hits from /var/log/squid/access.log. If someone needs to analyze which websites are being accessed from the network, SARG may be a very good tool. SARG, or Squid Analysis Report Generator (http://sarg.sourceforge.net) analyzes the log, and generates a web based table where one can easily analyze proxy traffic.

Although SARG can be installed using YUM, I have faced problems with CentOS 6. So, I went for tarball installation instead. And believe, it's really easy unlike many tarball installtions.

So, let's start:

[root@busy-bee2 ~]# yum install gcc make wget httpd [root@busy-bee2 ~]# wget http://sourceforge.net/projects/sarg/files/sarg/sarg-2.3.1/sarg-2.3.1.tar.gz/download [root@busy-bee2 ~]# tar zxvf sarg-2.3.1.tar.gz [root@busy-bee2 ~]# cd sarg-2.3.1 [root@busy-bee2 ~]# ./configure [root@busy-bee2 ~]# make [root@busy-bee2 ~]# make install

Time to modify the conf file

[root@busy-bee2 ~]# vim /usr/local/etc/sarg.conf

There are a lot of options, and it is always recommended to go through them. However, we'll be editing only the ones that we need.

#### sarg.conf#### access_log /var/log/squid/access.log date_format e ## since here we use date format DD-MM-YYYY overwrite_report yes ## because I don't want multiple sarg reports for the same day output_dir /var/www/html/squid-reports

Time for a test run

[root@busy-bee2 ~]# sarg -x

We have used to the "-x" parameter for to view detail information on the run (used for debugging). If all goes well, there should be a report generated at /var/www/html/squid-reports directory which can be accessed from the web browser using the address http://IP/squid-reports

Sarg in Browser

Now, we'd be adding a scheduled task to run SARG at 02:30 everyday. To find out the executable file for SARG, we could use the 'which' command-

[root@busy-bee2 ~]# which sarg ## OUTPUT ## /usr/local/bin/sarg [root@busy-bee2 ~]# crontab -e 30 2 * * * /usr/local/bin/sarg [root@busy-bee2 ~]# service crond restart [root@busy-bee2 ~]# chkconfig crond on

Troubleshooting

If there is problem viewing the SARG page, here are a few tips:

Check whether the Firewall is blocking (iptables)
Check if there is a file /etc/httpd/conf.d/sarg.conf. There is a line "allow from". Modify it to suit your needs.
Verify that there is directory "/var/www/html/squid-reports"

http://amar-linux.blogspot.com/2012/02/sarg-on-centos-6.html

-------------###############################################################################################

SARG – Squid Analysis Report Generator and Internet Bandwidth Monitoring Tool

SARG is an open source tool that allows you to analyse the squid log files and generates beautiful reports in HTML format with informations about users, IP addresses, top accessed sites, total bandwidth usage, elapsed time, downloads, access denied websites, daily reports, weekly reports and monthly reports.

The SARG is very handy tool to view how much internet bandwidth is utilized by individual machines on the network and can watch on which websites the network’s users are accessing.

Install Sarg Squid Log Analyzer in Linux

In this article I will guide you on how to install and configure SARG – Squid Analysis Report Generator on RHEL/CentOS/Fedora and Debian/Ubuntu/Linux Mint systems.

Installing Sarg – Squid Log Analyzer in Linux

I assume that you already installed, configured and tested Squid server as a transparent proxy and DNS for the name resolution in caching mode. If not, please install and configure them first before moving further installation of Sarg.

Important: Please remember without the Squid and DNS setup, no use of installing sarg on the system it will won’t work at all. So, it’s a request to install them first before proceeding further to Sarg installation.

Follow these guides to install DNS and Squid in your Linux systems:

Install Cache-Only DNS Server

Install Squid as Transparent Proxy

Step 1: Installing Sarg from Source

The ‘sarg‘ package by default not included in RedHat based distributions, so we need to manually compile and install it from source tarball. For this, we need some additional pre-requisites packages to be installed on the system before compiling it from source.

On RedHat/CentOS/Fedora

# yum install –y gcc gd gd-devel make perl-GD wget httpd

Once you’ve installed all the required packages, download the latest sarg source tarball or you may use the following wget command to download and install it as shown below.

# wget http://liquidtelecom.dl.sourceforge.net/project/sarg/sarg/sarg-2.3.10/sarg-2.3.10.tar.gz # tar -xvzf sarg-2.3.10.tar.gz # cd sarg-2.3.10 # ./configure # make # make install

On Debian/Ubuntu/Linux Mint

On Debian based distributions, sarg package can be easily install from the default repositories using apt-get package manager.

$ sudo apt-get install sarg

Step 2: Configuring Sarg

Now it’s time to edit some parameters in SARG main configuration file. The file contains lots of options to edit, but we will only edit required parameters like:

Access logs path
Output directory
Date Format
Overwrite report for the same date.

Open sarg.conf file with your choice of editor and make changes as shown below.

# vi /usr/local/etc/sarg.conf [On RedHat based systems]

$ sudo nano /etc/sarg/sarg.conf [On Debian based systems]

Now Uncomment and add the original path to your squid access log file.

# sarg.conf # # TAG: access_log file # Where is the access.log file # sarg -l file # access_log /var/log/squid/access.log

Next, add the correct Output directory path to save the generate squid reports in that directory. Please note, under Debian based distributions the Apache web root directory is ‘/var/www‘. So, please be careful while adding correct web root paths under your Linux distributions.

# TAG: output_dir # The reports will be saved in that directory # sarg -o dir # output_dir /var/www/html/squid-reports

Set the correct date format for reports. For example, ‘date_format e‘ will display reports in ‘dd/mm/yy‘ format.

# TAG: date_format # Date format in reports: e (European=dd/mm/yy), u (American=mm/dd/yy), w (Weekly=yy.ww) # date_format e

Next, uncomment and set Overwrite report to ‘Yes’.

# TAG: overwrite_report yes|no # yes - if report date already exist then will be overwritten. # no - if report date already exist then will be renamed to filename.n, filename.n+1 # overwrite_report yes

That’s it! Save and close the file.

Step 3: Generating Sarg Report

Once, you’ve done with the configuration part, it’s time to generate the squid log report using the following command.

# sarg -x [On RedHat based systems]

# sudo sarg -x [On Debian based systems]

Sample Output

[root@localhost squid]# sarg -x SARG: Init SARG: Loading configuration from /usr/local/etc/sarg.conf SARG: Deleting temporary directory "/tmp/sarg" SARG: Parameters: SARG: Hostname or IP address (-a) = SARG: Useragent log (-b) = SARG: Exclude file (-c) = SARG: Date from-until (-d) = SARG: Email address to send reports (-e) = SARG: Config file (-f) = /usr/local/etc/sarg.conf SARG: Date format (-g) = USA (mm/dd/yyyy) SARG: IP report (-i) = No SARG: Keep temporary files (-k) = No SARG: Input log (-l) = /var/log/squid/access.log SARG: Resolve IP Address (-n) = No SARG: Output dir (-o) = /var/www/html/squid-reports/ SARG: Use Ip Address instead of userid (-p) = No SARG: Accessed site (-s) = SARG: Time (-t) = SARG: User (-u) = SARG: Temporary dir (-w) = /tmp/sarg SARG: Debug messages (-x) = Yes SARG: Process messages (-z) = No SARG: Previous reports to keep (--lastlog) = 0 SARG: SARG: sarg version: 2.3.7 May-30-2013 SARG: Reading access log file: /var/log/squid/access.log SARG: Records in file: 355859, reading: 100.00% SARG: Records read: 355859, written: 355859, excluded: 0 SARG: Squid log format SARG: Period: 2014 Jan 21 SARG: Sorting log /tmp/sarg/172_16_16_55.user_unsort ......

Note: The ‘sarg -x’ command will read the ‘sarg.conf‘ configuration file and takes the squid ‘access.log‘ path and generates a report in html format.

--------

Main sarg configuration file: /usr/local/etc/sarg.conf

[root@linuxbox ~]# vim /usr/local/etc/sarg.conf

# This is sample configuration, that you need to setup

access_log /var/log/squid/access.log

graphs yes

graph_days_bytes_bar_color orange

title “Squid User Access Reports”

temporary_dir /tmp

output_dir /var/www/html/reports/

[root@linuxbox ~]# vi /etc/httpd/conf.d/sarg.conf

Alias /sarg /var/www/html/reports/daily

DirectoryIndex index.html

Order deny,allow

Deny from all

# add IP address you allow

Allow from 192.168.0.2

</Directory>

-------------htp password----------

Alias /sarg /var/www/html/reports/daily

DirectoryIndex index.html

Order deny,allow

# Deny from all

# Allow from 127.0.0.1

Allow from all

</Directory>

AuthUserFile /var/www/html/reports/daily/sarg-htpasswd

AuthName "sarg"

AuthType Basic

require valid-user

ExpiresActive On

ExpiresDefault M310

</directory>

--------------------------------------------------------

[root@mail ~]# htpasswd -cm /usr/share/mailgraph/mailgraph-htpasswd admin

New password:

Re-type new password:

Now create a script to create daily statistics.

[root@linuxbox ~]# vim sarglog.sh

#!/bin/bash

#Get current date

TODAY=$(date +%d/%m/%Y)

#Get one week ago today

YESTERDAY=$(date –date “1 day ago” +%d/%m/%Y)

/usr/local/bin/sarg -l /var/log/squid/access.log -o /var/www/html/reports/daily -z -d $YESTERDAY-$TODAY

exit 0

[root@lan ~]# chmod 755 sarglog.sh

[root@lan ~]# ./sarglog.sh # run

# run at 1:00 AM every day

[root@linuxbox ~]# crontab -e

00 01 * * * /root/sarglog.sh

-----

Step 4: Assessing Sarg Report

The generated reports placed under ‘/var/www/html/squid-reports/‘ or ‘/var/www/squid-reports/‘ which can be accessed from the web browser using the address.

http://localhost/squid-reports OR http://ip-address/squid-reports

Sarg Main Window

Proxy Authentication Failures

Step 5: Automatic Generating Sarg Report

To automate the process of generating sarg report in given span of time via cron jobs. For example, let’s assume you want to generate reports on hourly basis automatically, to do this, you need to configure a Cron job.

# crontab -e

Next, add the following line at the bottom of the file. Save and close it.

* */1 * * * /usr/local/bin/sarg -x

The above Cron rule will generate SARG report every 1 hour.

Reference Links

Sarg Homepage

That’s it with SARG! I will be coming up with few more interesting articles on Linux, till then stay tuned to TecMint.com and don’t forget to add your valuable comments.

http://www.tecmint.com/sarg-squid-analysis-report-generator-and-internet-bandwidth-monitoring-tool/

ERROR

configure: gd.h was not found so the graphs won't be available in the report

configure: pcre.h was not found so the regexp won't be available in the hostalias

– libstdc++-devel rpm

– pcre-devel

– gd-devel

-----