Performance

В случае SIMD мы берем сразу пачку операндов, делаем одно и то же действие над всеми разом и сохраняем пачку результатов. Для процессора это проще, чем несколько раз выполнить одинаковые действия. Существует огромное количество расширений команд процессоров с SIMD-инструкциями, например: MMX, SSE-SSE4, AVX, AVX2, AVX512, NEON.

Book

https://www.kernel.org/pub/linux/kernel/people/paulmck/perfbook/perfbook.2015.01.31a.pdf

https://habrahabr.ru/company/intel/blog/282738/

http://www.brendangregg.com/linuxperf.html

http://www.techtalkshub.com/optimizing-linux-servers/

http://devo.ps/blog/2013/03/06/troubleshooting-5minutes-on-a-yet-unknown-box.html

http://habrahabr.ru/post/216827/ load command

http://habrahabr.ru/company/selectel/blog/222839/ SysDig

http://www.brendangregg.com/USEmethod/use-linux.html

http://en.wikipedia.org/wiki/Comparison_of_web_hosting_control_panels

http://yusufarslan.net/choosing-control-panel-server-management

http://habrahabr.ru/post/215227/ Ruby with GUI

http://blog.scoutapp.com/articles/2015/02/24/understanding-linuxs-cpu-stats

System monitoring with Web GUI

http://linuxdash.com/ PHP, but there is python clone

https://github.com/abimaelmartell/system_monitor

https://github.com/martinrusev/amonone Python +Tornado

http://cockpit-project.org/

https://github.com/masterzh01/sofia python+ flask

Alerta

https://github.com/guardian/alerta

https://github.com/alerta/alerta-docs

while true; sleep 1; do top -b | head -n2 >> ~/top.log; done

watch -n1 tail 1000 ~/top.log

http://www.slideshare.net/brendangregg/scale2015-linux-perfprofiling

https://news.ycombinator.com/item?id=7212935

http://aosabook.org/en/ Performance of open-source-apps

http://www.infoq.com/presentations/top-10-performance-myths

http://habrahabr.ru/post/209446/

https://techtalk.intersec.com/2013/07/memory-part-1-memory-types/

https://techtalk.intersec.com/2013/07/memory-part-2-understanding-process-memory/

http://engineering.linkedin.com/performance/optimizing-linux-memory-management-low-latency-high-throughput-databases

https://news.ycombinator.com/item?id=6516974

http://www.perfdynamics.com/papers.html

http://www.infoq.com/news/2013/06/Native-Performance

http://channel9.msdn.com/Events/Build/2013/4-329

http://dtrace.org/blogs/brendan/2013/06/08/linux-performance-analysis-and-tools/

http://www.eclipsecon.org/2013/sessions/profiling-cc-applications-using-eclipse

http://www.joyent.com/blog/linux-performance-analysis-and-tools-brendan-gregg-s-talk-at-scale-11x

Monitoring Tools

http://aarvik.dk/linux-monitoring-tools-suggestions-from-hacker-news/

https://news.ycombinator.com/item?id=7125153

https://news.ycombinator.com/item?id=7124720

http://www.paperplanes.de/2013/1/10/virtues-of-monitoring-redux.html

http://www.techtraits.com/Programming/Monitoring/Python/2013/02/17/Monitoring-python-servers-with-pyformance-and-graphite/

https://github.com/armon/statsite#readme

http://andrewwdeane.blogspot.in/2012/05/system-monitoring.html

http://www.youtube.com/playlist?list=PLD2AE32F507F10481&feature=plcp

http://joyent.com/blog/linux-performance-analysis-and-tools-brendan-gregg-s-talk-at-scale-11x

http://duartes.org/gustavo/blog/best-of

http://blog.bugsense.com/post/26442766999/optimizing-your-linux-server-for-memory-based-nosql

http://lethain.com/introduction-to-architecting-systems-for-scale/

http://www.leansoftwarearchitecture.com/

http://habrahabr.ru/company/intel/blog/168473/

http://habrahabr.ru/post/167837/ profiling of already running program

http://aphyr.github.com/riemann/ written in Clojure distributed system monitor

http://vimeo.com/45807716

http://www.codersnotes.com/sleepy Profiler for C++/Windows (Like Zoom for Windows)

http://www.jeffknupp.com/blog/2012/07/10/software-optimization-a-systematic-approach/

http://www.pvk.ca/Blog/2012/07/03/binary-search-star-eliminates-star-branch-mispredictions/

http://www.planetdevops.net

http://blackanvil.blogspot.com/2011/12/integrating-grinder-performance-data.html

http://blackanvil.blogspot.com/2011/12/quickstatd-simple-tool-to-get.html

https://bitbucket.org/travis_bear/quickstatd

http://www.planetdevops.net/?p=12283

http://theagileadmin.com/2012/03/30/monitorin-aint-easy/

http://jedi.be/blog/2012/01/03/monitoring-wonderland-survey-introduction/

http://jedi.be/blog/2012/01/04/monitoring-wonderland-visualization/

http://en.wikipedia.org/wiki/Comparison_of_network_monitoring_systems

http://nethogs.sourceforge.net/

PINBA

http://pinba.org/wiki/Main_Page

http://habrahabr.ru/company/badoo/blog/149695/

SINFO

http://www.ant.uni-bremen.de/whomes/rinas/sinfo

http://www.linuxjournal.com/content/sinfo-advanced-network-monitoring

Dependency:

zypper in boost-devel

http://think-async.com/

http://www.datadoghq.com/2012/08/easy-app-metrics-with-pup/

Performance Unix Command Line Tools

http://www.thegeekstuff.com/2011/07/iostat-vmstat-mpstat-examples/

http://www.thegeekstuff.com/2011/03/sar-examples/

http://www.cyberciti.biz/tips/how-do-i-find-out-linux-cpu-utilization.html

http://linux.die.net/man/1/fuser

http://www.atoptool.nl/http://www.cyberciti.biz/tips/top-linux-monitoring-tools.html monitoring http://habrahabr.ru/blogs/linux/114082/ ntop and others

http://studyhat.blogspot.com/2009/10/how-do-i-find-out-linux-cpu-utilization.html

http://unix.stackexchange.com/questions/27076/how-can-i-receive-top-like-cpu-statistics-from-the-shell

http://studyhat.blogspot.com/2009/10/linux-memory-use.html

http://studyhat.blogspot.com/2010/09/cpuload-and-send-email-to-admin.html

http://home.clara.net/drdsl/OpenSource/perfmon.html

http://crtags.blogspot.com/2013/12/ptrace-now-available.html

https://github.com/psypete/public-bin/blob/public-bin/src/system/tracepid

http://dtrace.org/blogs/brendan/2011/12/18/visualizing-device-utilization/

http://elinux.org/Runtime_Memory_Measurement

http://freecode.com/projects/dstat

TOP

http://www.youtube.com/watch?v=yFKRsLj_Jhg

http://habrahabr.ru/post/114082/

http://publib.boulder.ibm.com/infocenter/javasdk/tools/index.jsp?topic=%2Fcom.ibm.java.doc.igaa%2F_1vg0001475cb4a-1190e2e0f74-8000_1007.html

http://meandubuntu.wordpress.com/2008/12/05/top-of-the-world-ma/

ATOP

https://lwn.net/Articles/387202/

NTOP

http://listgateway.unipi.it/pipermail/ntop/

You can also create HTML output file (dump) of the network status using ntop. Apart from the command line, you can also launch the web version of the ntop once you’ve started the ntopd service and visit http://{ip-address}:3000

/proc/stat file

http://www.centos.org/docs/5/html/Deployment_Guide-en-US/ch-proc.html

http://stackoverflow.com/questions/5514119/accurately-calculating-cpu-utilization-in-linux-using-proc-stat

http://www.linuxhowtos.org/System/procstat.htm

#include <stdio.h>

#include <unistd.h>

#include <fcntl.h>

#include <string.h>

int main(int argc, const char *argv[]) {

char path[32], data[2048];

int tps = sysconf(_SC_CLK_TCK);

sprintf(path, "/proc/%s/stat", argv[1]);

int fd = open(path, O_RDONLY);

if (fd < 0) {

perror("open");

return -1;

}

if (read(fd, data, 2048) == -1) {

perror("read");

return -1;

}

close(fd);

char name[1024];

long unsigned int utime, virt;

long int rss;

sscanf(data, "%*d %s %*c %*d %*d %*d %*d %*d %*u %*u %*u %*u %*u %lu"

" %*u %*d %*d %*d %*d %*d %*d %*u %lu %ld", name, &utime, &virt, &rss);

printf("%s active %lus, virtual size: %ld kB, resident pages: %ld\n",

name, utime/tps, virt/1024, rss);

return 0;

}

Usage: program <pid>

Profiling

http://goog-perftools.sourceforge.net/doc/cpu_profiler.html

http://www.perfdynamics.com/iBook/ppa_new.html

https://perf.wiki.kernel.org/index.php/Tutorial

http://pycounters.readthedocs.org/en/latest/index.html

Tuning Linux for Oracle

http://www.puschitz.com/TuningLinuxForOracle.shtml

http://coskan.wordpress.com/

DISK and File system benchmarking

http://habrahabr.ru/post/154235/

http://honglus.blogspot.com/2010/02/file-system-benchmarking.html

http://pommi.nethuis.nl/bonnie-to-google-chart/

http://honglus.blogspot.com/2010/01/double-network-throughput-by-tuning.html

Load average

http://en.wikipedia.org/wiki/Load_(computing)

http://www.linuxjournal.com/article/9001

Load average (returned by w, uptime and tload) number tells you how many processes on average are using or waiting on CPU for execution. If it is close to or over 1.0 * (NUMBER OF CPUs), it means the system is constantly busy with something. If load average is constantly raising, it means that system cannot keep up with the demand and tasks start to pile up. Monitoring load average instead of CPU utilization for system "health" has two advantages:

- Load averages given by system are already averaged. They don't fluctuate that much so you don't get that problem with parsing "ps" output.
- Some app may be thrashing disk and rendering system unresponsive. In this case CPU utilization might be low, but load average would still be high indicating a problem.

Script to show load average with timestamp

while true; do

L=$(awk '{ print $1 }' /proc/loadavg);

D=$(date +%H:%M:%S);

echo -e "$D\t$L";

sleep 1;

done

ipcs

http://www.real-world-systems.com/docs/ipcs.1.html

http://blog.tanelpoder.com/2009/01/22/identifying-shared-memory-segment-users-using-lsof/

Context Switch http://en.wikipedia.org/wiki/Context_switch

CPU Cache http://en.wikipedia.org/wiki/CPU_cache

affinity http://pypi.python.org/pypi/affinity

http://habrahabr.ru/post/144639/ Perl + Gui + log Parsing

http://amon.cx/

http://www.cyberciti.biz/tips/top-linux-monitoring-tools.html

http://dtrace.org/blogs/brendan/2011/12/16/flame-graphs/

http://dtrace.org/blogs/brendan/2012/03/07/the-use-method-linux-performance-checklist/

http://dtrace.org/blogs/brendan/2012/03/17/linux-kernel-performance-flame-graphs/

Performance Co-Pilot

http://www.oss.sgi.com/projects/pcp/

http://blahg.josefsipek.net/?p=438

While using top press “1” can show load of each cpu(core)

Profiling

http://www.pixelbeat.org/programming/profiling/

http://lttng.org/

http://code.google.com/p/jrfonseca/wiki/Gprof2Dot

http://goog-perftools.sourceforge.net/doc/cpu_profiler.html

http://www.reddit.com/r/Python/comments/ln85u/gprof2dot_a_python_script_to_convert_the_output/

System monitoring

http://www.tummy.com/journals/entries/jafo_20110925_154554

http://startuptools.pbworks.com/w/page/17974963/FrontPage

https://www.scoutapp.com/

http://en.wikipedia.org/wiki/List_of_performance_analysis_tools

http://stackoverflow.com/questions/1875167/performance-profiling-on-linux

Web GUI

http://phpsysinfo.sourceforge.net/

http://sourceforge.net/projects/procexp/

X-Window

http://conky.sourceforge.net/

http://sourceforge.net/apps/mediawiki/procexp/index.php?title=Main_Page python/PyQT

Python and shell scripts

https://github.com/arbuckle/linux-dash Python

https://github.com/afaqurk/linux-dash PHP (same logic as above)

http://scoutapp.github.io/scout_realtime/ RUBY

https://news.ycombinator.com/item?id=7375696 RUBY with GUI

https://github.com/Jahaja/psdash Python

https://github.com/abimaelmartell/system_monitor

https://github.com/k3oni/pydash depends from Django

http://nicolargo.github.io/glances/

There's no installation, just git clone and move to your www directory.

$ git clone https://github.com/afaqurk/linux-dash.git

$ mv linux-dash/ /var/www/

http://news.ycombinator.com/item?id=4372450

https://github.com/colinhowe/monner

https://github.com/pixelb/scripts/blob/master/scripts/ps_mem.py

https://bitbucket.org/travis_bear/quickstatd/src

http://colby.id.au/node/39/

#!/bin/bash

# by Paul Colby (http://colby.id.au)

PREV_TOTAL=0

PREV_IDLE=0

while true; do

CPU=(`cat /proc/stat | grep '^cpu '`) # Get the total CPU statistics.

unset CPU[0] # Discard the "cpu" prefix.

IDLE=${CPU[4]} # Get the idle CPU time.

# Calculate the total CPU time.

TOTAL=0

for VALUE in "${CPU[@]}"; do

let "TOTAL=$TOTAL+$VALUE"

done

# Calculate the CPU usage since we last checked.

let "DIFF_IDLE=$IDLE-$PREV_IDLE"

let "DIFF_TOTAL=$TOTAL-$PREV_TOTAL"

let "DIFF_USAGE=(1000*($DIFF_TOTAL-$DIFF_IDLE)/$DIFF_TOTAL+5)/10"

echo -en "\rCPU: $DIFF_USAGE% \b\b"

# Remember the total and idle CPU times for the next check.

PREV_TOTAL="$TOTAL"

PREV_IDLE="$IDLE"

# Wait before checking again.

sleep 1

done

http://www.centos.org/docs/5/html/Deployment_Guide-en-US/ch-proc.html

Each process has an entry in /proc/ with its pid. For Linux and BSD, threads inside a process are also processes (well light ones) and are located in /proc/%pid%/task/, with a content similar to the /proc/%pid% folder. So I only have to read stat to get the data I need, as well as statm for the memory consumption. Read the rest here:

http://matt.eifelle.com/2008/10/16/monitoring-cpu-usage-in-multithreaded-applications/

http://guichaz.free.fr/iotop/

http://code.google.com/p/psutil/

https://github.com/cgoldberg/linux-metrics#readme

http://dag.wieers.com/home-made/dstat/

http://www.pixelbeat.org/scripts/ps_mem.py

python ps_mem.py -p 46502 -w 2

Shell

http://www.pixelbeat.org/scripts/sysinfo

NMON

http://nmon.sourceforge.net/pmwiki.php?n=Site.CompilingNmon

Makefile

CFLAGS=-g -O2 -D JFS -D GETUSER -Wall -D LARGEMEM

LDFLAGS=-lncurses -g

FILE=lmon.c

nmon_power_sles11: $(FILE)

cc -o nmon_power_sles11 $(FILE) $(CFLAGS) $(LDFLAGS) -D POWER

http://www.ibm.com/developerworks/wikis/display/WikiPtype/nmon

https://www.ibm.com/developerworks/aix/library/au-nmon_analyser/

http://www.ibm.com/developerworks/wikis/display/WikiPtype/nmonconsolidator

nmon2web

http://ca.astound.net/~baspence/AIXtip/nmon2web.htm

http://www.itrus.com/WhitePapers/NMON_2_Web/html/readme.html

http://www.ibm.com/developerworks/forums/thread.jspa?threadID=426853

http://www.ibm.com/developerworks/cloud/library/cl-cloudclusterperform

ance/

https://www.ibm.com/developerworks/wikis/display/WikiPtype/pGraph

GUI to sar, nmon

http://sourceforge.net/projects/ksar/ Java

https://www.ibm.com/developerworks/wikis/display/WikiPtype/pGraph

http://honglus.blogspot.com/2011/02/graphing-sar-output.html

mpstat Processor related statistics

http://dtrace.org/blogs/brendan/2011/06/18/mpstat-videos/

mpstat -P ALL 2 5 displays 5 reports with 2 sec interval for all processors

vmstat memory and CPU activity

http://dtrace.org/blogs/brendan/2011/04/27/vmstat-videos/

Monitoring Frameworks

http://perfinsp.sourceforge.net/

http://oprofile.sourceforge.net/news/

http://serverfault.com/questions/44/what-tool-do-you-use-to-monitor-your-servers

http://en.wikipedia.org/wiki/Comparison_of_network_monitoring_systems

http://sixrevisions.com/tools/10-free-server-network-monitoring-tools-that-kick-ass/

http://habrahabr.ru/blogs/sysadm/134599/ RDDTools

http://mmonit.com/

http://munin-monitoring.org/

http://www.slac.stanford.edu/xorg/nmtf/nmtf-tools.html

http://sourceware.org/systemtap/wiki/HomePage

Plotting sar with RDDTool (perl) http://www.trickytools.com/php/sar2rrd.php

http://serverfault.com/questions/50772/is-there-a-paging-version-of-watch

http://blog.monitis.com/index.php/2011/02/22/11-top-server-management-monitoring-software/

http://serverfault.com/questions/313237/making-sense-of-ubuntu-server-status-info

http://www.orcaware.com/orca/

http://sixrevisions.com/tools/10-free-server-network-monitoring-tools-that-kick-ass/

http://www.monitortools.com/performance/ http://www.ahinc.com/aix/perform.htm http://www.net-snmp.org/docs/mibs/host.html http://software.intel.com/en-us/articles/intel-performance-counter-monitor/ http://support.hyperic.com/display/SIGAR/Home http://blog.scoutapp.com/articles/2009/07/31/understanding-load-averages

http://serverfault.com/questions/74863/linux-server-performance-monitoring

Performance Co-Pilot

http://blahg.josefsipek.net/?p=437

Ganglia : http://ganglia.info/

http://sourceforge.net/apps/trac/ganglia/wiki

http://www.ibm.com/developerworks/wikis/display/WikiPtype/ganglia

5 second sampling

http://blog.vuksan.com/2010/12/14/misconceptions-about-rrd-storage/

if you do not start gmetad you will see:

error collecting ganglia data (127.0.0.1:8652): fsockopen

Ganglia uses following defaults to create RRDs

RRAs "RRA:AVERAGE:0.5:1:244" "RRA:AVERAGE:0.5:24:244" "RRA:AVERAGE:0.5:168:244" "RRA:AVERAGE:0.5:672:244" "RRA:AVERAGE:0.5:5760:374"

This will create multiple circular buffers within the same RRD database file. In order to make sense out of this you need to know what the polling interval is ie. how often do you write into RRDs. In Ganglia's case the default is 15 seconds so

- "RRA:AVERAGE:0.5:1:244" says write actual values (:1:) for every polling interval. Save last 244 of those so in our case we'll have 61 minutes worth of actual data points. Since it's a circular buffer data older than 61 minutes will be "dropped"
- "RRA:AVERAGE:0.5:24:244" says average 24 values (:24:), 24 * 15 seconds = 360 seconds = 6 minutes. 244 of those times 6 is a whole day

You can modify the defaults in /etc/ganglia/gmetad.conf file. You can also use gmetad-python which allows you to write your own plugins and store metric data in both RRD format, SQL or any other storage engine of your choice.

killall gmetad

ls /etc/ganglia

conf.d gmetad.conf gmond.conf

/usr/sbin/gmetad

/usr/sbin/gmond

gmetad -h

gmetad 3.3.7

The Ganglia Meta Daemon (gmetad) collects information from

multiple gmond or gmetad data sources, saves the information to local

round-robin databases, and exports XML which is the concatentation of

all data sources

Usage: gmetad [OPTIONS]...

-h, --help Print help and exit

-V, --version Print version and exit

-c, --conf=STRING Location of gmetad configuration file

(default=`/etc/ganglia/gmetad.conf')

-d, --debug=INT Debug level. If greater than zero, daemon will stay in

foreground. (default=`0')

-p, --pid-file=STRING Write process-id to file

gmond -h

gmond 3.3.7

The Ganglia Monitoring Daemon (gmond) listens to the cluster

message channel, stores the data in-memory and when requested

will output an XML description of the state of the cluster

Usage: gmond [OPTIONS]...

-h, --help Print help and exit

-V, --version Print version and exit

-c, --conf=STRING Location of gmond configuration file

(default=`/etc/ganglia/gmond.conf')

-l, --location=STRING Location of this host in the cluster

'rack,rank,plane'. (default=`0,0,0')

-d, --debug=INT Debug level. If greater than zero, daemon will stay in

foreground. (default=`0')

-f, --foreground Run in foreground (don't daemonize) (default=off)

-t, --default_config Print the default configuration to stdout and exit

(default=off)

-m, --metrics Print the list of metrics this gmond supports

(default=off)

-b, --bandwidth Calculate minimum bandwidth use for configuration

(default=off)

-r, --convert=STRING Convert a 2.5.x configuration file to the new 3.x

format

-p, --pid-file=STRING Write process-id to file

Zabbix

http://habrahabr.ru/post/144477/

http://habrahabr.ru/post/137641/

Book: Zabbix 1.8 Network Monitoring

SYMON SYMUX

http://wpd.home.xs4all.nl/symon/index.html

http://genoverly.com/articles/15/

symon+symux->rrd

rrdbotd(snmp)->rrd

rrd->drraw рисовалка

CollectD

http://collectd.org/

http://blog.docker.io/2013/07/effortless-monitoring-with-collectd-graphite-and-docker/

http://habrahabr.ru/post/162087/

http://mailman.verplant.org/pipermail/collectd/

http://www.control-alt-del.org/2012/03/28/collectd-esper-amqp-opentsdbgraphite-oh-my/

https://groups.google.com/forum/?hl=en&fromgroups#!forum/collectd-web-users

man collectd

/usr/sbin/collectd -h

-C <file> Configuration file.

Default: /etc/collectd.conf

-t Test config and exit.

-T Test plugin read and exit.

-P <file> PID-file.

Default: /var/run/collectd.pid

-f Don't fork to the background.

-h Display help (this message)

Builtin defaults:

Config file /etc/collectd.conf

PID file /var/run/collectd.pid

Plugin directory /usr/lib64/collectd

Data directory /var/lib/collectd

/opt/collectd/sbin/collectd -h

Usage: collectd [OPTIONS]

Available options:

General:

-C <file> Configuration file.

Default: /opt/collectd/etc/collectd.conf

-t Test config and exit.

-T Test plugin read and exit.

-P <file> PID-file.

Default: /opt/collectd/var/run/collectd.pid

-f Don't fork to the background.

-h Display help (this message)

Builtin defaults:

Config file /opt/collectd/etc/collectd.conf

PID file /opt/collectd/var/run/collectd.pid

Plugin directory /opt/collectd/lib/collectd

Data directory /opt/collectd/var/lib/collectd

collectd 5.1.0, http://collectd.org/

/var/log # tail collectd.log

[2012-06-08 21:28:55] plugin_dispatch_values: No data sets registered. Could the types database be read? Check your `TypesDB' setting!

Yast

perl-Config-General

perl-Regexp-Common

rrdtool-dev

RRDTool plugin depends from librrd http://oss.oetiker.ch/rrdtool/doc/librrd.en.html

http://collectd.org/documentation/manpages/collectd.conf.5.shtml#plugin_rrdtool

How do I use --with-librrd?

If you installed librrd libraries in a non-standard (or non-system) path you need to specify them when running the configure script.

Otherwise it will not find them and build the binaries without linking against the library.

You need to set the PATH as given to the --prefix option when compiling the library. The script actually looks for the two subdirectories PATH/include and PATH/lib, so check for their existence if things don't work. If, for example, you installed RRDTool in /opt/rrdtool-x.y.z you need to run configure like this:

$ ./configure --with-librrd=/opt/rrdtool-x.y.z

$ ls /usr/lib64/librrd*

/usr/lib64/librrd.so.4 /usr/lib64/librrd.so.4.0.3 /usr/lib64/librrd_th.so.4 /usr/lib64/librrd_th.so.4.0.3

$ ls /usr/include/rrd.h

http://www.vitki.net/book/page/using-cacti-and-collectd-monitoring-centos

http://archive09.linux.com/feature/151982

http://habrahabr.ru/blogs/sysadm/93205/

http://dev-loki.blogspot.com/2012/02/collectd-on-opensuse.html

http://serverfault.com/search?q=collectd

http://www.hps.com/~tpg/notebook/collectd.php

http://www.cloudgurus.net/blog/2010/06/14/Collectd_part_1.html

http://jedi.be/blog/2012/01/04/monitoring-wonderland-visualization/

http://www.nickgerner.com/

http://perfwatcher.org/

How to configure collectd to manage cluster:

http://www.hps.com/~tpg/notebook/collectd.php

http://serverfault.com/questions/114750/how-to-combine-rrd-files-to-an-overview-graph-rrdtool-collectd

Writing plugin for collectd:

http://www.planetdevops.net/?p=14721

http://www.planetdevops.net/?p=14722

Web-Gui to CollectD

http://aprilmayjune.org/2009/09/16/collectd-drrawcgi-zoom-into-your-graphs-like-you-used-to-with-cacti/

http://web.taranis.org/drraw/

http://joemiller.me/2011/04/14/collectd-graphite-plugin/

http://gabenell.blogspot.com/2010/06/setting-up-collectd-collection3-on.html

http://holmwood.id.au/~lindsay/2009/09/08/graphing-collectd-statistics-in-the-browser-with-visage/

http://collectdweb.appspot.com/documentation/

http://octo.it/c4/

http://www.hps.com/~tpg/notebook/collectd.php

http://pommi.nethuis.nl/collectd-graph-panel-v0-3/

collection3 - gui for collectd:

http://mailman.verplant.org/pipermail/collectd/2009-March/002494.html

http://wiki.herzbube.ch/index.php/Collectd#collection3

http://trac.zentyal.org/wiki/Documentation/Community/HowTo/AdvancedMonitoring

cp -r collectd-5.10/contrib/collection3 /var/www

Modify File htpaccess.conf

Options Indexes FollowSymLinks MultiViews

AllowOverride all

Order allow,deny

allow from all

</Directory>

Using lighthttpd with collection3 http://www.vyatta.org/forum/viewtopic.php?t=2473

http://octo.it/c4/ - collection4

Collectl - about 1 sample every 10 seconds

http://collectl.sourceforge.net/

http://sourceforge.net/projects/collectl/files/collectl/collectl-3.6.3/

http://sourceforge.net/projects/collectl-utils/

http://sourceforge.net/projects/collectl-utils/files/collectl-utils-3.2.0/

http://rpm.pbone.net/index.php3/stat/4/idpl/18101644/dir/other/com/collectl-3.6.3-2.noarch.rpm.html

http://rpm.pbone.net/index.php3/stat/4/idpl/17891661/dir/other/com/collectl-utils-3.2.0-1.noarch.rpm.html

http://martincarstenbach.wordpress.com/2011/08/05/an-introduction-to-collectl/

http://collectl-utils.sourceforge.net/colmux.html

http://10.68.104.51:8080/colplot/

http://honglus.blogspot.com/2011/10/collectl-all-in-one-tool-for-collecting.html

# collectl -h

This is a subset of the most common switches and even the descriptions are

abbreviated. To see all type 'collectl -x', to get started just type 'collectl'

usage: collectl [switches]

-c, --count count collect this number of samples and exit

-f, --filename file name of directory/file to write to

-i, --interval int collection interval in seconds [default=1]

-o, --options options misc formatting options, --showoptions for all

d|D - include date in output

T - include time in output

z - turn off compression of plot files

-p, --playback file playback results from 'file'

-P, --plot generate output in 'plot' format

-s, --subsys subsys specify one or more subsystems [default=cdn]

--verbose display output in verbose format (automatically

selected when brief doesn't make sense)

Various types of help

-h, --help print this text

-v, --version print version

-V, --showdefs print operational defaults

-x, --helpextend extended help, more details descriptions too

-X, --helpall shows all help concatenated together

--showoptions show all the options

--showsubsys show all the subsystems

--showsubopts show all subsystem specific options

--showtopopts show --top options

--showheader show file header that 'would be' generated

--showcolheaders show column headers that 'would be' generated

--showslabaliases for SLUB allocator, show non-root aliases

--showrootslabs same as --showslabaliases but use 'root' names

> collectl --showheader

# Collectl: V3.6.0-3 HiRes: 1 Options: --showheader

# Host: coenewdb02 DaemonOpts:

# Distro: SUSE Linux Enterprise Server 11 (x86_64) Platform: System x3950 X5 -[7145AC1]-

# SubSys: cdn Options: Interval: 1 NumCPUs: 64 [HYPER] NumBud: 0 Flags: i

# Filters: NfsFilt: EnvFilt:

# HZ: 100 Arch: x86_64-linux-thread-multi PageSize: 4096

# Cpu: GenuineIntel Speed(MHz): 1064.000 Cores: 8 Siblings: 16 Nodes: 4

# Kernel: 2.6.32.36-0.5-default Memory: 1058823120 kB Swap: 52428116 kB

# NumDisks: 16 DiskNames: sdc sdb sda sdd sdg sdh sdi sdn sdm sdo sde sdj sdl sdp sdk sdf

# NumNets: 6 NetNames: lo: eth0:10000 eth1:10000 eth2:0 eth3:65535 usb0:

# SCSI: DA:0:01:00:00 DA:1:01:00:00 DA:2:01:00:00 DA:3:01:00:00 DA:4:00:00:00

1) Generate raw data file:

collectl -scjm -f /tmp/bla it generates cpu/sys/inter/ctxsw per processor

see "man collectl" for help. Example of flags for -s: (subsystems)

c - CPU

j - Interrupts

s - Sockets

d - Disk

m - memory

y - Slabs

-P generates data in plot format: summary goes into .tab file and the separate detail file is generated per subsystem

2) Generate plottable files from raw file:

collectl -p existing_filename_here.raw.gz -P -f /tmp -ocz

Here:

-p: playback file name

-P: generate output in plot format

-o: output options: c - create(overwrite existing file)

Colplot installation

/usr/share/collectl/plotfiles

http://home.clara.net/drdsl/OpenSource/perfmon.html

http://www.circle4.com/jaqui/papers/webunuk.html

http://docs.oracle.com/cd/B19306_01/server.102/b28051/tdppt_realtime.htm

http://docs.oracle.com/cd/B19306_01/server.102/b14211/ch23_os.htm

http://www.reddit.com/r/Python/comments/mshap/i_made_a_script_that_sends_nagios_data_to/

hwinfo - it one-ups, easily, lspci, lsusb and freakin' everything else in this field.

RDDTools Nagios

http://www.cacti.net/

https://www.icinga.org/

http://www.reddit.com/r/Python/comments/mshap/i_made_a_script_that_sends_nagios_data_to/

Circus

http://blog.ziade.org/2012/04/23/a-web-admin-for-circus/

http://circus.readthedocs.org/en/latest/index.html

GRAPHITE

http://www.aosabook.org/en/graphite.html

http://blog.tagged.com/2011/06/site-monitoring-tagged/

http://jondot.github.com/graphene/

https://gist.github.com/obfuscurity/c83e02b55e0288c72409

http://graphite.readthedocs.org/en/1.0/tools.html

https://lists.launchpad.net/graphite-dev/maillist.html

http://pypi.python.org/pypi/graphite-metrics/12.04.35

http://css.dzone.com/articles/python-getting-data-graphite-0

http://www.jedi.be/blog/2012/01/04/monitoring-wonderland-visualization/

http://coreygoldberg.blogspot.com/2012/04/installing-graphite-099-on-ubuntu-1204.html

http://blog.sflow.com/2012/01/graphite.html

http://codeascraft.etsy.com/2011/02/15/measure-anything-measure-everything/

http://www.robgolding.com/blog/2012/01/14/django-in-production-part-3---automation-and-monitoring/

http://www.devco.net/archives/2011/10/08/gdash-graphite-dashboard.php

http://news.ycombinator.com/item?id=3903292

Statsd

https://github.com/sivy/py-statsd#readme

https://github.com/armon/statsite

https://github.com/sivy/pystatsd

http://agiletesting.blogspot.com/2011/04/installing-and-configuring-graphite.html

https://github.com/obfuscurity/tasseo

RRDTool

http://oss.oetiker.ch/rrdtool/doc/rrdcached.en.html

http://net.doit.wisc.edu/~dwcarder/rrdcache/

http://habrahabr.ru/post/30655/

http://habrahabr.ru/post/135086/

http://habrahabr.ru/post/88100/

http://habrahabr.ru/post/134599/

Page updated

Google Sites

Report abuse