LeoTask: Lightweight-Productive-Reliable parallel task running and results aggregation (MapReduce on multicore)

Post date: Nov 21, 2015 9:48:17 AM

LeoTask is a parallel task running and results aggregation (MapReduce) framework. It is a free and open-source project designed to facilitate running computational intensive tasks [1]. The framework implements the MapReduce model, allocating tasks to multi-cores of a computer and aggregating results according to a XML based configuration file. The framework includes mechanisms to automatically recover applications from interruptions caused by accidents (e.g. Power Cut). Applications using the framework can continue running after an interruption without losing its calculated results.

For more details please visit the framework's homepage: http://mleoking.github.io/LeoTask/

Download | Introduction | Applications | Discussion

Features:

• • Automatic & parallel parameter space exploration.

  • Flexible & configuration-based result aggregation.

  • Programming model focusing only on the key logic.

  • Reliable & automatic interruption recovery.

  • Ultra lightweight ~ 300KB Jar.

Utilities:

• • All dynamic & cloneable networks structures: a node, a link, a network, a network set (within which networks can overlap with each other), multiplex networks.

  • Integration with Gnuplot: hybrid programming with Gnuplot, output statistic results as Gnuplot scripts.

  • Network generation according to common network models: random networks, scale-free networks, etc.

  • DelimitedReader: a sophisticated reader that explores CSV (Comma-Separated Values) files like a database.

  • Fast random number generator based on the Mersenne Twister algorithm.

  • Versatile curve fitter and function value optimizer (minimizer).