Gentoo Clustering

How To Create A Linux Cluster Using Gentoo Linux 

I have decided on a course of action for the clustering of the network. The documentation is organized in the table of contents. 

Table of Contents

Part I: Installation of Gentoo on the headnode
Part II: DHCP Server Configuration

References

[1] High Performance Computing on Gentoo Linux
[2] Diskless Nodes with Gentoo
[3] Host Based Authentication using OpenSSH
[4] openMosix Cluster on Gentoo
[5] Gentoo Linux rsync Mirrors Policy and Guide
[6] Gentoo Distcc Documentation
[7] Gentoo Linux x86 Handbook
[8] HOWTO Setup NIS 
[9] Gentoo - LTSP Guide
[10] Gentoo Linux Cron Guide

 

My Thoughts Before Having Started

A small project that I'm currently working on is to deploy the Gentoo Linux OpenMosix kernel sources to 58 IBM Blade machines in the University of Toronto Blade cluster.

This project has turned out to be quite challenging and will require a significant amount of work. I do however believe that the information contained here will be useful to other individuals who would like to set up a cluster on their university servers should they have the chance. 

There are in fact several goals that I am trying to keep in mind while setting up the system. The challenge lies in the fact that I have 58 separate machines to which I would like to deploy a customized Gentoo kernel. More importantly, I would like to be able to manage all 58 blades simultaneously. It would be nice if when an update or patch were applied to one node, those changes could be propagated to all other nodes in the network. I have several ideas for this approach:

First Approach for Rapid Deployment

My first approach would be to create an installation profile for the Gentoo installer which is a new feature of Gentoo with release 2006.0. This installation profile will include a kernel configuration file for the particular kernel that I would like deployed on the blades. As it stands, the openMosix kernel patches are only available for Linux 2.4. The patch for 2.6 is still in beta and I would like for my installation to be as stable as possible.

With an installation profile, the Gentoo installation should proceed without too much intervention - preferably with no intervention. The only problem is that I will have to emerge the openMosix daemon and monitoring tools for a large number of machines manually. This will be time consuming - but not quite as time consuming as installating a separate Gentoo installation for every blade.

In order to streamline the delivery of the installation, I will have to setup a network boot server (TFTP) on the master node and boot every node from the master node. This, I don't think, should pose much of a problem since there are many tutorials for network booting a Linux installation. 

Second Approach for Rapid Deployment

Failing the first approach, the easiest thing to do would be to simply perform one complete installation of a single node configured exactly the way I would like and then to make an image of the disk. Since all the blade are identical machines, it would be a simple task to clone the image on all the drives for all the blades in the rack. This solution is attractive since it requires no boot server and no mucking around with the Gentoo installer.