Mastering Linux

Why we want a master?

If we want several computer with the same system and parameters, it will be boring to do the same thing each time we want to do it. Insert a CDROM in every computers is boring too, thats why we use netbooting to install all computers.But the netboot doesn t reproduce the CDROM setup, we know the questions and answers.

What is a master

Basic or a non master

This is the simplest way to do a master.

get the CDROM of your favorite distribution, and use the same boot parameters. you ca adapt those parameters for nfs, or http for example.

This method is exactly the same as CDROM, you will answer to every questions.

It's not very interesting because every computer will be different(or can be), but useful if you want a CDOM sometime. It's very simple and very law cost in resources.

Vanilla

A vanilla Master is the very basic, simplest install of a distribution, nothing more, just the mandatory packages. It's like the CDROM install and we do next, answer to question like root passwords, no additional users, no packages, dhcp networking.

This is the base system for a better master.

Not vanilla

We add some packages, users and passwords (ldap, nis,...). We can create sub master for server computers and for desktop computers.

We can have master for:

• Web server
• Database server
• basic desktop (usual environment)
• developer desktop (with compiler, sources editor,...)

Usually the specificity of those master include the presetting for database, web server, and all services.

From vanilla to other flavor

Vanilla

In my opinion, it's just interesting as a basic master for creating new master (evolving), or recover booting. Vanilla master haven t any parameters, so every new system have the same hostname or root password.

We want a master different hostname for each config, it will be better if we can load those parameters from the boot command line or in a database.

For making this master, we install a new computer and make a backup. when we want to create a new computer we restore this master to the new computer.

More than Vanilla

This is the same, but we add some packages, configure some services. This is the step 1 just after an install.

Vanilla or not

Depends on you use case, but usually we never use vanilla master. In fact we use more than vanilla master but we name it as "Vanilla" because it will be our real basic configurations and all other master evolve from this one.

Masters or servants ?

Monolithic master

We install a server, make a complete backup with different tools:

• dump (dd, dump, xfsdump,..)
• rsync
• other backup software

We boot the computer in diskless mode and modify the partition table of the disk, make the filesystem, swap and mount the partitions. After we can restore the system

• restore (dd, restore, xfsrestore,...)
• rsync
• other backup software

After we install the master, we must change some settings like hostname, root password or may be add some packages. For doing this, we can use a post-mastering (before the reboot) script, or a first boot script.

This is a old fashion way to doing this, but very efficient and stable. Easy to maintain (evolve and use) because anything else is required. But any change make a variation for the next computer compared to the old one.

Dynamic master

Not really servants, just a joke.

We want a master different hostname for each config, it will be better if we can load those parameters from the boot command line or in a database. post mastering or fist boot scripts are interesting, but not for the whole config, just for little thing. We use gPXE booting with http server it will be better if we can send the whole config of the new computer in the command line, like this we are sure to install and reinstall the exact same server if we use the same (gPXE)boot parameters.

Now we will talk about real things, how can we use pxe boot scripting for installing our Linux.

Debian Master

Maintream possibilities

The debian distribution have a system for doing master installer, this is d-i (debian installer), it s use by all debian like system like ubuntu. We can also use debootstrap for bootstrap a computer, but fir this we must boot on a live linux and create partitions before start debootstrap.

In fact debootstrap is more interesting for installing virtual server/desktop, and it's a wrapper over d-i.

There are many other system for install debian but d-i is mainstream and very handy.

Debian installer

At first this is some web site for understanding more about d-i and preseed environment.

• Debian installer
• Preseed
• Debconf

Netboot with preseed

I add this in the gPXE boot file, and i select in the syslinux menu for install a Debian linux system.

LABEL Net install for Debian Desktop

  MENU LABEL  ^1 - Net install for Debian Desktop

  kernel http://bootserver.my.zone.tld/pxelinux.cfg/kernels/vmlinuz.debian

  APPEND initrd=http://bootserver.my.zone.tld/pxelinux.cfg/kernels/initrd.img.debian auto=true netcfg/choose_interface=eth0 priority=critical  preseed/interactive=false url=http://bootserver.my.zone.tld/pxelinux.cfg/preseed.cfg

When the debian start, the debian installer use the preseed.cfg settings and select the local, create the files systems, add users, add packages and more.

So if we install 100 computers, all of them are the same (and we are sure of that), but with different hostname.

Create the preseed file

This is an exemple of a preseed.cfg file for an ubuntu system with a french keyboard.

###

#  preseed_ubuntu.cfg

###

d-i debian-installer/locale string en_US

d-i debian-installer/locale select en_US.UTF-8

d-i languagechooser/language-name-fb    select English

d-i console-tools/archs string skip-config

d-i console-keymaps-at/keymap select fr-latin1

d-i debian-installer/locale string en_US

d-i console-tools/archs select at

d-i console-keymaps-at/keymap select Europe France

d-i console-keymaps/keymap string fr-latin1

d-i languagechooser/language-name-fb select French

d-i grub-installer/only_debian boolean true

d-i finish-install/reboot_in_progress note

popularity-contest popularity-contest/participate boolean 

Preseed on the fly

Now we can imagine to create on the file this kind of file with some template. i will explain how to do this, and what is necessary to templating and what are static.

I will split the file in many parts and include all of them for creating the preseed.ini file.

• preseed_ubuntu.cfg
• preseed_mirror.cfg
• preseed_disk.cfg
• preseed_config.cfg
• preseed_package.cfg
• debconf_setup.cfg
• debconf_command.cfg

Until my php applet is good enough, i will not publish it, but i can tell you how it works. You can see above the part of the preseed file i send by http, i insert some comment that specify the include file, so all the file can be static but i loved to have a dynamic preseed_package.cfggenerate from this pressed.ini.

;;;

; tasksel:      enter all meta-packages separated by commas

; packages:     enter all packages separated by space

; including     only for the target host (not recursive)

;;;

[defaults]

        config_files    = preseed_ubuntu.cfg preseed_mirror.cfg preseed_disk.cfg preseed_config.cfg debconf_setup.cfg debconf_command.cfg

        tasksel         =

        packages        = openssh-server debconf-utils gpm quotatool snmpd snmp puppet augeas-tools

        distrib         = maverick

        disk_device     = /dev/sda

        host_type       = desktop

;

[maverick]

        mirror          = mirror.my.zone.tld

        distrib_path    = /ubuntu

;

[lucid]

        mirror          = mirror.my.zone.tld

        distrib_path    = /ubuntu

;

[desktop-env]

        packages        = kde-full

;

[kde]

        packages        = kde-devel 

;

[desktop]

        including       = kde desktop-env dev_pkg

        tasksel         = Ubuntu desktop, OpenSSH server, LAMP server

        disk_device     = /dev/sda

        desktop         = kde

        packages        = ocsinventory-agent ffmpeg vlc ubuntu-virt-mgmt virt-top kvm-pxe chromium-browser-l10n

;

[server]

        tasksel         = OpenSSH server, LAMP server

        packages        = mysqltuner samba-tools

;

[minimal]

        tasksel         = OpenSSH server

        packages        =

;

[dev_pkg]

        packages        = xulrunner-dev libacl1-dev libaio-dev libsnmp-dev 

The php script load this ini file and create the preseed like this if wee choose [desktop] in the boot sequence,

http://bootserver.my.zone.tld/pxelinux.cfg/preseed.cfg/preseed.php?profile=desktop

• include [kde] [desktop-env] [dev_pkg]
• include all config_files from [defaults]
• [defaults] -> include [maverick]

so the virtual preseed.ini file is like this one.

[desktop]

        config_files    = preseed_ubuntu.cfg preseed_mirror.cfg preseed_disk.cfg preseed_config.cfg debconf_setup.cfg debconf_command.cfg

        mirror          = mirror.my.zone.tld

        distrib_path    = /ubuntu

        distrib         = maverick

        host_type       = desktop

        disk_device     = /dev/sda

        tasksel         = Ubuntu desktop, OpenSSH server, LAMP server

        disk_device     = /dev/sda

        desktop         = kde

packages = ocsinventory-agent ffmpeg vlc ubuntu-virt-mgmt virt-top kvm-pxe chromium-browser-l10n xulrunner-dev libacl1-dev libaio-dev libsnmp-dev openssh-server debconf-utils gpm quotatool snmpd snmp puppet augeas-tools kde-full kde-devel

The "virtual" debconf_packages.cfg is create from the packages and tasksel lines of this ini file. The other part are basicaly included from the template.

A simplest php generator can be like this.

http://bootserver.my.zone.tld/pxelinux.cfg/preseed.cfg/preseed.php?profile=desktop

<?php

  header ( "Content-type: text/plain" );

  $host_type = $_GET["profile"];

  include("preseed_ubuntu.cfg");

  include("preseed_mirror.cfg");

  include("preseed_disk.cfg");

  include("preseed_config.cfg");

  include("preseed_packages-". $host_type .".cfg");

  include("debconf_setup.cfg");

  include("debconf_command.cfg");

?>

And we create all the preseed_packages-*.cfg files with statics entries. It's simple and enough for many usecase.

Redhat like Master

Maintream possibilities

Redhat and Centos have kickstart for unattended install. IT a very interesting way, and usefull. if you have an Redhat/Fedora/Centos, you can find without doing anything a kickstart config file.

We can also use another system for this like dump/restore but it s not interesting at all.

Kickstart

The lazy way

Find a Redhat setup and get the /root/anaconda-ks.cfg file. This file is valid setup for duplicate this server.

The handy way

Look at the documentation and create some kickstart config.

You can choose at boot time which one you want.

• Fedora_last-ks.cfg => for a Fedora Desktop
• Centos_5.5-ks.cfg => for a Centos Server
• Redhat_6.0-ks.cfg => for a Redhat 6.0 Server

You can use %include for the common parts, but it can be useful to use a template and web php scripts like debian.

The cobbler way

I am not a user of cobbler, but it's interesting, perhaps i will talk about it in the future.

Have a look at the web site and enjoy.

Kickstart Example

# Kickstart file automatically generated by anaconda.

#version=RHEL6

install

url --url=http://bootserver.my.zone.tld/Redhat/6.0/

lang en_US.UTF-8

keyboard fr-latin1

network --device eth0 --bootproto dhcp

rootpw  --iscrypted $1$XXXXXXXXXXXXXXXXXXX

reboot

firewall --disabled

authconfig --useshadow  --passalgo=md5 

selinux --disabled

timezone --utc Europe/Paris

bootloader --location=mbr --append="console=ttyS0 crashkernel=auto"

%packages

@Base

@Core

@base

@compat-libraries

@console-internet

@core

@debugging

@development

@directory-client

@hardware-monitoring

@java-platform

@large-systems

@network-file-system-client

@nfs-file-server

@performance

@server-platform

@server-policy

%end

Gentoo Master

Source distrib limits

Nothing, in fact thinking you can have a master in a gentoo environment is a little strange, because it s a source distrib, and with the exact same setup, you will generate a different server all around the week, because new version of packages and dependencies is reveled.

Maintream possibilities

You can create binaries repositories and generate all your setup from a "master server".

• Install a server (gentoo or not)
• Create a chroot env, mkdir /chroot
• untar stage3 and snaphot in the chroot
• mount the proc, dev partition
• chroot to this directory
• /usr/portage/scripts/bootstrap.sh
• emerge -e world
• emerge all additional packages
• quickpkg (for creating all the binary packages)

when you install a server to the same, but in the make.conf file use

• PORTAGE_BINHOST="http://myserver.my.zone.tld/entoo-packages/"
• EMERGE_DEFAULT_OPTS="-q -j3 -gK"

That's all.

When you update your master, you can update other computers. but with this method, all computers (server and desktop) have the same USE flags, or you must have severals master, and it's very expensive in time and disk space.

Create a master is very long, around 1 day or more for gentoo, and less than 1 hour for Redhat or debian.

Conclusions

With this methods we can install and setup many servers and computers, all is unattended and create reproductive setup. the costs of this infrastructure is only 1 sever (dhcpd, gPXE, DNS, Web server, and some distrib mirrors but it s not mandatory)