custom_xcp

The Kernel Compile Guide

Install required packages

sudo apt-get install libncurses5-dev kernel-package

Create a source directory

mkdir ~/src

cd ~/src

Download the kernel source into your source directory

apt-get source linux-image-$(uname -r)

Configure your Kernel

cd linux-3.2.0

make menuconfig

Speed up the build

export CONCURRENCY_LEVEL=2

General rule is CONCURRENCY_LEVEL = number of processor cores + 1

Compile the Kernel

time fakeroot make-kpkg --initrd --append-to-version=-tweak kernel-image kernel-headers

You can change -tweak to anything you wish

Install your compiled Kernel

cd ~/src

sudo dpkg -i linux-image-3.2.24-tweak_3.2.24-tweak-10.00.Custom_amd64.deb

sudo dpkg -i linux-headers-3.2.24-tweak_3.2.24-tweak-10.00.Custom_amd64.deb

Reboot

Ubuntu should boot with your new Kernel by default, if not you may have to edit Grub or enable the Grub menu to be displayed during boot.

Introduction

Xen is a type 1, bare-metal virtual machine monitor (or hypervisor), which provides the ability to run one or more operating system instances on the same physical machine. Xen, like other types of virtualization, is useful for many use cases such as server consolidation and isolation of production and development environments (Eg.: corporate and personal environments on the same system).

As of Ubuntu 11.10 (Oneiric), the default kernel included in Ubuntu can be used directly with the Xen hypervisor as the management (or control) domain (dom0 or "Domain0" in Xen terminology).

Our example uses LVM for virtual disks and network bridging for virtual network cards. It also assumes Xen 4.1 (the version available in 12.04). It assumes a familiarity with general virtualization issues, as well as with the specific Xen terminology. Please see the Xen wiki (see http://wiki.xen.org/wiki/Xen_Overview) for more information.

During Installation of Ubuntu

- Note: For VMware Workstation test users: On VM settings enable "Virtualize Intel VT-x/EPT or AMD-V/RVI"

During the install of Ubuntu for the Partitioning method choose "Guided - use the entire disk and setup LVM". Then, when prompted to enter "Amount of volume group to use for guided partitioning" enter a value large enough for the Xen dom0 system, leaving the rest for virtual disks. Enter a value smaller than the size of your installation drive. For example 100 GB should be large enough for a minimal Xen dom0 system. Keep in mind that in our model stay inside that guest (dom0) all installation media for guest OSs and other useful files, so that guest must have enough space on it.

After Installation of Ubuntu

Install GUI

sudo apt-get update sudo apt-get install ubuntu-desktop

To skip the login screen completely, boot into the console and then start the GUI

sudo gedit /etc/default/grub

Change line GRUB_CMDLINE_LINUX_DEFAULT="quiet splash" to GRUB_CMDLINE_LINUX_DEFAULT="text".

Save and exit.

sudo update-grub

Reboot and you should come up directly in tty1.

To logoff "Unity" (default Ubuntu desktop) from command line type "gnome-session-quit".

Install Windows Remote Desktop

We understand that this is the best approach to have remote access to "Ubuntu Server 12.04 LTS" into a cross-plataform environment. The "XRDP" is an implementation of the "Remote Desktop" standards from Microsoft and works on the same way as for Windows. Allows remote desktop access via native Windows client machines (or "RDESKTOP" on Ubuntu), does not require loading the "Ubuntu Server 12.04 LTS" GUI (Graphical User Interface) on boot and allows multiple simultaneous sessions. To use "RDESKTOP" on Ubuntu 12.04 LTS with "TSCLIENT" see my post on http://superuser.com/questions/420291/ubuntu-12-04-how-to-get-tsclient-back.

With "XRDP" you can easily use Microsoft RDP to connect to Ubuntu without any configuration. All you need to do is install the "xrdp" package, then open Remote Desktop Connection from Windows and connect. That's it, nothing to configure.

Without wasting anymore of your time, let’s get going.

- sudo apt-get install xrdp

Next, open Windows Remote Desktop Connection (RDP) and type Ubuntu Server hostname or IP address.

Off Course You Can Use the Great SSH

As you may already know, SSH is a secure communication protocol that lets you remotely access networked computers. It is known as a replacement for Telnet which is very unsecure. While Telnet sends traffic in plain text, SSH on the other hand uses a secure protocol to communicate.

Run the commands below to install SSH Server.

- sudo apt-get install openssh-server

To log in on remote machine type on your terminal

- ssh <remote_user>@<ip_or_name>

Installing Xen (XCP - Xen Cloud Platform)

XCP (Xen Cloud Platform) is the open source version similar to Citrix XenServer that uses the Xen Hypervisor. XCP uses XAPI or XenAPI to manage Xen hosts. XCP is based on CentOS 5.5.

Project Kronos is an initiative to port the XAPI tool stack to Debian and Ubuntu. It is a management stack implemented in OCaml that configures and controls Xen hosts, attached storage, networking and virtual machine life cycle. It exposes a HTTP API and provides a command line interface (xe) for resource management.

XenCenter is Windows desktop application by Citrix that is distributed with XenServer for managing servers running XenServer (the equivalent of linux is OpenXenManager). It uses XAPI for talking to Xen resource pools. Since we are setting up XAPI, we can use XenCenter to manage the server.

Why use XCP-XAPI on Debian/Ubuntu when XCP appliance exists?

- Manage dom0 using a configuration management framework (Puppet, Chef...)
- Apply security updates to dom0 root file system
- Run Xen version 4.1.
- Ubuntu Server 12.04 is a LTS release that is supported for 5 years

Installing and configuring Xen Hypervisor

- Install the Xen Hypervisor
  - - sudo apt-get install xen-hypervisor
- Setup GRUB to boot the Xen Hypervisor
  - - sudo sed -i 's/GRUB_DEFAULT=.*\+/GRUB_DEFAULT="Xen 4.1-amd64"/' /etc/default/grub
- Disable apparmor at boot
  - - sudo sed -i 's/GRUB_CMDLINE_LINUX=.*\+/GRUB_CMDLINE_LINUX="apparmor=0"/' /etc/default/grub

- - Restrict "dom0" to 1GB of memory and 1 VCPU (example) (see "Why should I dedicate fixed amount of memory for Xen "dom0"?")
    - sudo gedit /etc/default/grub
    - After GRUB_CMDLINE_LINUX="apparmor=0" add the line GRUB_CMDLINE_XEN="dom0_mem=1G,max:1G dom0_max_vcpus=1"

- Update Grub with the config changes we just made
  - - sudo update-grub
- Reboot the server so that Xen boots on the server
  - - sudo reboot
- Once the server is back online ensure that Xen is running
  - - cat /proc/xen/capabilities should display "control_d"
      - Note: To stop or start xcp-xapi sudo /etc/init.d/xcp-xapi stop (or start)
- Installing and configuring XAPI (XenAPI)
  - - Install XCP-XAPI
      - sudo apt-get install xcp-xapi
        -> Choose "bridge" when prompted for network backend
    - Setup the default toolstack
      - sudo gedit /etc/default/xen
        -> Set "TOOLSTACK=xapi"
    - Disable xend from starting at boot
      - sudo sed -i -e 's/xend_start$/#xend_start/' -e 's/xend_stop$/#xend_stop/' /etc/init.d/xend
      - Note: Only xend the deamon needs to be disabled from starting, "/etc/init.d/xend" handles other things like modules and xenfs. Do not disable it from the runlevel. Disable service xendomains
        sudo update-rc.d xendomains disable
    - Fix for "qemu" which emulates the console does not have the keymaps in the correct location
      - sudo mkdir /usr/share/qemu; sudo ln -s /usr/share/qemu-linaro/keymaps /usr/share/qemu/keymaps
    - Network configuration
      - This section describes how to set up Linux bridging in Xen. It assumes eth0 is both your primary interface to dom0 and the interface you want your VMs to use. It also assumes that you will use manually IP configuration.
        sudo apt-get install bridge-utils
        Note: If not already installed
        As you are working with a desktop install, disable "Network Manager"
        sudo stop network-manager
        sudo gedit /etc/NetworkManager/NetworkManager.conf
        Now make sure the "managed" line contains "managed=false" Save and exit
        Since Network Manager is not managing your network interfaces anymore, you must manually enter that information. Below we explain how.
        Setup bridge networking
        sudo gedit /etc/network/interfaces
        Create a bond called xenbr0. The file should look like this for a static network configuration:
        # This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). # The loopback network interface auto lo iface lo inet loopback # Xen network interface for "dom0" auto xenbr0 iface xenbr0 inet static # IP address address 192.168.1.111 # Subnet mask netmask 255.255.255.0 # Default Gateway gateway 192.168.1.1 # DNS Server dns-nameservers 192.168.1.1 bridge_ports eth0 iface eth0 inet manual # The primary network interface # auto eth0 # iface eth0 inet dhcp
        Configure xcp to use "bridge" networking instead of "openswitch"
        sudo gedit /etc/xcp/network.conf
        -> Replace “openswitch” with “bridge”
        Note: For VMware Workstation test users: - Configure with "bridge" for network adapter and run these comands on host: sudo chmod ugo+rwx /dev/vmnet0 sudo chown <username> /dev/vmnet0 sudo chown :<usergroup> /dev/vmnet0 Eg.: sudo chmod ugo+rwx /dev/vmnet0 sudo chown eduardo /dev/vmnet0 sudo chown :eduardo /dev/vmnet0
        To test whether the network is working run the command
        sudo /etc/init.d/networking restart
    - All set! Ready to reboot and let xcp-xapi toolstack take over
      - sudo reboot
    - On restart confirm that xcp is working
      - sudo xe vm-list
        This should list the control domain
        " uuid (RO) : dbcf74d2-ee50-edd5-d44d-b81fc8ba1777 name-label (RW): Control domain on host: ubuntu-xenserver-1 power-state (RO): running "
        -> If your output looks similar, "xapi" is running on the server, if you get “Connection refused” then xapi is not setup correctly!
    - Configure Storage Repository for Use With XAPI
      - NFS servers are a common form of shared filesystem infrastructure, and can be used as a storage repository substrate for virtual disks. As NFS storage repositories are shared, the virtual disks stored in them allow VMs to be started on any server in a resource pool and to be migrated between them using XenMotion. When you configure an NFS storage repository, you simply provide the hostname or IP address of the NFS server and the path to a directory that will be used to contain the storage repository (if this resource is on another machine, it is not our case). The NFS server must be configured to export the specified path to all servers in the pool.
        To show your volume group (VG)
        sudo pvs
        Create a LV with X GBs
        sudo lvcreate -L <X>GB -n <StorageRepositoryName> /dev/<VG>
        Eg1.: sudo lvcreate -L 25GB -n StorageRepository /dev/ubuntus1204 Eg2.: sudo lvcreate -l 100%FREE -n StorageRepository /dev/ubuntus1204
        Register the logical volume for use with XAPI
        sudo xe sr-create type=ext shared=true name-label=<StorageRepositoryName> device-config:device=/dev/<VG>/<StorageRepositoryName>
        Eg.: sudo xe sr-create type=ext shared=true name-label=StorageRepository device-config:device=/dev/ubuntus1204/StorageRepository
        This should display the Storage Repository
        sudo xe sr-list name-label=<StorageRepositoryName>
        Eg.: sudo xe sr-list name-label=StorageRepository
        " uuid ( RO): 37bc5263-c9fc-8876-d24c-d5927f1bbed2 name-label ( RW): StorageRepository name-description ( RW): host ( RO): ubuntus1204 type ( RO): ext content-type ( RO): "
    - Configure a ISO Repository for Use With XAPI
      - An ISO Repository contains ISOs (disk images) with operational systems to perform the installations.
        Then the following example makes a storage repository called ISOs
        sudo xe sr-create name-label=<LocalISORepositoryName> type=iso shared=true device-config:location=<FolderPath> device-config:legacy_mode=true content-type=iso
        Eg.: sudo mkdir -p /var/opt/xen/LocalISORepository/ sudo xe sr-create name-label=LocalISORepository type=iso shared=true device-config:location=/var/opt/xen/LocalISORepository/ device-config:legacy_mode=true content-type=iso
        This should display the ISO Repository
        sudo xe sr-list name-label=<LocalISORepositoryName>
        Eg.: sudo xe sr-list name-label=LocalISORepository
        " uuid ( RO): 26edb27b-72fc-af56-ad2f-4d15a8d8e3f7 name-label ( RW): LocalISORepository name-description ( RW): host ( RO): ubuntus1204 type ( RO): iso content-type ( RO): iso "

Download and install XenCenter (Windows)

Go to http://citrixxperience.com/2012/02/24/download-and-install-citrix-xencenter-6-0/ and download the ISO "XenServer-6.1-install-cd.iso" (if the version is 6.1, of course).

- The installer "XenCenter.msi" is inside. Note that this ISO is also the installation CD of XenCenter.

Start XenCenter and click "Add New Server". Put in the server ip, username and password as setup during installation, let the magic begin!

Download and install OpenXenManager (Alternative to XenCenter) (Ubuntu 12.04 Server LTS)

OpenXenManager is a free clone of the XenCenter and is supposed to be fully compatible with both XenServer and the freeware XCP. The program comes bundled as a bunch of Python scripts, contained in a single archive, so the deployment is a breeze pretty much everywhere. You won't get any visual integration with your desktop theme, though.

You might want to consider this program if you're going for an all-free combo setup with XCP perhaps, or if you require GUI management from Linux, as OpenXenManager does run on non-Windows machine, unlike XenCenter, which is for Windows only. Now, the official page looks a little out of date, so you might want to take long-term support into consideration, too. But enough talking, let's OpenXenManager in action.

Installation

You can download OpenXenManager from http://sourceforge.net/projects/openxenmanager/ or Install it on your Ubuntu Server 12.04 LTS machine using the following command:

sudo apt-get install openxenmanager

Ubuntu Installation

I used the latest Ubuntu 12.10 amd64 server ISO from ubuntu.com.

For example 10 GB or even 5 GB should be large enough for a minimal Xen Dom0 system. Entering a percentage of maximum size (e.g. 25%) is also a reasonable choice.

For everything else, just defaults are fine, and recommend to install OpenSSH, but nothing else from the installer package selections.

Install the pre-requisites

apt-get install python-dev

apt-get build-dep xen

apt-get install libc6-dev libglib2.0-dev libyajl-dev yajl-tools libbz2-dev bison flex zlib1g-dev git-core texinfo debhelper debconf-utils debootstrap fakeroot

Download and build the source

wget http://bits.xensource.com/oss-xen/release/4.2.1/xen-4.2.1.tar.gz

tar zxvf xen-4.2.1.tar.gz

cd xen-4.2.1

./configure

vi ./.config

PYTHON_PREFIX_ARG=--install-layout=deb

make world

make deb

This creates a "dist" directory which contains the results of your build.

Install

cd ./dist

./install.sh

# Important

update-grub

ls -al /boot/xen*

# You'll have something like the below

-rw-r--r-- 1 root root 802314 Feb 18 09:11 /boot/xen-4.2.1.gz

lrwxrwxrwx 1 root root 12 Feb 18 09:11 /boot/xen-4.2.gz -> xen-4.2.1.gz

lrwxrwxrwx 1 root root 12 Feb 18 09:11 /boot/xen-4.gz -> xen-4.2.1.gz

lrwxrwxrwx 1 root root 12 Feb 18 09:11 /boot/xen.gz -> xen-4.2.1.gz

-rw-r--r-- 1 root root 15388780 Feb 18 09:11 /boot/xen-syms-4.2.1

# Setup the init scripts, purposely skipping xend since 'xl' is the standard now

update-rc.d xencommons defaults 19 18

update-rc.d xendomains defaults 21 20

update-rc.d xen-watchdog defaults 22 23

grep Xen /boot/grub/grub.cfg

# You should see "Ubuntu GNU/Linux, with Xen hypervisor" in there, that's where I got the below from

# Now set it as the boot default so we have Xen upon rebooting

sed -i 's/GRUB_DEFAULT=.*+/GRUB_DEFAULT="Ubuntu GNU/Linux, with Xen hypervisor"/' /etc/default/grub

sudo dpkg-divert --add --rename --divert /etc/grub.d/09_linux_xen /etc/grub.d/20_linux_xen

# Update grub with the new default

update-grub

# Reboot

reboot

# You know it's working if 'xl' is working

xl list

Name ID Mem VCPUs State Time(s)

Domain-0 0 7660 2 r----- 6.9

Create a test VM

xen-create-image is very handy for a quick way to get a VM running:

xen-create-image --hostname=blah

--memory=512mb

--vcpus=2

--lvm=some_lvm_volume_group_with_free_space

--dhcp

--pygrub

--dist=precise

Re-use the files

You can save that 'dist' directory and install it on other machines for the future.

tar zcvf xen-dist.tgz ./xen-4.2.1/dist

Make deb

You'll notice above I used 'make deb', which you can technically skip if you want. All that did was create 'dist/xen-upstream-4.2.1.deb', which is a helper debian package that includes a list of all of the files in dist/.

This package doesn't really do anything, it's just there to help you keep track of the Xen files you've manually installed. You can install the package like this:

dpkg -i ./xen-upstream-4.2.1.deb

$ ldconfig -p |grep -i xen

If there is no output, it means that your lib path is notincluded into ld configuration, please find ‘libxenstore.so’ to confirmlocation first:

$ find / -name‘libxenstore.so*’

Then you will see the location and assume it is ‘/usr/lib64’,then create file ‘/etc/ld.so.conf.d/lib64.conf’:

$ touch /etc/ld.so.conf.d/lib64.conf

$ echo '/usr/lib64'>> /etc/ld.so.conf.d/lib64.conf

$ /sbin/ldconfig/etc/ld.so.conf