Virtualization

As a strike against the industry leading VMWare, Citrix releases it's Xen Enterprise Virtualization platform to the public for free. Note this does not mean it's been released OpenSource as the Xen hypervisor is, it's just free to download and use on as many installations as you like. The difference is that it's free as in beer, not as in freedom. If Citrix wanted to change the license back to pay only after it hooks a million customers it can.

BOSTON » 2/23/2009 » Citrix Systems, Inc. (Nasdaq: CTXS), the global leader in application delivery infrastructure, today unveiled a groundbreaking new version of Citrix® XenServer™ – the company’s enterprise-class, cloud-proven virtualization platform – that will be offered free of charge to any user for unlimited production deployment.  While basic hypervisors have been free for years, most have had limited practical use in real world environments.  With this new release, XenServer sets an entirely new standard for free virtualization with the addition of powerful new features like centralized multi-node management, multi-server resource sharing and full live motion.  Dramatically lowering the entry price of virtualization also helps address today’s challenging economic climate by making enterprise-class virtualization far more accessible to businesses of all sizes, regardless of budget.

Xen Enterprise is running in 5000 datacenters worldwide. Downloads will be available in March. Here's the official announcement.

Scenario: In the Dom0 (Host) you have a file that you export to the DomU (Guest) and it appears as an entire hard drive and you want to make it larger.

Example- Dom0: /srv/xen/diskimage.img -> DomU: /dev/xvda

If you're using diskimages for your DomU drives you may need to increase their size if any of the DomU partitions become full.

Resize the Xen Diskimage in Dom0

1. Create a backup of the diskimage from Dom0

2. Shutdown the DomU

3. Add extra space to the diskimage using dd. This will add 1GB to the DomU image. Adjust count=  depending on how much you want to add. If you want a sparse file use seek= to define the entire disk size.

dd if=/dev/zero bs=1M count=1024  >> ./diskimage.img

or if you want a sparse file

dd if=/dev/zero bs=1 count=0 seek=1G  >> ./diskimage.img

4. Boot the domU

Your disk should now be larger. You will need to use traditional tools inside the DomU to make the partitions and filesystems larger.

Following are examples for Partitions and LVM.

Expanding DomU Partitions from within DomU

In this example we're using /dev/xvda as the example DomU device name, change this depending on your setup. Note this tutorial only works for resizing the last partition on the diskimage drive.

1. Start the DomU and log in as root

2. Start fdisk /dev/xvda

3. Delete the last partition and recreate it with the start cylinder being identical as before and the ending cylinder being the default (end of disk)

4. Exit fdisk

5. You may have to reboot the DomU before going on.

5. Resize the filesystem on the partition - resize2fs /dev/xvda1

That's really it! You can only hot resize if the Filesystem is getting larger. If you need to shrink it then you'll have to take the Volume offline first. Isn't this easier than dealing with partitions that are too small?

If the partition you want to resize is in the middle of the DomU drive you're in a bit of a pickle. For example if you want to resize / you have problems.

• /boot - /dev/xvda1
• / - /dev/xvda2
• /var - /dev/xvda3

This is the primary reason to using LVM. The solution to this problem isn't very elegant. You basically need to make another disk image and attatch it to the DomU in exactly the same manner as you attached /dev/xvda. The new drive should appear as /dev/xvdb if that's the way we entered it in the DomU config. Once it's done you need to restart DomU, fdisk and format the drive. Once formatted you can mount it and copy all of /var over, change /etc/fstab to map /var to /dev/xvdb1 and reboot the DomU again. Once rebooted you can delete /dev/xvda3 and resize /dev/xvda2.

This process is really no different than if you had a real server but you don't have to install a physical hard drive. I think this shows why LVM is such an improvement over physical partitions.

Scenario: In the Dom0 (Host) you have a file that you export to the DomU (Guest) and it appears as a hard drive partition and you want to make it larger.

Example- Dom0: /srv/xen/diskimage.img -> DomU: /dev/xvda1

Resize the Disk Image in Dom0

1. Shutdown the DomU

2. Create a backup of the Disk Image using the cp command

3. Add extra space to the diskimage using dd. This will add 1GB to the DomU image. Adjust count= depending on how much you want to add. If you want a sparse file use seek= to define the entire disk size.

dd if=/dev/zero bs=1M count=1024 >> ./diskimage.img

or if you want a sparse file

dd if=/dev/zero bs=1 count=0 seek=1G >> diskimage.img

4. Boot the domU

Your view /proc/partitions your disk should now be larger. You will need to use traditional tools inside the DomU to make the partitions/LVs and filesystems larger as outlined below.

Expanding DomU Partitions

In this example we're using /dev/xvda as the drive name inside the DomU. Due to limitations in the way traditional partitions work this tutorial is only useful for resizing the last partition on the DomU drive.

1. Start the DomU and log in as root

2. Run fdisk /dev/xvda

3. Note the last partitions start cylinder, then delete the last partition and recreate it with the start cylinder being the same as before and the end cylinder being the default (end of disk)

4. Save this and exit fdisk

5. Run partprobe to update the kernels partition table. View /proc/partitions to see if the last partition is now larger. If not, reboot.

5. Resize the filesystem on the partition, for example using xvda2: resize2fs /dev/xvda2

Expanding DomU Logical Volumes

If you configured your DomU to use LVM and your Logical Volume is not big enough you can resize it.

In the DomU we get the Logical Volume name by using the lvdisplay command.

[ root@vs /srv/xen ] lvdisplay
--- Logical volume ---
LV Name                /dev/vgsys/lvvirt
VG Name                vgsys
LV UUID                XMWzWW-oZih-A5uH-91Sa-7l1y-8cqf-18KcNB
LV Write Access        read/write
LV Status              available
# open                 1
LV Size                60.00 GB
Current LE             1920
Segments               2
Allocation             inherit
Read ahead sectors     auto
- currently set to     256
Block device           253:2

According to lvdisplay the path to our Logical Volume is /dev/vgsys/lvvirt.

Hot Resizing Logical Volume:

Resizing the Logical Volume is actually very easy, you'll be amazed.

lvresize --size +10G /dev/vgsys/lvvirt

This lvresize command specifies to add 10GB to /dev/vgsys/lvvirt. This will only take a second.

Hot Resizing the filesystem:

resize2fs /dev/mapper/vgsys-lvvirt

That's really it! You can only hot resize if the Filesystem is getting larger. If you need to shrink it then you'll have to take the Volume offline first. Isn't this easier than dealing with partitions that are too small?

This isn't really a howto per se but rather a pointer to RPMs for KVM on CentOS/RHEL.

You can get KVM 80 for x86_64 here.

http://www.lfarkas.org/linux/packages/centos/5/x86_64/

The same site doesn't seem to have i386 RPMs but they do have i386 SRPMS that you could build.

http://www.lfarkas.org/linux/packages/centos/5/SRPMS/

I don't have time to do any sort of tutorial on building KVM SRPMs because I'm not currently using KVM for any projects. In the future I'll get to it but probably not too soon.

Download xen-tools, install rpmstrap and install xen-tools. If there's a newever version of xen-tools available substitute that filename

yum install -y rpmstrap 
wget http://xen-tools.org/software/xen-tools/xen-tools-3.9.tar.gz
tar -xzvpf xen-tools-3.9.tar.gz
cd xen-tools-3.9/
make install

I've not using xen-tools that much but I wanted to put together a tutorial anyway. Let me know how it goes.

Copy and paste the following command blocks depending on your cpu architecture.

For x86_64

wget http://downloads.sourceforge.net/enomalism/enomalism.public 
wget http://downloads.sourceforge.net/enomalism/enomalism001.pubkey.asc
rpm --import enomalism.public
rpm --import enomalism001.pubkey.asc
wget http://downloads.sourceforge.net/enomalism/libvirt-0.4.1-1.CentOS5.x86_64.rpm?modtime=1205833130&big_mirror=0
yum install /libvirt-0.4.1-1.CentOS5.x86_64.rpm
wget http://downloads.sourceforge.net/enomalism/Enomalism2-2.1-py24-noarch.rpm?modtime=1223313658&big_mirror=0
yum install Enomalism2-2.1-py24-noarch.rpm
/etc/init.d/mysqld start
chkconfig mysqld on

For x86_32

wget http://downloads.sourceforge.net/enomalism/enomalism.public 
wget http://downloads.sourceforge.net/enomalism/enomalism001.pubkey.asc
rpm --import enomalism.public
rpm --import enomalism001.pubkey.asc
wget http://downloads.sourceforge.net/enomalism/libvirt-0.4.1-1.CentOS5.i386.rpm?modtime=1205833113&big_mirror=0
yum install libvirt-0.4.1-1.CentOS5.i386.rpm
wget http://downloads.sourceforge.net/enomalism/Enomalism2-2.1-py24-noarch.rpm?modtime=1223313658&big_mirror=0
yum install Enomalism2-2.1-py24-noarch.rpm
/etc/init.d/mysqld start
chkconfig mysqld on

Set a mysql root password if you don't already have one. Substitute your password for .

   mysqladmin password  

Configure Enomalism. Substitute your mysql_root_password, your desired_enomalism_username and desired_enomalism_password.

cd /opt/enomalism2
scripts/init-db.sh mysql_root_password desired_enomalism_username desirec_enomalism_password
  
cp default.cfg config/$HOSTNAME.cfg
uuidgen
#Copy the output from uuidgen to the clipboard

vi config/$HOSTNAME.cfg

• Change sqlobject.dburi="mysql://enomalism2:zx45qw12@localhost/enomalism2" to reflect your proper MySQL username and password.
• Change enomalism2.self="5fe6f05e-7ee0-11dc-ba7c-0011d88b8e81" paste the output of uuidgen here
  
• Change enomalism2.baseurl="http://127.0.0.1:8080/rest/" to the IP/hostname you use to access the Enomalism web interface.
• Change enomalism2.ip_addr="1.2.3.4" to the IP/hostname you use to access the Enomalism web interface, this will be used later for clustering.

Configuring VNC access to your VMs

• /etc/libvirt/qemu.conf (NOTE: If this file is missing or is a directory, you probably did not install the 0.4.1 version of libvirt!)

  # VNC is configured to listen on 127.0.0.1 by default.
  # To make it listen on all public interfaces, uncomment
  # this next option.
  #
  # NB, strong recommendation to enable TLS + x509 certificate
  # verification when allowing public access
  #
  vnc_listen = "0.0.0.0"
  

• /etc/xen/xend-config.sxp

  # The interface for VNC servers to listen on. Defaults
  # to 127.0.0.1  To restore old 'listen everywhere' behaviour
  # set this to 0.0.0.0
  (vnc-listen '0.0.0.0')
  

Running Enomalism

service enomalism2.sh start

• If no errors occur point your browser to: http://server:8080 , where "server" is the IP or the hostname of the enomalism server.
• After the install completes, click on the bottom link and log in using
  • username: admin
  • password: password

There are a lot of problems with the new VirtualBox 2 series. I've found many things don't work as they're supposed, I get lockups for no reason and the new features just don't warrent a full version increase. I've rolled back all my projects at work to VirtualBox 1.6.6 and decided in non-mission critical situations (home use) I'd continue using VirtualBox 2.02 and just deal with the bugs. Well, I'm officially back-pedaling on that now. VBox 2 is buggy as heck and I'm not going to continue using it.

In order to downgrade your VMs from 2.x to 1.6.6 you'll need to edit a few xml files. If you had 1.x installed and upgraded to 2.x then it will have molested your xml files so you'll need to edit them. For each VM there is an xml file in the Machines directory containing the configuration data.

/home/grant/.VirtualBox/Machines/Linux Mint/Linux Mint.xml

 <HardwareVirtExNestedPaging enabled="false"/>

  <GuestProperties>

        <GuestProperty name="/VirtualBox/GuestAdd/Vbgl/Video/SavedMode" value="1280x960x32" timestamp="1224693483507107000" flags=""/>

</GuestProperties>

1. First we need to download the YUM repository file for the updated Xen.

wget http://www.gitco.de/linux/x86_64/centos/5/CentOS-GITCO.repo -O /etc/yum.repos.d/gitco.repo

2. Uninstall old Virtualization files

yum groupremove Virtualization

3. Install the relevant packages using YUM

yum groupinstall -y Virtualization

Yum will probably want to upgrade some other files along with the ones we've chosen.

Warning! If you get an error message from grubby this is bad!

Installing: kernel-xen                   ####################### [ 9/13]
grubby fatal error: unable to find a suitable template

This means that your grub.conf file couldn't be written to for whaterver reason so you won't be able to successfully reboot. If you get this message you need to edit your /boot/grub/grub.conf file and make the kernel lines match the kernel you installed.

Get your installed kernel version:

[ root@vs / ] rpm -q kernel-xen

kernel-xen-2.6.18-128.4.1.el5

Now edit your /boot/grub/grub.conf to match this

# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd0,0)
#          kernel /vmlinuz-version ro root=/dev/vgsys/lvroot
#          initrd /initrd-version.img
#boot=/dev/sda
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title CentOS (2.6.18-128.4.1.el5xen)
root (hd0,0)
kernel /xen.gz-3.4.0
module /vmlinuz-2.6.18-128.4.1.el5xen ro root=/dev/vgsys/lvroot rhgb quiet
module /initrd-2.6.18-128.4.1.el5xen.img

4. Reboot - no really I mean it.

5. Try it out by using the xm dmesg command

\ \/ /___ _ __   |___ / |___ / / _ \
\  // _ \ '_ \    |_ \   |_ \| | | |
/  \  __/ | | |  ___) | ___) | |_| |
/_/\_\___|_| |_| |____(_)____(_)___/

(XEN) Xen version 3.3.0 (This email address is being protected from spambots. You need JavaScript enabled to view it.) (gcc version 4.1.2 20071124 (Red Hat 4.1.2-42)) Tue Sep  9 20:18:14 CEST 2008
(XEN) Latest ChangeSet: unavailable
(XEN) Command line:
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 2 seconds
(XEN) Disc information:
(XEN)  Found 1 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) Xen-e820 RAM map:

That's about all. If you have any questions drop a comment here.

So here's a quick howto on getting KVM to work on CentOS5.x. I'll add more tutorials when I start using KVM more. For now I spend all my time on VirtualBox at work and Xen everywhere else. KVM is maturing fast but still not where Xen is for the most part. It also needs VT support in the CPUs which means not everyone can use it. My 4 core, 8 vcpu Xeon machine in the garage won't run KVM so I'm not spending a lot of time with it right now. One of the reasons I don't use KVM in a production environment is it moves too fast and the distros are just too far behind as you'll see in the howto

1. First you need to see if you're CPU has the needed hardware VT support

egrep 'vmx|svm' /proc/cpuinfo

If you have an Intel CPU with VT support you should get back vmx and svm if it's AMD with VT support. If you get nothing you need to use Xen or VirtualBox as both support Virtualization without the hardware VT support.

2. Install KVM and QEMU

yum install kvm kmod-kvm qemu

The current version of KVM is 36 which is ancient (August 2007) but stable as far as KVM goes. Even with this version I've gotten lockups and strange behavior. The most recent version of KVM is 74 so you can imagine what's been added since version 36. If you want to check it out yourself here's the changelog.

Scenario: In the Dom0 (Host) you have an LVM Logical Volume that you export to the DomU (Guest) and it appears as a hard drive partition which you want to make larger.

Example- Dom0: /dev/VolGroup00/LogVol00 -> DomU: /dev/xvda1

Resizing an LVM Logical Volume used as a DomU partition

When you use an LVM Logical Volume as a DomU partition it will show up in the DomU as /dev/xvda1 as apposed to /dev/xvda. The line in the DomU config file will look something like this.

disk = ['phy:vgsys/lvvirt,xvda1,w']

Get the Logical Volume name and path

So you gave your Xen virtual machine a 10GB Logical Volume and not it's not enough eh? This is easy to fix.

We start by using the lvdisplay command to get the path to the Logical Volume

[ root@vs /srv/xen ] lvdisplay
--- Logical volume ---
LV Name                /dev/vgsys/lvvirt
VG Name                vgsys
LV UUID                XMWzWW-oZih-A5uH-91Sa-7l1y-8cqf-18KcNB
LV Write Access        read/write
LV Status              available
# open                 1
LV Size                60.00 GB
Current LE             1920
Segments               2
Allocation             inherit
Read ahead sectors     auto
- currently set to     256
Block device           253:2