Grant McWilliams

Virtualization Blog

New download section for Xen Cloud Platform/Xenserver scripts

As I build tools to help manage my clouds I will upload them here. So far I have three - mktemplate.sh, lstemplate.sh and lsvms.sh.

  1. mktemplate.sh is a wizard that creates XCP/Xenserver templates.
  2. lstemplate.sh lists XCP/Xenserver templates showing name-label and UUID sorted with colors. Optionally it shows the template description.
  3. lsvms.sh lists XCP/Xenserver Virtual Machines including name, running state, UUID and it's host.

In each of these cases I needed this functionality in a nice easy to use command. I also wanted the output to be parsable if possible ie. it needed to be line oriented and each field separated by white space. 

There will be many more tools coming as I get time and I have needs. I'd love to have a tool that lists VM hardware (nics, disks, cds) in a nice manner, a tool to list disks and the srs they reside on with flags (rw etc..). Stay tuned.

The tools have been moved to github so the most convenient way to install them is to install git and do a git clone.

Intall Git on XCP - http://grantmcwilliams.com/item/652-install-git-on-xcp-host

Git clone - git clone https://github.com/Xenapi-Admin-Project/xenapi-admin-tools

Add a comment

Updated Xen Howtos

There's been a flurry of activity around The Man, The Myth, The Legend in the Xen Howtos section and for good reason - CentOS6 was released. It all started as a simple update to my installation tutorials but ultimately I spent half a week on it. There were some issues with how I was doing things because CentOS5 used the old Xen kernel and CentOS6 uses the new libvirt kernel. As of RHEL5/CentOS6 Xen Host (dom0) support is no longer in the kernel. However, Xen Guests (DomU) is and  is handled by libvirt. Last week the last bits of Xen Dom0 support were merged into Linux Kernel 3.0. This means that going forward all Linux distributions will have Xen Dom0 ability unless the distributors remove it.

I've written and tested two xen tutorials this week.

During the process of writing these tutorials I shrunk the size of the Disk Image. I did this because I like nice small VM disk images (and sparse too) so I can duplicate them and move them around easily. It's fairly easy to resize a disk image so I updated all four tutorials on how to resize Dom0 Disk Images and Logical Volumes as well as DomU Logical Volumes and partitions. I attempted to make it clearer too what the scenario was so people would know what the tutorial was trying to accomplish.

As always have lots of fun and let me know if something doesn't work via the comments.

Add a comment

Read more: Updated Xen Howtos

Xen Enterprise to be free

As a strike against the industry leading VMWare, Citrix releases it's Xen Enterprise Virtualization platform to the public for free. Note this does not mean it's been released OpenSource as the Xen hypervisor is, it's just free to download and use on as many installations as you like. The difference is that it's free as in beer, not as in freedom. If Citrix wanted to change the license back to pay only after it hooks a million customers it can.

 

Game Changing Move Goes Well Beyond Free Hypervisors to Accelerate the Adoption of World-Class Virtual Infrastructure for Both Enterprises and Cloud Providers

BOSTON » 2/23/2009 » Citrix Systems, Inc. (Nasdaq: CTXS), the global leader in application delivery infrastructure, today unveiled a groundbreaking new version of Citrix® XenServer™ – the company’s enterprise-class, cloud-proven virtualization platform – that will be offered free of charge to any user for unlimited production deployment.  While basic hypervisors have been free for years, most have had limited practical use in real world environments.  With this new release, XenServer sets an entirely new standard for free virtualization with the addition of powerful new features like centralized multi-node management, multi-server resource sharing and full live motion.  Dramatically lowering the entry price of virtualization also helps address today’s challenging economic climate by making enterprise-class virtualization far more accessible to businesses of all sizes, regardless of budget.

 

Xen Enterprise is running in 5000 datacenters worldwide. Downloads will be available in March. Here's the official announcement.

 

 

 

Add a comment

Xen disk images vs LVM

 

In setting up a Xen virtualized web server with about 15 GB of static files I questioned the wisdom of using a disk image as apposed to LVM. I read a bunch of forums about which is better and it seems the disk images are easier to manage because you can just copy to make backups etc.. LVM you just can't move it and pop it up whereever you want but you have more flexibility in resizeing and backing them up within the host OS. However what I didn't find on the forums is performance tests. People theororized about the speed difference but it seems nobodies sat down and done some tests. It's my goal that after summer vacation to put together a performance whitepaper comparing all the virtual machine solutions. For now I'll just put up xen disk images against LVM.

So I created a 10 GB empty disk image using dd and I created a 10 GB LVM Logical volume and assigned both of them to the domU.

 

name = "centtest"
memory = "512"
disk = [ 'tap:aio:/srv/xen/centos5.img,xvda,w', 'tap:aio:/srv/xen/centtest.img,xvdb,w', 'phy:/dev/vgsys/lvtest,xvdc,w' ]
vif = [ 'bridge=xenbr0', ]
bootloader="/usr/bin/pygrub"
vcpus=1
on_reboot = 'restart'
on_crash = 'restart'
Add a comment

Read more: Xen disk images vs LVM

Convert a Qemu image to Vbox vdi

 

To convert a QEMU qcow format disk to VirtualBox vdi format you need to have Qemu installed. Qemu has tools to help convert disk formats.

grant@workstation:~$ qemu-img convert hda-qcow.img -O raw hda.img
grant@workstation:~$ VBoxManage convertdd hda.img hda.vdi
VirtualBox Command Line Management Interface Version 1.6.0
(C) 2005-2008 Sun Microsystems, Inc.
All rights reserved.

Converting VDI: from DD image file="hda.img" to file="hda.vdi"...
Creating fixed image with size 1024966656 bytes (978MB)...


This will convert a QEMU qcow format disk image to VirtualBox vdi format.

 

 

Add a comment

Virtual disk formats

The problem with Virtualization systems is none of them want to standardize on one particular disk format. Actually they all do, they want their format to be the standard.

  • VMware has VMDK
  • QEMU has qcow and qcow2
  • UML has cow
  • Parallels has HDD
  • VirtualPC uses VHD
  • VirtualBox has VDI
  • Xen uses raw disk images

To make matters more confusing VirtualBox has some support for VMDK, Commercial Xen supports VHD, the open source Xen supports qcow2 and Qemu can convert between many formats. So what I'm doing about this is putting up a series of Howtos in the Tech -> Virtualization areas on how to convert and use various disk formats in different Virtualization systems. To start with I just put together the following two Howtos.

Add a comment

Read more: Virtual disk formats

Intel chipsets with IOMMU support

When Intel and AMD added hardware virtualization support to their CPUs it was a boon to those of us trying to virtualize Operating Systems that don't know they're being virtualized. KVM the somewhat native Linux virtualizer built into the kernel only operates in this mode so you need a CPU with Intel VT-x or AMD-V support built into it and the ability to turn it on in the BIOS. This seems like a really great thing but in reality anyone wanting to extract the maximum performance out of their Virtualization machine is going to be using paravirtualization anyway (sorry KVM). Everyone is working on paravirtuaization including the KVM folks. So the addition to VT extensions in the CPU are really not getting used much besides those that don't have a choice (ahem, KVM). A year from now people using KVM will be using paravirtualized drivers because they're just plain faster. VirtualBox uses a mix of software emulation and virtualization depending on what's fastest. Xen will run in both Full Virtualization (HVM) or paravirtualization (PV) modes depending on how you set it up. There are limitations to PV mode though becuase it's using the same QEMU code that KVM uses. They will reach parity in installation modes between PV and HVM soon though. The cool thing about everyone using QEMU (KVM, Xen, VirtualBox etc..) is that if you change the code once the rest get the updates. Isn't that the way it's supposed to work in the Open Source world?

Anyway the point of this article is not to talk about VT but IOMMU. The problem with Virtualization systems like Parallels and Xen are that they're passing PCI devices through to guest operating systems which is a good thing but because they haven't had support in the CPUs it's been a bit hacked up. There's been a number of security vulnerabilities with pci passthrough in Xen. All of this is changing because Intel and AMD are adding device virtualization to their kernels. Intel announced it in 2006 and has a number of chipsets that support it and AMD will have their first out next year.

Add a comment

Read more: Intel chipsets with IOMMU support

Convert raw disk images to vdi format

If you're moving from a real server installation to a VirtualBox virtualized configuration you may want to take your real physical disk and just turn it into a virtual disk. There are advantages to creating a new disk and rsyncing your OS into it but this tutorial will show you how to make an exact copy of it. Note the exact copy will be the same size as the real physical disk so make sure you have enough drive space. This is most useful for Operating System images with shared storage for data. I wouldn't advise anyone to make a 1TB copy of their new drive and turn it into a VDI file!

To get the image from the disk use the dd command.

  1. dd if=/dev/hda of=./hda.img
  2. VBoxManage convertdd hda.img hda.vdi
grant@workstation:~$ dd if=/dev/hda of=./hda.img
grant@workstation:~$ VBoxManage convertdd hda.img hda.vdi
VirtualBox Command Line Management Interface Version 1.6.0
(C) 2005-2008 Sun Microsystems, Inc.
All rights reserved.

Converting VDI: from DD image file="hda.img" to file="hda.vdi"...
Creating fixed image with size 1024966656 bytes (978MB)...

This will make an exact copy of /dev/hda to the raw image file hda.img. Then VBoxManage will convert the raw disk hda.img to hda.vdi for use with VirtualBox.

Add a comment