Grant McWilliams
  • Food
    • Blog
    • Websites
    • Seattle Food Events
    • Seattle Farmers Markets
  • Travel
    • Travel Blog
    • Travel Photos
    • Travel Links
    • Trip Journals
      • Europe - 2015
      • New Orleans - 2013
      • Central Mexico - 2010
      • Toulouse/Paris - 2009
      • Paris/Lyon - 2008
      • Peru/Ecuador - 2007
      • British Columbia - 2007
      • Central Mexico - 2006
      • Eastern Europe - 2006
      • Washington D.C. - 2006
      • Where's the rest?
  • Technology
    • Gadgets
      • Blog
      • Websites
      • Android
      • Nokia n900
      • SailfishOS
    • Technology
      • Blog
      • Websites
    • Virtualization
      • Virtualization Blog
      • Websites
      • Xen Howtos
      • Xenserver Howtos
      • KVM Howtos
      • VirtualBox Howtos
      • Virtual OS images
      • Xenapi Admin Tools github
    • Programming
      • Blog
      • Bash
      • Python
      • Websites
      • Downloads
    • Transportation
      • Blog
      • Websites
    • Sustainable Tech
      • Blog
      • Websites
  • Photography
    • Blog
    • Portfolio
  • Random
    • Grantianity
      • Basics
      • Grantisms
      • The River of Life
      • Bus Maintainance
      • Believing in Yourself
    • Creative Writing
    • Login/Logout
    • About

Yet another Linux based Mobile OS (is there hope?)

Details
Category: Gadget Blog
November 21, 2012

As a long time Maemo user (OS2007 on a Nokia n800, OS2008 on a Nokia N810 and Maemo 5 on a Nokia n900) and really like having an open Linux mobile platform. Even though I get shorted on all the Android apps out there I have virtually everything I need (short of Yelp and OneBusAway apps) and a whole lot more. Having a full featured Linux OS on my phone is very useful for a variety of reasons one of which I get a real terminal. I have an app that uses rsync to synchronize my media server to my phone so I never have to manage my music/podcasts via the mobile interface. Multitasking on the n900 is amazing in comparison to Apple's iOS or Android.

However, the n900 keeps getting older and I'd like to have something newer. I'm not about to give up my awesome mobile OS though for a nice piece of hardware running a Java stack. Let's not forget that Nokia make phenomenal hardware even if it was expensive. Their stuff is rock solid and I've had to reboot my n900 fewer times in the last 18 months than I did per week on my Android 2.2 phone.

With all of that in mind I've been waiting for a replacement for Maemo 5. Nokia was working on Maemo 6 then merged their efforts with Intel's Moblin which proved to be a disaster. It took an extra 2 years to get nowhere before Meego 1.2 was released. Nokia put it on one phone, the n9 (which I'd still like to have) and then abandoned it. Intel then announced it was going to carry the torch alone which nobody believed since Intel can't design a piece of software to save their lives. Soon after they too abandoned it. The community project of Mer picked up the source code and started working on it which was supposed to be used by the Vivaldi tablet which never went anywhere. 

Intel then partnered with Samsung to work on Tizen which hasn't gone anywhere either. So where does this leave me? Using my couple year old n900 for a bit longer. However, there may be yet another ray of hope (but I'm not holding my breath) - Sailfish OS. 

Apparently Sailfish OS from Jola is based on MeeGo but I'm not sure if it's using the Mer codebase or they forked it from MeeGo. Here's a video of Sailfish OS running on a Nokia n950 (my dream phone) and it looks pretty neat. They're saying that they will be able to release in early 2013. Until I see something I can buy or use I'll wait patiently and try not to get too excited. 

Xenapi Admin Tools moving to github

Details
Category: Virtualization Blog
November 18, 2012

Up until about now I've been developing the Xenapi Admin Tools on my local cloud. I've been maintaining revisions on a local Subversion server which has been accessible by only the Xenapi Admin Project team. Now that we're slowly moving our projects public for inclusion in the Xen Cloud Platform's github I wanted to push Xenapi Admin Tools to github which is partly done as of today. 

The URL for the repo is https://github.com/Xenapi-Admin-Project/xenapi-admin-tools. Please feel free to browse the code and the development docs which outline the Xenapi Admin Tools spec and it's built in functions. I also have a yum repo file and a SPEC file for when I start creating rpms for Xen Cloud Platform/Xenserver. For now I would not consider that anything but alpha. At some point you'll be able to just install the Xenapi-admin-repo rpm and then yum install xenapi-admin-tools to install all of the tools including their manpages, config files and more. Currently a lot of these things don't exist. Also there are two branches of code in the github repo - 3.0 and 4.0. Not all tools are available in 4.0 yet as I'm still rewriting them. The difference is massive speedups with 4.0. Stay tuned for more news.

Add a hard disk to a VM

Details
Category: Xenserver Howtos
November 5, 2012

Installing from an XCP/Xenserver template usually gives you one Virtual Disk to install the operating system on. Depending on your needs this disk may not be large enough. Following is a tutorial on how to add an additional disk to a virtual machine.

Terminology: 

 

Virtual Machine - A virtual machine is a computer that's virtualized and running on a hypervisor. In our case the hypervisor is Xen Cloud Platform/Xenserver. The Virtual Machine can be running any operating system.

Virtual Disk Image - Think of a Virtual Disk Image as a hard drive.

Storage Repository - A "box" storing Virtual Disk Images. Think of this as an external box storing virtual hard drives. The virtual hard drives are the Virtual Disk Images mentioned above.

Virtual Block Device - A Virtual Block Device connects a Virtual Disk Image to a Virtual Machine. In traditional computer terms you could think of it as the cable. 

The process for adding a hard drive to a real computer

  1. Insert the disk in the hard drive box
  2. Connect the cable to the hard drive box
  3. Insert the cable into the Computer

The process of adding a new Virtual Disk for a Virtual Machine is 

  1. Create a new Virtual Disk Image
  2. Create a new Virtual Block Device for it
  3. Connect the Virtual Block Device to the Virtual Machine

 

1. Get available free space

You will need to know how much free space is available on your Storage Repository.

[ root@cloud2 ~ ] xe sr-list
uuid ( RO)                : 36bf480a-5df9-4453-50f0-2bac4a86cb42
              name-label ( RW): localsr-cloud1
    name-description ( RW): 
                         host ( RO): cloud1.acs.edcc.edu
                        type ( RO): lvm
           content-type ( RO): user

Now that we have the Storage Repository's UUID number (36bf480a-5df9-4453-50f0-2bac4a86cb42) we can use xe sr-list again to give us the physical size and how much space is being utilized.

[ root@cloud2 ~ ] xe sr-list uuid=36bf480a-5df9-4453-50f0-2bac4a86cb42 \
params=physical-utilisation,physical-size
physical-utilisation ( RO) : 214752559104 physical-size ( RO): 991600574464

Quick math (991600574464 - 214752559104 = 776848015360) shows us that we have about 776 MB free.

2. Create the Virtual Disk Image

Now that we know the available space on the storage repository we can make a new Virtual Disk Image using xe vdi-create.

[ root@cloud2 ~ ] xe vdi-create sr-uuid=bd1ac90d-7c23-dc07-dfa3-edc9f1cd73c4 \
name-label=DATADISK type=user virtual-size=100GiB
ee9c5daa-392c-4a0d-a5c1-4ebb7caabd73

This command outputs the VDI's UUID. You can get information about any VDI using the xe vdi-list command.

[ root@cloud2 ~ ] xe vdi-list uuid=ee9c5daa-392c-4a0d-a5c1-4ebb7caabd73 
uuid ( RO)                : ee9c5daa-392c-4a0d-a5c1-4ebb7caabd73
          name-label ( RW): DATADISK
 name-description ( RW): 
                  sr-uuid ( RO): bd1ac90d-7c23-dc07-dfa3-edc9f1cd73c4
            virtual-size ( RO): 107374182400
               sharable ( RO): false
             read-only ( RO): false

The result of xe vm-list shows that the virtual size of the VDI is about 100 GB and it's name-label is DATADISK. To add this new disk to a VM I'll need to get the VM's UUID number by using xe vm-list.

[ root@cloud2 ~ ] xe vm-list name-label=CentOS6
     uuid ( RO)           : cefb9f88-0424-6701-5ba1-070490c69203
     name-label ( RW): CentOS6
      power-state ( RO): running

 

3. Get the available Virtual Block Device numbers

We will also need to know which Virtual Block Device numbers are available. We can use the xe vm-param-get command for this.

[ root@cloud2 ~ ] xe vm-param-get uuid=cefb9f88-0424-6701-5ba1-070490c69203 \ 
param-name=allowed-VBD-devices
7; 8; 9; 10; 11; 12; 13; 14; 15

In summary:

  1. VDI UUID is ee9c5daa-392c-4a0d-a5c1-4ebb7caabd73
  2. VM UUID is cefb9f88-0424-6701-5ba1-070490c69203
  3. Available VBD numbers are 7, 8, 9, 10, 11, 12, 13, 14 and 15

 

4. Create the Virtual Block Device

Create the Virtual Block Device (VBD) using the xe vbd-create command and the VM UUID, VDI UUID and the first available VBD number.

[ root@cloud2 ~ ] xe vbd-create device=7 vm-uuid=cefb9f88-0424-6701-5ba1-070490c69203 \ 
vdi-uuid=ee9c5daa-392c-4a0d-a5c1-4ebb7caabd73 bootable=false mode=RW type=Disk
333ab620-3ee1-0420-d31a-217e4ef1df45

I created Virtual Block Device 7 (device=7). Using device=0 would have given me a /dev/dev/xvda which I already have. The xe-param-get command showed my first available Virtual Block Device number was 7. Notice that we associated the Virtual Disk Image (VDI) to the Virtual Machine (VM) by using a Virtual Block Device (VBD).

5. Plug in the disk to the VM

The VM won't see the disk yet as it hasn't been "plugged in". We can do this by either rebooting the VM or using the xe vbd-plug command. Let's plug the VBD into the running VM.

[ root@cloud2 ~ ]  xe vbd-plug uuid=333ab620-3ee1-0420-d31a-217e4ef1df45

6. Verify that it worked

Log into the VM via ssh or xenconsole and see if the disk appeared by catting /proc/partitions.

[root@Centos6 ~]# cat /proc/partitions
major minor  #blocks  name

 202        0    8388608 xvda
 202        1     102400 xvda1
 202        2    8285184 xvda2
 253        0    7733248 dm-0
 253        1     524288 dm-1
 202       16  104857600 xvdb

Add a CD to a VM

Details
Category: Xenserver Howtos
November 1, 2012

Prerequisites

  • XCP/Xenserver

Adding a CD to a running VM is not a difficult task if you know which commands to use. By adding I mean we're going to insert a virtual CD disk into a virtual machine using our little virtual hands. ;-)

 

1. Get the name of your Virtual Machine

In this case the name is CentOS6 and the UUID is cefb9f88-0424-6701-5ba1-070490c69203.

[ root@cloud2 ~/bin ] xe vm-list
     uuid ( RO)           : cefb9f88-0424-6701-5ba1-070490c69203
          name-label ( RW): CentOS6
         power-state ( RO): running

 

2. Get the name of the CD disk

In this case our disk is named CentOS-6.3-x86_64-LiveCD.iso.

[ root@cloud2 ~/bin ] xe cd-list
    uuid ( RO)          : 0549c68a-e38d-4cd8-9974-ba0b9167ff5a
        name-label ( RW): CentOS-6.3-x86_64-LiveCD.iso

 

3. Identify a free Virtual Block Device number

Use the VM UUID that we retrieved in step 1.

[ root@cloud2 ~/bin ] xe vm-param-get uuid=cefb9f88-0424-6701-5ba1-070490c69203 \
param-name=allowed-VBD-devices 5; 6; 7; 8; 9; 10; 11; 12; 13; 14; 15

 

4. Add the CD to the VM. Use the VM UUID from step 1, the CD name from step 2 and the VBD device number from step 3.

[ root@cloud2 ~/bin ] xe vm-cd-add uuid=cefb9f88-0424-6701-5ba1-070490c69203 \
device=5 cd-name=CentOS-6.3-x86_64-LiveCD.iso

 

5. Verify using xe vm-cd-list

 By using xe vm-cd-list we can list the CD's currently plugged into our VM.

 

[ root@cloud2 ~/bin ] xe vm-cd-list uuid=cefb9f88-0424-6701-5ba1-070490c69203 
CD 0 VBD:
uuid ( RO)             : 40f6b5c9-14fd-4379-d82c-d0ff34472a04
    vm-name-label ( RO): 955300270
            empty ( RO): false
       userdevice ( RW): 5


CD 0 VDI:
uuid ( RO)             : 0549c68a-e38d-4cd8-9974-ba0b9167ff5a
       name-label ( RW): CentOS-6.3-x86_64-LiveCD.iso
    sr-name-label ( RO): NFS ISO
     virtual-size ( RO): 725614592

 

6. Unplugging the CD disk

When done with the CD you can unplug it even easier. We specify the VM UUID and tell it to eject the CD which it does. 

 

 [ root@cloud2 ~/bin ] xe vm-cd-eject uuid=cefb9f88-0424-6701-5ba1-070490c69203

 

Create an LVM CD repository

Details
Category: Xenserver Howtos
October 29, 2012

 Prerequisites 

  • XCP/Xenserver

 Utilizing an ISO image in a VM's cdrom drive is fairly easy to do but because of the limited size of the Control Domain's (dom0) operating system partition it's difficult to download ISO images to /opt/xensource/packages/iso and it isn't really recommended to put them there anyway. In this tutorial we'll create a CD repository using an local Logical Volume.

First we need to know the name of the LVM Volume Group. This is taken from the Storage Repository's UUID. To get this we'll use xe host-list. 

[root@cloud1 ~]# xe sr-list type=lvm
uuid ( RO)                : 36bf480a-5df9-4453-50f0-2bac4a86cb42
          name-label ( RW): localsr-cloud1
    name-description ( RW): 
                host ( RO): cloud1.acs.edcc.edu
                type ( RO): lvm
        content-type ( RO): user

Using xe sr-list type=lvm shows only our local Storage Repository which has the UUID of 36bf480a-5df9-4453-50f0-2bac4a86cb42. We'll now use the vgs command to give us the names of all Volume Groups including VG_XenStorage-36bf480a-5df9-4453-50f0-2bac4a86cb42 which matches our SR UUID.

Read more …

Create an EXT CD repository

Details
Category: Xenserver Howtos
October 29, 2012

 Prerequisites 

  • XCP/Xenserver

 Utilizing an ISO image in a VM's cdrom drive is fairly easy to do but because of the limited size of the Control Domain's (dom0) operating system partition it's difficult to download ISO images to /opt/xensource/packages/iso and it isn't really recommended to put them there anyway. In this tutorial we'll create a CD repository using an additional hard drive on Dom0.

First we need to know the device name of the disk. 

[root@cloud1 media]# cat /proc/partitions 
major minor  #blocks  name
   8        0  976762584 sda
   8        1    4194304 sda1
   8        2    4194304 sda2
   8        3  968371393 sda3
   8       16  234431064 sdb

Read more …

Create an NFS CD repository

Details
Category: Xenserver Howtos
October 28, 2012

 Prerequisites 

  • XCP/Xenserver

 Utilizing an ISO image in a VM's cdrom drive is fairly easy to do but because of the limited size of the Control Domain's (dom0) operating system partition it's difficult to download ISO images to /opt/xensource/packages/iso and it isn't really recommended to put them there anyway. In this tutorial we'll create a CD repository using an NFS share.

In our example we'll be using a share on the cloud0 host named /media/NFSISO. To set this up on cloud0 you'd log into cloud0 as root and add this line to the /etc/exports file of your NFS server. 

 

/media/NFSISO/ *(rw)

 

I'd recommend that you secure your NFS share more tightly than I've done here but for the purpose of this tutorial we'll go with it. We need to make a directory that we can mount our NFS share on first.

Read more …

List XCP Host information

Details
Category: Virtualization Blog
August 20, 2012

I've been working on ways of getting information to the XCP/Xenserver Admins eyes faster than the standard xe commandline tool provides. This tool - lshosts is a rewrite of lshostvms.sh which showed each host and how many running VMs were on it, something I often would like to know. While rewriting it to include some of the better structure of my newer tools I started adding features. Now it displays either the Host's name-label or UUID, the number of running VMs, the CPU type, CPU cores, CPU speed, Total Memory, Free Memory and Network backend type. 

As an added bonus I've added a -c option so the output is in CSV format. All future commands should have this option and I'll be retrofitting older commands when I get time.

Download it from the XCP Downloads section. http://grantmcwilliams.com/tech/virtualization/downloads/category/4-xen-cloud-platform

Reboot stuck VMs

Details
Category: Xenserver Howtos
August 20, 2012

Prerequisites

  • XCP/Xenserver

 

Sometimes I get a stuck Virtual Machine that just won't go down and it's usually due to a lack of memory in the VM. When I issue a shutdown command from within the VM it starts the shutdown process but hangs part way through. Executing xe vm-shutdown --force uuid=<insert UUID here>  does nothing but lock up the terminal. If this happens to you follow the steps below to forcefully shut the VM down.  

  1. xe task-list (find the pending tasks UUID)
  2. xe task-cancel uuid=<task UUID>
  3. xe vm-list (note the VM's UUID)
  4. list_domains (find the VM's UUID and note the domain id)
  5. /opt/xensource/debug/destroy_domain -domid XX (where XX is the domain id from step 2)
  6. xe vm-shutdown --force uuid=<UUID from step 1>
This will canceling the tasks that may be locking any new tasks e.g. the shutdown commands, destroying the domain and then shutting the VM down. I've had to do this several times on an Apache webserver that's getting pummeled from the Internet. 
 

The steps above in script form (if you trust me). Step one has to be entered in manually. The rest can be copied and pasted.

name="Name Label"
TASK=$(xe task-list status=pending --minimal)
xe task-cancel uuid="$TASK"
VMUUID=$(xe vm-list name-label="$name" --minimal)
DOMID=$(xe vm-list uuid="$VMUUID" params=dom-id --minimal) /opt/xensource/debug/destroy_domain -domid "$DOMID" xe vm-shutdown --force uuid="$VMUUID"

 

 

 

 

 

 

Tool to list XCP/Xenserver networks

Details
Category: Virtualization Blog
August 9, 2012

My newest XCP tool is to list networks in a quick concise manner. By default lsnetworks shows the network name, the bridge it's associated with, the VLAN tag (if there is one), the VMs that have a network interface on it and the number of that interface. 

It can be found in the xenapi-admin-tools github.

 

 

 

Because XCP/Xenserver relies so much on UUID numbers I've provided a -u option which lists every object by UUID number if it has one. This won't be quite as useful as an interactive tool but if you're copy and pasting UUID's into xe commands this will give you a quick summary of them in relation to networks. 

 

  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11

Page 7 of 56

Search