Migrating VMs from Hyper-V 2008 or 2008 R2 to Hyper-V 2012 R2

There has been a recent explosion of questions around this in the Hyper-V TechNet forum over the past two weeks.  So I decided that I would blog about this a bit.

The primary question is: How can I migrate from Hyper-V 2008 (or Hyper-V 2008 R2) to Hyper-V 2012 R2.

There are lots of very well meaning folks that provide the advice of: "Export your VMs from your 2008 era Hyper-V Server and then Import those VMs to your 2012 R2 Hyper-V Server."

Obviously, they never tested this.  Because, IT DOES NOT WORK.

First of all, lets test this common advice:  Export a VM from Hyper-V 2008 / 2008 R2 and import direct to Hyper-V 2012 R2.

1. Create your VM Export
2. copy the export folder to a Hyper-V 2012 R2 system
3. attempt to import.

You will instantly get this:  "Hyper-V did not find virtual machines to import from location"
And you look, and everything is right there in that folder.  What gives!

The next piece of well meaning advice is to create a new VM configuration using the existing VHD in that export folder. 
(this will work, but if you have snapshots you are screwed - all of that snapshot history is lost, and lots of folks connect to the incorrect virtual disk and freak out that years of history was lost.)
 
If you were going to do this in the first place, why not just copy out the VHDs and save yourself some effort and be done with it.  This is viable option 1.

Here is the option that many folks overlook / are not aware of (as it was a new feature of Hyper-V 2012 R2:

Copy the VM folder direct from the Hyper-V 2008 R2 system to the Hyper-V 2012 R2 system and Import. 

Hyper-V 2012 R2 reads the XML configuration and imports the VM asking you a couple questions to fix things up. 
This is viable option 2 (actually the easiest if you have additional hardware with Hyper-V 2012 R2 already built).

We could stop there but not to be left without choices;  you can in-place upgrade from your Hyper-V 2008 / 2008 R2 era system to Hyper-V 2012 and then again to Hyper-V 2012 R2.  This will update the VM configurations as you go, and you will be all good.  Now we have a viable option 3.

Suppose that all you have is a VM Export.  Then what? 
Remember that error message at the beginning; Hyper-V 2012 R2 cannot read the VM export from Hyper-V 2008 / 2008 R2.  Now, we have other options.

Take your VM folder that you exported from your Hyper-V 2008 R2 system and copy it to a Hyper-V 2012 system.  Then import.  Success!

Now what?  You want Hyper-V 2012 R2.  You have a few viable options to take this from Hyper-V 2012 to Hyper-V 2012 R2: 

In-place upgrade the Hyper-V 2012 system to Hyper-V 2012 R2.  This is viable option 4.
Export the VMs, then import them to your Hyper-V 2012 R2 system.  This is viable option 5.

Thinking out of the box, are there other options?

I am always assuming that you have backups of your systems.  And you have tested restoring those backups, and you know those backups are indeed good and useful.  This gives another option. 
Restore your VMs to the Hyper-V 2012 R2 system as new VMs.  This becomes viable option 6.

There you have it.  Six options to test and choose from.  All of which are considered supported. And will save you the panic of realizing that going straight from a Hyper-V 2008 / R2 VM Export to 2012 R2 will not work.

Handling Import-VM errors in Server 2012 Hyper-V

Import-VM.  A great cmdlet for Hyper-V.

Couple that being able to import a VM that was not exported and you have a golden feature.

And if all the stars align it works like a charm.  Also notice that in the GUI you import a path, here you import a configuration file.

But, what if you have problems to fix?  It isn’t as straightforward.  You have to fix the source VM to match your host.

Even though there is an example under Get-Help it isn’t easy to visually parse and is difficult to follow, so lets add some color and additional explanation.

 

I have a VM.  That VM was running on Hyper-V Server X and I export a copy and move it to Hyper-V Server Y.

These servers are not set up the same, they are in different environments.

I try Import-VM and get the following results:

Import-VM -Path 'C:\users\Public\VmPxe\Virtual Machines\339C3412-E83F-4EA7-9DE6-F9D388B196E2.XML'

Import-VM : Unable to import virtual machine due to configuration errors.
Please use Compare-VM to repair the virtual machine.
At line:1 char:1
+ Import-VM -Path 'C:\users\Public\VmPxe\Virtual
Machines\339C3412-E83F-4EA7-9DE6- ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~
    + CategoryInfo          : NotSpecified: (:) [Import-VM], VirtualizationOpe
   rationFailedException
    + FullyQualifiedErrorId : Microsoft.HyperV.PowerShell.Commands.ImportVMCom
   mand

Not that error are easy to understand in PowerShell and your mileage varies by the folks that wrote a set, but I at least know that something is wrong and I have another cmdlet to find out what.  Fair enough.

Compare-VM -Path 'C:\users\Public\VmPxe\Virtual Machines\339C3412-E83F-4EA7-9DE6-F9D388B196E2.XML'

VM                 : Microsoft.HyperV.PowerShell.VirtualMachine
OperationType      : ImportVirtualMachine
Destination        : BJELKS4
Path               : C:\users\Public\VmPxe\Virtual
                     Machines\339C3412-E83F-4EA7-9DE6-F9D388B196E2.XML
SnapshotPath       : C:\users\Public\VmPxe\Snapshots
VhdDestinationPath :
VhdSourcePath      :
Incompatibilities  : {33012, 33012}

Excellent, I have incompatibilities 33012, twice.  What is that all about???  I guess that I need to capture that report to see the errors as I can see that it is in an array ( the {} told me so ).

$compareResult = Compare-VM -Path 'C:\users\Public\VmPxe\Virtual Machines\339C3412-E83F-4EA7-9DE6-F9D388B196E2.XML'

$compareResult.Incompatibilities

Message                                     MessageId Source
-------                                     --------- ------
Could not find Ethernet...                      33012 Microsoft.HyperV.Power...
Could not find Ethernet...                      33012 Microsoft.HyperV.Power...

$compareResult.Incompatibilities[0].Message
Could not find Ethernet switch 'VMs'.

Okay, some useful information.  Now, I need to fix that.  But how?  I have a Message, MessageId, and a Source.  I bet the Source is a Source object that can be acted upon. <light bulb on>

Lets just try to disconnect the VM from the virtual switch, but how? 

If I look at the –Examples under Get-Help I scan through and see that I can take this $compareResult and act upon the .Source objects.  What I have declared as variable $compareResult MSFT refers to as a “Compatibility Report”.  But the important part is that it is more than a report.

That array of objects behind $compareResult.Incompatibilities are actually objects.  They are references to the objects that are having problems.  Even though it began as an XML file.

In my case these are the VMNetworkAdapter objects of the VM that I am attempting to import.  So I can’t search on the name of the VMNetworkAdapter and are forced to work through the list of objects.  At this time it is a kind of temporary VM that is in a state of limbo, it is declared but not committed.

Lets work through and fix the VM that I am attempting to import.

$compareResult.Incompatibilities[0].Source | Disconnect-VMNetworkAdapter

Okay, that Disconnected one of the VMNetworkAdapaters, but I want to connect the other one to the proper Virtual Switch.

$compareResult.Incompatibilities[1].Source | Connect-VMNetworkAdapter -SwitchName "Virtual Network0"

compare-VM -CompatibilityReport $compareResult

VM                 : Microsoft.HyperV.PowerShell.VirtualMachine
OperationType      : ImportVirtualMachine
Destination        : BJELKS4
Path               : C:\users\Public\VmPxe\Virtual
                     Machines\339C3412-E83F-4EA7-9DE6-F9D388B196E2.XML
SnapshotPath       : C:\users\Public\VmPxe\Snapshots
VhdDestinationPath :
VhdSourcePath      :
Incompatibilities  :

Okay, looking good, no errors.  No what?  How do I import this silly Compatibility Report that I just modified?

Well, the option is there.

Import-VM -CompatibilityReport $compareResult

Name  State CPUUsage(%) MemoryAssigned(M) Uptime   Status
----  ----- ----------- ----------------- ------   ------
VmPxe Off   0           0                 00:00:00 Operating normally

My only big comment on this is that I have to act upon each individual .Incompatibilities.Message in the array on a case by case basis.  That is a bit of a pain.  And it took a while.  Especially because you have to be a bit familiar with the available PowerShell cmdlets to know what you might be able to apply to any particular object.

Seems that I need a script to automatically manage the incompatibilities…

Importing the XenApp EVA is really this easy.

I’m going to import the XenApp 6 Evaluation Virtual Appliance into Citrix XenServer 5.6 using a new feature of XenServer 5.6 and XenCenter 5.6 called the Disk Image Import Wizard.

You can also watch this in action here: http://www.citrix.com/tv/#videos/2368

I will assume that you have already downloaded and installed XenServer 5.6 and XenCenter 5.6.

First, go to the download website for the Citrix XenApp EVA. http://www.Citrix.com/tryxenapp

A login with a MyCitrix user account is required. Be sure to download two items: the Getting Started Guide and the Evaluation Virtual Appliance package for XenApp 6.

After downloading the Evaluation Virtual Appliance, execute the CitrixXA6EVA.exe to expand the self extracting archive into a folder that contains a VHD and licensing documents.

Now, open (and read) the getting started guide. The import wizard requires some important information regarding the configuration of the virtual machine. On page four, the guide states that the EVA requires at least 2GB of RAM and the VHD requires at least 30GB of storage space. So make sure that your XenServer has a Storage Repository with at least 30GB of available space.

Open XenCenter and connect to a XenServer 5.6 host (using version 5.6 is required). In the left hand tree view select the host and then right click for the context menu and choose the Disk Image Import.

In the import wizard browse to and select the VHD and choose next.

Enter the information for the virtual machine. The VM needs a name, the number of virtual processors to assign and an amount of memory.

Note: I am unselecting the Run Operating System Fixups option because I know that this VM contains Server 2008 R2 and that it was built for Hyper-V. Server 2008 R2 can handle hardware changes pretty well (single to multiple processors, critical boot devices, etc.). XenServer also presents a critical boot device controller similar to the Hyper-V IDE specific boot device.

Then begin the import.

Once the import completes, select done.

At this time: power on the virtual machine and login at the logon screen. The domain and user account and password are in the Getting Started Guide.

If the screen of the VM flickers, Windows Server 2008 R2 is busy setting up some of the new hardware devices that it is detecting. You should still be able to login and complete the setup of the Evaluation Virtual Appliance environment.

If you would like to optimize the virtual machine for XenServer, install the XenServer tools within the virtual machine.

There is no “smoke and mirrors” involved here, it is literally this easy using the Disk Image Import wizard.

Importing the virtual machine succeeded but with the following warning.

When importing a virtual machine to Hyper-V R2 you might see the following error dialog:

Importing the virtual machine succeeded but with the following warning.

Import completed with warnings.

I have seen this error quite a bit, and I must say that it is no reason for panic.  Your VM is safe.

If you open the error and read the detail, you will see what really went wrong.  (Click on that See details down arrow).

Well, psyche.  That details box is rarely helpful – it simply points you to the proper event log- then you being digging.

Each time I have seen this error the repair has been the same.  Simply open the settings of the virtual machine and connect the virtual NIC to the proper virtual network switch.

Also, each time I have seen this the leading events have been the same.  I created the Export from Hyper-V v1 and I am importing to Hyper-V R2.

Linux vm from VMware to XenServer the videos

If you have been following, you will note quite a few posts related to importing / migrating Linux virtual machines from VMware to XenServer.

I realize that many folks don’t use XenServer – but the basic steps of repairing after migration apply to Hyper-V just as well as to XenServer – the steps are the same if you want your VM to boot. However, PV enablement on Hyper-V does not exist, you just need to install the vm tools.

Here are the links in case you missed them:

• PV enabling an HVM from VMware on XenServer (CentOS RedHat)
• PV enabling an HVM from VMware on XenServer (SLES and Debian)
• Migrating Debian from VMWare ESX to XenServer
• Migrating SLES from VMWare ESX to XenServer
• Migrating CentOS from VMWare ESX to XenServer
• Migrating RedHat from VMware to XenServer

I have turned three of these into short (less than 10 minutes) video presentations, just to add a bit more information than in the articles.

• Repairing SLES 10 boot loader after import to XenServer
• PV enabling SLES 10 vm after import to XenServer
• PV enabling CentOS VM after import to XenServer

PV enabling an HVM from VMware on XenServer (SLES and Debian)

As a condition for paravirtualization to work, a kernel that supports the Xen hypervisor needs to be installed and booted in the virtual machine. Simply installing the XenServer tools within the vm does not enable paravirtualization of the vm.

In this example; the virtual machine was exported as an OVF package from VMware and imported into XenServer using XenConvert 2.0.1.

Installing the XenServer Supported Kernel:

1. After import, boot the virtual machine and open the console.

2. (optional) update the modules within the vm to the latest revision

a. If the kernel-xen package is installed from an online repository – best practice is to fully update the distribution to avoid problems between package build revisions.

3. Install the Linux Xen kernel.

a. yast install kernel-xenpae

i. the xen aware kernel is installed and entries are created in grub

ii. x64 can use kernel-xen, x86 requires kernel-xenpae

iii. This is not the same as installing “xen” which installs a dom0 kernel for running vms, not a domU kernel for running as a vm.

iv. yast is the package installer for SLES, Debian uses apt (apt-get).

4. Modify the grub boot loader menu (the default entries are not pygrub compatible)

Open /boot/grub/menu.lst in the editor of your choice

a. Remove the kernel entry with ‘gz’ in the name

b. Rename the first “module” entry to “kernel”

c. Rename the second “module” entry to “initrd”

i. SuSE and Debian require that entries that point to root device locations described by a direct path such as: “/dev/hd*” or “/dev/sd*” be modified to point to /dev/xvd*

d. (optional) Modify the title of this entry

e. Edit the line “default=” to point to the modified xen kernel entry

i. The entries begin counting at 0 – the first entry in the list is 0, the second entry is 1 and so on

ii. In our example the desired default entry “0”

f. (optional) Comment the “hiddenmenu” line if it is there (this will allow a kernel choice during boot if needed for recovery)

g. Save your changes

1. Edit fstab because of the disk device changes

a. open /etc/fstab in the editor of your choice.

b. Replace the “hd*” entries with “xvd*”

c. Save changes

2. Shut down the guest but do not reboot.

a. Shutdown now -h

Edit the VM record of the SLES VM to convert it to PV boot mode

In this example the VM is named “sles”

5. From the console of the XenServer host execute the following xe commands:

a. xe vm-list name-label=sles params=uuid (retrieve the UUID of the vm)

b. xe vm-param-set uuid=<vm uuid> HVM-boot-policy=”” (clear the HVM boot mode)

c. xe vm-param-set uuid=<vm uuid> PV-bootloader=pygrub (set pygrub as the boot loader)

d. xe vm-param-set uuid=<vm uuid> PV-args="console=tty0 xencons=tty" (set the display arguments)

i. Other possible options are: “console=hvc0 xencons=hvc” or “console=tty0” or “console=hvc0”

6. xe vm-disk-list uuid=<vm uuid> (this is to discover the UUID of the interface of the virtual disk)

7. xe vbd-param-set uuid=<vbd uuid> bootable=true (this sets the disk device as bootable)

The vm should now boot paravirtualized using a Xen aware kernel.

When booting the virtual machine, it should start up in text-mode with the high-speed PV kernel. If the virtual machine fails to boot, the most likely cause is an incorrect grub configuration; run the xe-edit-bootloader (i.e. xe-edit-bootloader –n sles) script at the XenServer host console to edit the grub.conf of the virtual machine until it boots.

Note: If the VM boots and mouse and keyboard control does not work properly, closing and re-opening XenCenter generally resolves this issue. If the issue is still not resolved, try other console settings for PV-args, being sure to reboot the vm and close and re-open XenCenter between each setting change.

Installing the XenServer Tools within the virtual machine:

Install the XenServer tools within the guest:

1. Boot the paravirtualized VM (if not already running) into the xen kernel.

2. Select the console tab of the VM

3. Select and right-click the name of the virtual machine and click "Install XenServer Tools"

4. Acknowledge the warning.

5. At the top of the console window you will notice that the "xs-tools.iso" is attached to the DVD drive. And the Linux device id within the vm.

6. Within the console of the virtual machine:

a. mkdir /media/cdrom (Create a mount point for the ISO)

b. mount /dev/xvdd /media/cdrom (mount the DVD device)

c. cd /media/cdrom/Linux (change to the dvd root / Linux folder)

d. bash install.sh (run the installation script)

e. answer “y” to accept the changes

f. cd ~ (to return to home)

g. umount /dev/xvdd (to cleanly dismount the ISO)

h. In the DVD Drive, set the selection to “<empty>”

i. reboot (to complete the tool installation)

7. Following reboot the general tab of the virtual machine should report the Virtualization state of the virtual machine as “Optimized”

Distribution Notes

Many Linux distributions have differences that affect the process above. In general the process is similar between the distributions.

Removal of VMware Tools was tested following import to XenServer and I do not recommend removal of VMware Tools after the VM has been migrated to XenServer. If it is desired to remove VMware Tools, the vm must be running on a VMware platform when the uninstall command is executed within the VM ( rpm -e VMwareTools ).

Some distributions have a kernel-xenpae in addition to the kernel-xen. If PAE support is desired (or required) in the virtual machine, please substitute kernel-xenpae in place of kernel-xen in the instructions. Please see the distribution notes for full details.

The OVF and OVA dilemma

Here is a really good definition that I can thank my manager for of “OVF”:

OVF is a vendor- and platform-independent format to describe virtual machine metadata and create portable virtual-machine packages. These packages do not rely on a specific host platform, virtualization platform, or guest operating system. However, the virtual disk referenced by the OVF package must be compatible with the target hypervisor.

The Open Virtualization Format is a new and developing standard from the DMTF (Distributed Management Task Force). Its purpose is to form a common framework that any virtualization vendor can use to define a virtual workload. This workload can be simply a single virtual machine; this workload could also contain many virtual machines and also include complex descriptions for networking, applications, EULAs, and other entities. In theory, a single OVF could define all of the workloads within an entire enterprise and those workloads could be transported to a remote site for DR, or moved to a new datacenter. The possibilities are many.

The standard is here: http://www.dmtf.org/initiatives/vman_initiative/OVF_Tech_Note_Digital.pdf

The concept seems simple enough.

When talking to IT Professionals there is always time spent describing OVF and OVA, and where each fits.

“OVF” is the acronym for the standard.

What is referred to as an “OVF Package” is the collection of the workload states, plus a description meta file, plus any other entities that are defined in the description file.

BTW – the description file is also called the OVF file, or OVF meta file or OVF descriptor. This is why this gets really confusing really fast.

Here are the two important terms that get thrown around:

OVF / OVA – these are not the same thing.

An OVF is a collection of items in a single folder. Most commonly this is a description file (*.ovf) a manifest file (*.mf), and virtual machine state files (*.vhd or *.vmdk)

An OVA is a single file. The OVA is the OVF folder contents all zipped into a single file. The purpose of the OVA is when you want to take an OVF and share it, or give it as a download. The OVA needs to be opened into the OVF before it can be consumed.

Currently there are a host of folks working on OVF, you can learn more about the companies involved at the web site: http://www.dmtf.org/initiatives/vman_initiative/

Most commonly folks are running into OVF with Citrix (Project Kensho and XenConvert 2); VMware (VMware Workstation 6.5, Virtual Center 3 update 3, OvfTool, etc.); Sun (Virtual Box 2.2); and IBM. The big problem to date has been that the standards have been evolving during the time that these tools have been available. For example: a VMware v3 produced OVF is distinctly different than a VMware v4 OVF package. Mainly due to the state of the standards at the time the software was written. So, currently there is little cross platform between vendors.

The other problem that is happening right now is that defining a workload in an OVF is great, however, since these are virtual machine based – the operating system within the virtual machine has to deal with the fact that the virtualization hardware underneath it has been modified. Having a virtual machine as an OVF is not the same as having a virtual machine and running it through a conversion engine that repairs the operating system within the vm.

This is all evolving and will come in time.

Virtual Machine Interoperability with Hyper-V and XenServer using OVF

Okay, I know this will sound a bit like marketing hype, but I hope to keep that to a dull roar.

For the past few months I have been involved with a virtual machine interoperability project called: "Project Kensho"

Project Kensho uses virtualization interoperability standards that are currently being developed by the DMTF (Distributed Management Task Force - the people who are bringing us lots of things such as CIM and DMI) to provide a framework for all of the virtualization vendors to use to talk the same language.

You can check the whole thing out here:
http://community.citrix.com/display/xs/Kensho

And..you won't see a bunch of screen shots, because I already did those (they are at the link above)

What does this mean to the administrator that is in the grind all day long?

You can take a VM and move it from one virtualization host to another.

You can export an entire enterprise application environment (some complex application that needs SQL, IIS, Active Directory - and those components must run on individual machines).

You can import items like these from other sources.

Here are some real world examples of what I use this for today:

Exporting a complex or large test environment.
I have an environment of 30 client virtual machines. I can export that to a package, sit on it, rebuild my hypervisor, import it back. I can also import those virtual machines to either XenServer or Hyper-V.
I also have an environment of 4 servers (the virtual disks are 36 Gb each) - but I can export and store and then import and use.

It actually ends up making my life a bit easier.

Go ahead, check it out.

The versatility of Hyper-V Export / Import - fixing Hyper-V VM failover and backup problems.

The Export and Import functions of Hyper-V are not well understood. Are they a simple tool just to move virtual machines about?


I challenge that thinking, the functions can be used for much more, such as 'fixing' some VM behavior in Hyper-V.


First of all - lets explore what these two function do:


Export collects the pieces of a Hyper-V virtual machine and collects them into a single folder (with the same name as the VM) on a volume or at a local path.


Import simply 'attaches' to the exported virtual machine, reads in the configuration, and allows you to run the VM from the location that you imported it from.


Cases where these functions are useful:


The obvious - export, move, import


The not so obvious:


Fixing Failover Clustering of a VM. - When a VM is created the default settings place the configuration and snapshots under the path %programdata% and the virtual disk files under \users\public. If you want your VM to be highly available, you need all of these bits in a shared storage location. Using Export to do this is a really simple solution to collect the VM together and to change its configuration, snapshot, and VHD location to that shared volume. *Poof* a bit of administration magic, you just fixed your VM for failover clustering.


Enabling a VM for easy backup and recovery - All administrators know that backup and restore is an exercise that involves chicken sacrifices, finger crossing, and sweaty palms. You quickly learn how good your backup plan is when you work through the exercise of a performing a staged disaster recovery of a particular VM or application within your enterprise.


Anyway, just like for failover clustering, having your VM neatly bundled in one location makes it easier to comb through volumes and folders within your backup catalog (not everyone uses DFS).


You can also Export a copy of your running VM, just for safe keeping, until that application is upgraded again, then you have a quick machine to use for testing and recovery.