Thursday, April 9, 2009

Never resize a VHD with snapshots or differencing disks – more about differencing disks

I keep seeing this item coming up in the forums over and over again.

For some reason I need to make the virtual disk of my VM larger (maybe I downloaded too many patches, maybe too many temp files, maybe too many log files).

The reason does not really matter.

So I open the Edit Disk wizard, I find the VHD and I Expand the virtual disk.

Excellent, now my VM has more space and I go into the operating system and diskpart and perform the actions that I need to for the OS to use the additional space.

This works all to well when all you have is a single VHD.  One VHD, no snapshots, no differencing disks.

I have written about snapshots quite a bit over the past year.

A snapshot is a moment in time.  The technology that Hyper-V uses to achieve this is a Differencing Disk.

The differencing disk is a child disk that is linked to its parent.  The easiest way to describe the concept of the differencing disk is to think of overlays.

In the era of the small hard disk we used to think about sectors and blocks.  But things have changed, and most folks don’t concern themselves with that anymore – a disk is a place for stuff and that place has a capacity and that is all we really concern ourselves with.  It is the blocks and sectors that form the map that we need to be aware of.  Think of it like a dart board, but wil ar more rings.

The root VHD is the map of a disk.  A digital representation of a physical hard disk.  It has blocks, sectors, etc.

The differencing disk is also a map just like its parent, however, it only contains the blocks and sectors of its parent that are different, than the parent.

To begin my analogy, I am going to think of the single VHD as a map or drawing.

DiskMapRoot

Adding a differencing disk (taking a snapshot) is like laying a piece of tracing paper over a drawing, and only coloring in some features (the changes from the point the snapshot was made). 

DiskMapChild1

The drawing does not change, only the way the empty places are colored in.

If you add a second differencing disk (now a chain of three virtual disks – a grandparent VHD, a parent differencing disk – a child differencing disk).  This lays a second piece of tracing paper over the drawing, and we proceed to change how the empty spaces are colored in.

DiskMapChild2

What happens when you resize the VHD?

When you resize the VHD, the disk map or drawing, is changed.  I have a different combination of sectors and blocks.

DiskMapResize

Now, the spaces that I colored in on my tracing paper (differencing disks) no longer aligns with the empty spaces in the drawing.  The result is a really ugly picture.

DiskMapResizeOverlay

The result with my VM is that I just broke the chain.  And I broke it right at the root.

There is currently no method (that I know of) that will allow resizing of a virtual disk that contains snapshots – differencing disks.

If you need to resize the virtual disk of a VM and there are snapshots then the snapshots must be deleted (and the VM turned off, then wait patiently).  This has the end effect of taking each piece of tracing paper and applying its colors onto the layer underneath.   The term is called merging.

The layers of tracing paper are applied back to the primary drawing, the root VHD.

In the end, you have a single VHD that you can safely resize.

Is this the only way you can end up with a single VHD?  No I can think of at least a couple more creative ways to end up with a single VHD for resizing.

The key is that you cannot resize a virtual disk with snapshots or differencing disks.

I hope that someone that is thinking about doing this finds this article in Google or Live Search and saves themselves some big recovery pain.

14 comments:

Eric Stoll said...

Of course I expanded a VHD that has snapshots without realizing what was going to happen. Is there any way to undo this?

BrianEh said...

The only way that I have had luck 'undoing' this is to restore the origional VHD from a backup.

Another possibility is to recreate the base VHD if you know the exact size and the exact items that were installed, then you repoint the first (oldest) AVHD to it.

If the differences are too great then the disk edit wizard won't allow the join to happen.

bala said...

Is there any linux utility/program to merge these snapshots. The utility should find the difference block by block and the modified block of orginal snapshot should be patched, something like.

Thanks,
Bala

BrianEh said...

There are utilities (not Linux) that have been developed that do the block by block comparison in an attempt to create something different.
I am unaware of any utility (at this time) that actually does a repair of a differencing disk chain where one member has been modified.
And you are right - it is the difference between manipulation at the block level vs. the file level.

robr said...

Ugh. Guess what I just did... to our Exchange Server.... and guess who doesn't have a backup of the original VHD. Is there a company that I can contact has the know how to write a tool to undo what I did? Obviously Microsoft could, but I don't think they do contracting work like this :).

BrianEh said...

It is rather a difficult process.

If you have any backup that you can return to that matches a point in time that a snapshot was taken, then this can be restored as a previous VHD.

I am guessing that the snapshot was condiered the undo tool for resizing the virtual disk? You don't have a back-up as well?

Any back-up can be used to restore your system, however your snapshot history is lost, but you would have your system.

This goes right along with the statement "don't use snapshots as a method of back-up" - they are not the same thing.

robr said...

I understand that if I had a backup of the VHD, I could use that. I do not have that backup. The IT company responsible for setting up and performing our backups never added the physical server (or any of the VMs) into the backup routine.

That being said, I'm working with Chris Eck to repair the damage that has been done, so there may be some light at the end of the tunnel. Huge thanks to Chris for taking his valuable Sunday afternoon to help me.

BrianEh said...

VHDTool.exe on CodePlex has just been updated to solve this very problem!!

Go check it out and sleep a little better (but make those backups!)

http://code.msdn.microsoft.com/vhdtool

robr said...

I was the guinea pig that tested it for Chris. He had hoped it would restore the VHD back to its previous bootable state, and while it didn't boot after, I did manage to recover all the data by rolling in the snapshot after the repair. Chris is pretty certain it did not boot because something was done to update it after the size change, which is possible as a third party was working on the issue with me.

Thanks a ton Chris!

BrianEh said...

I have seen a very similar issue (the not booting issue) seems that a VHD has both a virtual and physical geometry.

The physical geometry was the important part that was necessary to get the VHD to boot.

Anonymous said...

Nice Article.

If the snapshot is just the 'tracing paper' (AVHD) why does it have to merge when the snapshot is deleted and no longer needed?

Greg

BrianEh said...

Ah, the snapshot is deleted, the data isn't.

The implementation is that you delete the existance of the snapshot (the ability to return to a point in time) not the data of the snapshot.

The model that MSFT uses is designed to not lose data.

It is the same reason that if you delete a VM using the Hyper-V Manager that you end up with a single VHD left behind. (SCVMM is different - you perform a adelete that and it is all gone). Two differnet products, two different theories.

KDAD said...

Brian,

Thank you SO MUCH for this info and for mentioning vhdtool. That tool saved our production exchange server tonight. Ok, really it saved ME. I'm on my way over to thank the author now.

Gary

Kyle C said...

FYI, the vhdtool doesn't work on VHDX files. However, I was able to resolve this for a coworker. I made copies of both the vhdx and avhdx, then did an edit of the avhdx, which gave me the only option of reconnect with parent vhdx. I reconnected it to the new copy of the vhdx instead of the original. Then I ran an edit on the avhdx again, and this time was given the normal options of resize, convert, merge. I selected merge, then used the option of merging to a NEW VHDX file. The new file was back to the original size of the disk prior to the resize/snapshot debacle and was fully bootable. I attached this newly created VHDX to the a new VM created with the same settings as the old and then fired it up.