Creating disk fragmentation in virtual disks

Yes, you read that right. I WANT to fragment a hard drive. I want to do it well. I want it relatively quickly (one day vs. three years). And, you know what, it isn't as easy as you might expect.

I mean, I could install a Windows XP virtual machine, and then update it. But, it is use over time that causes fragmentation. The writing and deleting of files. The intermix of small and large files. Unless you have an old server sitting around you don't have it.

Oh, and I want to do this in a virtual machine. Hmm...So that means I can only go backward about six years at the very most. And who has had a single virtual machine running for six years? (at least that I can get my hands on..)

Needless to say, I have been busy the past couple days with PowerShell.

My theory is this: Write a bunch of little files. Randomly delete them. Write some big files. Randomly delete them and check the result.

My results have been promising.  I may need to tweak it a bit to be sure to break near non-standard disk sizing increments.  But for a first try it was not too bad.  On a side note.  Nothing can simulate fragmentation like a PC that has been running in continuous service for many years, without a good 3rd party defragmentation.

Here is my PowerShell script:

# This is a script designed to highly fragment a system disk in one pass.
# The theory is to nearly fill the disk with 128kb files, then to delete
# one-half of those files, then populate the disk with 2Mb files, then remove a portion of those.
#
# I am going to use randomization in the delete functions, therefore it is unknown exactly
# how many files might be deleted, because an empty array location might be chosen.
# Initiate the random number function
$Random=New-Object System.Random (get-date).millisecond
# Get the System Disk
$systemDisk = get-wmiObject win32_LogicalDisk | where {$_.DeviceId -eq $Env:SystemDrive}
# determine 10%
$tenPercent = $systemDisk.Size / 10
# Move to to the system Temp folder where we will do the workset-Location $Env:Temp
################################################################################ Write lots of 128kb files
###############################################################################
write-host "Write lots of 128kb files"
# An array for the names
$128kNames = @()
#Create the sample 128k file
$fileToCopy = "128kSample.txt"
for ( $i = 0; $i -lt 21843; $i++) { "1" | Out-File $fileToCopy -append }
While ($systemDisk.FreeSpace -gt $tenPercent) { Clear-Host Write-Host "Writing small files" Write-Host "Remaining free space is: " $systemDisk.FreeSpace Write-Host "Cycle: " $128kNames.Count
# Set the file name
$file = [string]$systemDisk.FreeSpace +".txt" copy $fileToCopy $file
# Write the file name to the array
$128kNames = $128kNames + $file


# Requery the disk $systemDisk = get-wmiObject win32_LogicalDisk | where {$_.DeviceId -eq $Env:SystemDrive} }
################################################################################ Delete half of the 128kb files
###############################################################################
write-host "Delete half of the 128kb files"
# 1/2 of names in the array
$half = $128kNames.Count / 2
While ($half -gt 0) {
Clear-Host Write-Host "Deleting small files" Write-Host "Remaining free space is: " $systemDisk.FreeSpace Write-Host "Countdown: " $half
$rndName = $128kNames[$Random.Next(0, $128kNames.Count)]
Remove-Item $rndName $half-- }
# Requery the disk
$systemDisk = get-wmiObject win32_LogicalDisk | where {$_.DeviceId -eq $Env:SystemDrive}
# Clear the small name array
$128kNames = 0
################################################################################ Write lots of 2Mb files
###############################################################################
write-host "Write lots of 2Mb files"
# An array for the names
$2mbNames = @()
#Create the sample 2Mb file
$fileToCopy = "2MbSample.txt"
for ( $i = 0; $i -lt 349512; $i++) { "0" | Out-File $fileToCopy -append }
While ($systemDisk.FreeSpace -gt $tenPercent) {
Clear-Host Write-Host "Writing large files" Write-Host "Remaining free space is: " $systemDisk.FreeSpace Write-Host "Cycle: " $2MbNames.Count
# Set the file name
$file = [string]$systemDisk.FreeSpace +"2mb.txt" copy $fileToCopy $file
# Write the file name to the array
$2mbNames = $2mbNames + $file
# Requery the disk
$systemDisk = get-wmiObject win32_LogicalDisk | where {$_.DeviceId -eq $Env:SystemDrive} }
################################################################################ Delete half of the 2Mb files
###############################################################################
write-host "Delete half of the 2Mb files"
# 1/2 of names in the array
$half = $2mbNames.Count / 2
While ($half -gt 0) {
Clear-Host Write-Host "Deleting large files" Write-Host "Remaining free space is: " $systemDisk.FreeSpace Write-Host "Countdown: " $half
$rndName = $2mbNames[$Random.Next(0, $2mbNames.Count)]
Remove-Item $rndName $half-- }
Clear-HostWrite-Host "Finally done"
# What is the damage?
# THIS DOES NOT WORK ON WIN XP AND EARLIER - THIS CLASS IS NOT AVAILABLE
# $sysVolume = get-wmiObject win32_volume | where {$_.DriveLetter -eq $Env:SystemDrive}

# $sysVolume.DefragAnalysis

The Azure VM Role reboot reset – understanding the pale blue cloud

1/5/2011 update:
Only one day has gone by since I originally posted this – and I must say that this has been a very interesting adventure.  The detiled discussion is in the comments.  However, this is a real and valid scenario that a developer should plan for.

Here is a bit more insight into the behavior of VMs in Azure – and one more point that VM Role is NOT a solution for Infrastructure as a Service.
Lest begin with a very simple, one VM scenario:
With a VM Role VM – you (the developer, the person that wants to run your VM on Azure) uploads a VHD into what is now VHD specific storage, in a specific datacenter.
You then create an application in the tool of your choice and define the VHD with the service settings – this links your VHD to a VM definition, firewall configuration, load balancer configuration, etc.
You deploy your VM Role centric service and sit back and wait – then test and voila! it works.
You do stuff with your VM, the VM life changes and all is happy – or is it.
Now, you – being a curious individual – click the “Reboot” button in the Azure portal.  You think, cool, I am rebooting my VM – but you aren’t you are actually resetting your service deployment.  You return to your VM Role VM to find changes missing.
This takes us into behaviors of the Azure Fabric.  On a quick note – if you wanted some type of persistence, you need to use Azure storage for that.  Back to the issue at had – your rolled back VM.  Lets explore a possibility for why this is.
BTW - This behavior is the same for Web Roles, and Worker Roles as well – but it is the Azure base OS image, not yours.
Basically what happened was no different than a revert to a previous snapshot using Hyper-V, or the old VirtualPC rollback mode.  When a VM is deployed there is a base VHD (this can be a base image – or your VM Role VHD) and there is a new differencing disk that is spawned off.
You selecting reboot actually tossed out the differencing disk which contains your latest changes and created a new one, thus reverting your VM Role VM.  This is all fine and dandy, however my biggest question is:  What are the implications upon authentication mechanism such as Active Directory – AD does not deal with rollbacks of itself or of domain joined machines very well at all.
My scenario is that you are using Azure Connect Services connecting back to a domain controller in your environment – you join the domain, someone clicks reboot and your machine is no longer domain joined or you have a mess of authentication errors.  Again, this is not the Azure model.
The Azure model I in this case is that your VM reboots at the fabric layer back to the base image (they recommend that you prepare with sysprep) and it re-joins your domain as a new machine – with all of the pre-installed software.
This is all about persistence and where that persistence resides.  In the VMs of your service there is no persistence, the persistence resides within your application and its interaction with Azure storage or writing back to some element within the enterprise.
This is important to understand, especially if you think of Azure as IaaS  - which you need to stop doing.  It is a platform.  It is similar to a hypervisor but it is not a hypervisor in your interaction with it as a developer or ITPro. 
In a nutshell what happened is that during the reboot of my VM the Azure Fabric considered the VM unhealthy and thus provisioned a new one.  It could be that the differencing disk could not be written back to the root VHD, it could be that “something” in my VM is not as the fabric wants it so it considered it bad and provisioned a new one.
Regardless – this is valid behavior – it is behavior to understand and plan for – and again emphasizes that if you want persistence you must design it in by writing your application state out to Azure Storage (or enterprise storage using Azure Connect) in some way in order to guarantee persistence.
All very interesting.