Paravirtulization under the hood and more

For those of you that are hard core virtualization folks, there is an excellent couple of articles over at xen.org by George Dunlap from Citrix.

In the ESX world and Hyper-V world the virtualization is closer to the HVM type or PVHVM when the OS is enlightened.  Xen has grown from a different root and started from the paravirutalization world (true PV, it is actually kind of interesting how the VMs themselves boot in this world).

This also gives a bit of background into the terminology and options that are available.

There is a part 1: http://blog.xen.org/index.php/2012/10/23/the-paravirtualization-spectrum-part-1-the-ends-of-the-spectrum/

and part 2: http://blog.xen.org/index.php/2012/10/31/the-paravirtualization-spectrum-part-2-from-poles-to-a-spectrum/

Personally, I think it good reading for anyone working with machines as it is a history of evolution in one aspect.

At this same time we have MSFT Research working on the Library OS.  This is an interesting abstraction of applications into VM type containers, application containers.  This is more similar to the traditional Xen PV model, where (technically) there isn’t a boot kernel in there, just the runtime components of the machine and the bootstrap comes from the xen hypervisor itself.  (at least that is my impression of it).

The MSFT research project known as Drawbridge: http://research.microsoft.com/en-us/projects/drawbridge/

And a bit more: http://research.microsoft.com/apps/pubs/default.aspx?id=141071

And a Channel9 presentation for the short attention spans among us:  http://channel9.msdn.com/Shows/Going+Deep/Drawbridge-An-Experimental-Library-Operating-System

Other MSFT Research OS projects: http://research.microsoft.com/en-us/groups/os/

Is this the future?  the Application level virtualization that was discussed many years ago.  Decoupling the application from the OS?  Not really the decoupling, but the forcing of an application into a container.  A container that it cannot get out of and affect other applications.

I look at this and think about traditional application compatibility issues going away, true application throttling, true isolation of a session (and its applications) within a Terminal Server.  That is what really makes me think about where this is all headed.  And we continue to be just at the beginning of it all.

My Azure Public Endpoint switcher

I recently worked through an issue with the Persistent VM beta in Windows Azure around remotely managing my VMs (machines as I will generically call them).

Now, my scenario is that I have multiple machines in Azure behind a single Service (DNS name at cloudapp.net).  You can of course bring your own DNS name and simply mask this, you are not stuck with it.

In order to get a remote session to each one, only one can have the RDP or SSH port at any time for the Service. 

So the setup resembles this:

• brianEhApp.cloudapp.net:3389 -> VM1:3389
• brianEhApp.cloudapp.net:56789 -> VM2:3389
• brianEhApp.cloudapp.net:54321 -> VM3:3389

This is relatively straightforward.  However, I ran into a situation what I was not able to authenticate to the higher ‘ephemeral’ ports that Azure was automatically generating as the administrative endpoint for my +1 machines.

So, I could use the Portal to change one VM then back out to the other and so on.

However, this was taking quite a long time.  And I am impatient.

So, I turned over to the Azure PowerShell cmdlets and whipped out the following script.  It switches the public endpoints for me and launches my administrative RDP session straight away.  Considerably faster than messing with the Portal.

$myVm = Read-Host -Prompt "what is the VM name you want to access via RDP?"
$myService = Read-Host -Prompt "what is the Service name where the VM resides?"
"The assumption is that the endpoint name is RDP"

# Check all the endpoints of all the VMs and move off 3389 to set desired VM to 3389
$deploymentVms = @()
$deploymentVms = Get-AzureVM -ServiceName $myService 

# First all VMs in the Service need to be off 3389 or there will be an error setting the correct one
foreach ($vm in $deploymentVms){
    $vmEnds = @()
    $vmEnds = Get-AzureEndpoint -VM $vm
    Foreach ($end in $vmEnds){
        If ($end.Port -eq 3389 -and $vm.Name -notmatch $myVm){
            $pubPort = Get-Random -Minimum 49152 -Maximum 65500
            Set-AzureEndpoint -Protocol tcp -LocalPort 3389 -PublicPort $pubPort -Name $end.Name -VM $vm | Update-AzureVM
        }
    }

}

# Now, set the desired VM Public endpoint to 3389
Get-AzureVM -ServiceName $myService -Name $myVm | Set-AzureEndpoint -Protocol tcp -LocalPort 3389 -PublicPort 3389 -Name RDP | Update-AzureVM

Get-AzureRemoteDesktopFile -ServiceName $myService -Name $myVm -Launch

Azure Virtual Machine the PowerShell Basics

Really, I am not going to go into a lot of explanation here.  I am assuming that you have some PowerShell background and need a bit to get started.  That is what I am covering.

You can find the cmdlets here:

https://www.windowsazure.com/en-us/manage/downloads/

Be sure to have your Azure management certificate properly stored in your Personal certificate store prior to connecting to your subscription.

These first commands are pretty much mandatory when you begin a PowerShell session.

Import the module:

​import-module 'C:\Program Files (x86)\Microsoft SDKs\Windows Azure\PowerShell\Azure\Azure.psd1'

Import a settings file (this speeds up as it lists all subscriptions you have access to - to create this file perform Export-AzurePublishSettingsFile (Visual Studio also uses this))

Import-AzurePublishSettingsFile 'C:\Users\Public\Documents\BrianEhServices.publishsettings'

Choose the subscription that you will interact with for your session:

Select-AzureSubscription -SubscriptionName "Sample Subscription"

Set the default Storage account that will be used (it must be in the same subscription)

Set-AzureSubscription -SubscriptionName "Sample Subscription" –CurrentStorageAccount SampleStorageAccount

By the way, do the above is handy.  Like me I assume that you all have alt least two Azure subscriptions (yours and the one your company has given you access to).  Using that settings file allows easy switching.  Now that you have that, go exploring.

If you have a Service or have deployed a Virtual Machine from the Gallery using the Portal you can query it, change it, dispose of it.

If you have VHD images you can manipulate those.  Now, disclaimer here, my experience so far is that CSUPLOAD from the Azure SDK is still the best way to get VHDs into Azure storage and it now supports both stateless and persistent VHD images.  It differentiates between the two because it registers them with the VHD repository for you and sets its life in motion.

But, I assume that you are getting itchy, so lets just begin with making a new Virtual Machine in a new Service.  The very same thing that you would get if you used a Gallery VM image (this is not the Quick VM as that would be New-AzureQuickVM).

First we need to find an image:

List all available images: Get-AzureVMImage

List all available in a table: Get-AzureVMImage | Format-Table

Find images that have been uploaded to your Storage account ('user' images): Get-AzureVMImage | where { ($_.Category -eq "user") }

Now I just want to use the Server 2012 Gallery image and create a VM (we will build on this command).

$svr2012Image = Get-AzureVMImage | where { ($_.Category -eq "Microsoft") -and ($_.Label -match "Server 2012" ) -and ($_.ImageName -match "Datacenter") }

Apply a customization configuration to the image:

$myImage = New-AzureVMConfig -Name MyNewMachine -InstanceSize ExtraSmall -ImageName $svr2012Image.ImageName
Add-AzureProvisioningConfig -VM $myImage -Windows –Password <complex password>

Create the Service and Provision the VM:

New-AzureVM -ServiceName "MyNewService" –VMs $myImage

Notice after it is finished that Azure automatically created an RDP endpoint to allow remote OS access (if it was a Linux image an SSH endpoint would have been created).

Azure IaaS Virtual Machine concepts

The Azure IaaS Virtual Machine is in Beta and anyone can request access.

For those have worked with Azure VMs in the past (Web, Worker, and VM Role) this is simply an addition, a new option.

There are some interesting contrasts, parallels, and dependencies when we consider both the PaaS and IaaS options (MSFT likes to draw the contrast that way, but fundamentally they are still machines what you can do and need to know is different).

MSFT already has lots of documents out there on the IaaS stuff.  I don’t intend to regurgitate what they have already documented.  They stuff they have not documented is far more interesting.

And there is a gallery of OS images that you can build machine from using the GUI portal.  And you can create and upload your own VHD images (a bit like VM Role).

In my next post I will begin exploring the PowerShell that is available to drive this stuff from the ground, that is a bit more interesting (at least to me).

First of all, what is the Virtual Machine?  If it isn’t a Web, Worker, or VM Role; what is it.

It is a bit like a VM Role, but without any agents to interact with the Azure Fabric, and the storage supports persistence.  Beyond persistence, it is just a machine.  Mind you, the way I state this it does not sound like much, but it took a lot for MSFT to work through extrapolating the PaaS provisioning and storage into supporting this.  There is a lot there.  Just go look up the TechEd talks.

Now, we all know that MSFT al about platforms, so the Virtual Machine by itself is just as uninteresting as it sounds.  Add in Virtual Networks and it gets a bit more interesting.

Virtual Networks allow defining of address allocation spaces.  And those can be divided into subnets.  If you know the SCVMM IP Pool model, it is similar to that with some implementation differences.

A Virtual Network is also a container or a boundary.  All machines within a Virtual Network can freely communicate with each other over the network. No matter the IP that was allocated to the VM, no IP subnet isolation.  A Virtual Network can also span services – provided they are in the same Affinity group and Region.  It all links.

A Virtual Network also supports the Gateway.  The Gateway is a way to ling the Virtual Network defined in Azure with an SSL VPN endpoint on the ground, in your enterprise.  This is just like any point-to-point VPN that you have used to connect branch offices, or to another business.  Before this we only had Azure Connect and its support for IPv6 and that it only acted as a machine to machine forwarder, fundamentally different.

One other concept is the VHD.  There are two types.  Image and OS. 

An Image can be provisioned into multiple VMs and is prepared so that Azure can customize it.  The Windows OS is prepared with sysprep, and the Linux OS is prepared with the Azure Linux Agent deprovision command.  The concept is the same.

When a new VM is created from an image it can and will be customized.  And, the Fabric also automatically creates a management endpoint (SSH or RDP).

An OS can be used by one Virtual Machine.  It is not provisioned, and therefore not customized.  It is just created an powered on.  You have to define everything, but it allows you to bring your own, pre-configured machine into Azure (without building it in space).

In the next few posts we will get a bit more into this and specifically driving it all with PowerShell.

The last concept is the Services.  IaaS Virtual Machines still run within a Service just like PaaS VMs.  But the two shall not mix.

You can create a Service without a deployment, this is androgynous, lacking either PaaS or IaaS gender.  As soon as the deployment is defined the service becomes either IaaS (meaning that it can run Virtual Machines) or PaaS (meaning that it can run VMs (or Roles)).  Thus the entire Service becomes stateless or persistent.

The two can interact over the network, but not through Azure Fabric Services.

I think that in the next post I am going to dive straight into the PowerShell.

PowerShell v3 everywhere

If you have not caught wind of this yet, you can download and install the “Windows Management Framework 3.0” on your boxes that are not Server 2012 or Windows 8.

As in the past this is more than just PowerShell, it includes WMI and WinRM compatibility updates.  (just not a big BITS update like v2 did).

You can find it here: http://www.microsoft.com/en-us/download/details.aspx?id=34595

This is for Windows 7 SP1 and Server 2008 SP2 or Server 2008 R2 SP1.

And, don’t forget Update-Help after you install it.

But you might as well be on your way to managing your Server 2012 / Windows 8 infrastructure the PowerShell way.

If you need .Net 4 you can find that for Server over here: http://www.microsoft.com/en-us/download/details.aspx?id=17718

And for Server Core here: http://www.microsoft.com/en-us/download/details.aspx?id=22833

If you run an OS that is older.  I am sorry, MSFT does not look back.  You can be one version old, but XP / 2003 is way out.