FirstLogonCommands to configure your images on deployment when user context is required

This is an old trick and I am sure that there are more elegant ways to handle this.  However, I thought I would share how I am using AutoAdminLogon and FirstLogonCommands in my sysprep unattend answer file to do some heavy lifting for me to drive modifying server settings and application install and configuration.

This came about through using an Azure VM role and being forced to complete as much setup as I can without having to get to the console of the VM.  It is amazing how much you can automate when you get creative.

Let me get one thing out of the way:  Can this be used securely?  Sure, why not, with some proper precautions.  First, encrypt your admin user password; second, don’t use the built-in administrator; third, know that no one can get to the console of the machine (totally headless); fourth, disable the admin account and logout as your last step.

All of the settings that I am referring to are in the “Microsoft-Windows-Shell-Setup” section of your unattend.xml answer file.

First – you have to create and provide a password for the local administrator account.  Note, I am naughty by using the built-in local administrator.

<UserAccounts>
  <AdministratorPassword>
    <Value>IdidntEncryptMineButYouShould</Value>
    <PlainText>true</PlainText>
  </AdministratorPassword>
</UserAccounts>

Then I need to enable AutoAdminLogon

<AutoLogon>
  <Password>
    <Value>IdidntEncryptMineButYouShould</Value>
    <PlainText>true</PlainText>
  </Password>
  <Username>Administrator</Username>
  <LogonCount>1</LogonCount>
  <Enabled>true</Enabled>
</AutoLogon>

Then I define all of my tasks that run when the first user with administrator credentials logs on to the machine:

<FirstLogonCommands>
  <SynchronousCommand wcm:action="add">
    <Order>1</Order>
    <CommandLine>C:\MyFolder\vjRedist64\install.exe /q</CommandLine>
    <Description>Install Visual J# Redistribution</Description>
  </SynchronousCommand>
  <SynchronousCommand wcm:action="add">
    <Order>2</Order>
    <CommandLine>%WINDIR%\System32\WindowsPowerShell\v1.0\PowerShell.exe -command start-sleep 120</CommandLine>
    <Description>Wait for j# install</Description>
  </SynchronousCommand>
  <SynchronousCommand wcm:action="add">
    <Order>3</Order>
    <CommandLine>%WINDIR%\System32\WindowsPowerShell\v1.0\PowerShell.exe -command Import-Module ServerManager; Add-WindowsFeature Web-Server; Add-WindowsFeature Web-Asp-Net; Add-WindowsFeature Web-Windows-Auth; Add-WindowsFeature Web-Metabase</CommandLine>
    <Description>Add ASP.Net and IIS6 Metabase compatibility</Description>
  </SynchronousCommand>
  <SynchronousCommand wcm:action="add">
    <Order>4</Order>
    <CommandLine>%WINDIR%\System32\WindowsPowerShell\v1.0\PowerShell.exe -command set-executionpolicy remotesigned -force >> C:\Users\Public\Documents\setExecution.log</CommandLine>
    <Description>Set the ExecutionPolicy to RemoteSigned for the setup script to run</Description>
  </SynchronousCommand>

</FirstLogonCommands>

Note that there is a sequence number, this way you can simulate a workflow by having having your tasks or scripts execute in a synchronous order.  You just have to watch out for those tasks that run off on their own threads and don’t execute within the context of the command windows where the script executes (that Visual J# installer is a perfect example).  You can manage these spawned processes with PowerShell, but not with a batch command.

Using the Azure Fabric to add certificates to your VM Role

A really useful feature of Azure is that it can inject elements into the Role instances as it applies the configuration.

This is super, extra useful because all roles are sysprep’d images.  This includes your VM Roles. 

If you follow the Azure rules for creating your VM Roles you must prepare the VHD image with sysprep.

I don’t think this is very important if you only have one instance – but the Azure assumption is multiple instances of any role.  With that assumption the use of sysprep applies.

The problem is certificates.  If I sysprep my VHD I break the private key of my certificate as a new private key is generated.

The Visual Studio interface does not have a Certificates tab for the VM Role.  However, don’t let this stop you.  It is a simple edit of the Service Definition and the Service Configuration.

In the ServiceDefinition.csdef add a Certificate entry that names the certificate and the certificate store in which to place it.

<Certificates>
  <Certificate name="MyCertificate" storeLocation="LocalMachine" storeName="My" />
</Certificates>

This example places the certificate “MyCertificate” in the Local Machine Personal store.

In the ServiceConfiguration.cscfg add a mapping entry for the certificate you added to your Azure Service and this Role.

<Certificates>
  <Certificate name="MyCertificate" thumbprint="8F4A08C8A0**************A**E482****CF4AB" thumbprintAlgorithm="sha1" />
</Certificates>

This maps the thumbprint that Azure knows to the name you assigned the certificate to the store in which to place it.  And since the certificate you load into Azure includes both the public and private keys the certificate is fully functional once the Role instance is provisioned.

Now, all this being described..  If you use a Web or Worker role – just use the Certificates tab in the GUI.  Hopefully as the VM Role evolves, it will become as easy – for all the same reasons.

PowerShell to add SSL binding to Default Web Site

Azure can be a bit perplexing at times – especially for those of us that are not developers.

This is where we all look at the VM Role as a way to get services running in Azure that have really complex set-ups.

At the same time, we can use Azure features to do part of the work for us.

Let me take the following scenario:  You have an application, this application requires specific IIS Role features, then some long application install, and last of all it has an involved set-up.  The scripting is all worked out, but it takes a really long time and there are a couple settings that have to be manually applied.

Your goal; automate as much as absolutely possible.

In Azure, you can add an SSL certificate to your Service – the Azure Service Certificate.

You can then configure Azure to automatically inject this certificate (public + private key) into your VM after it completes mini-setup in the Azure cloud.

Great!  I can use a certificate in my VM Role and the private key is not broken by the fact that I had to generalize the image with sysprep.  Now what?  I need to actually use this certificate.

I have two methods that I have worked out to add this certificate to the IIS default web site.

First of all – I created a self-signed certificate but I named it using my Azure Service DNS.  So the URL for my site is HTTP://MyService.CloudApp.Net 

(Don’t get all funny, you cannot purchase a cloudapp.net SSL certificate as this is Microsoft’s domain – you have to get your own domain, get your certificate and forward the DNS to your Azure Service URL).

Both of these methods I run within a PowerShell script – the different is how I achieve the results.

The common part before either method runs:

I had told Azure to put my certificate in the Local Machine\Personal store in the definition file:

After the VM exits mini-setup and my script runs I need to go find my certificate:

# Get all of the installed Local Computer certificates that can be used
# With Azure this is a Service Certificate that is defined in the configuration for this Role.

$allCerts = Get-ChildItem cert:\localmachine\my

# Find the Certificate that is defined for the domain name
foreach ($cert in $allCerts) {
    if ($cert.SubjectName.Name -match "CloudApp.Net") {
        $cloudCert = $cert
    }
}

Now, I have the certificate, let’s create the binding in IIS

Method 1 – the netsh way using apphost (Server 2008):

## Older Method to achieve the same thing.
$certThumb = $cloudCert.Thumbprint
## Generate a GUID for the SSL binding
$sslGuid = "{" + ([System.Guid]::NewGuid().toString()) + "}"
## Import the SSL certificate
netsh http add sslcert ipport=0.0.0.0:443 certhash=$certThumb appid=$sslGuid
## Create the SSL binding on port 443 for the Default Web Site
& "$env:windir\system32\inetsrv\appcmd.exe" set config -section:system.applicationHost/sites /+"[name='Default Web Site'].bindings.[protocol='https',bindingInformation='*:443:']" /commit:apphost

Method 2 – the PowerShell IIS module way (Server 2008 R2):

Import-Module WebAdministration

New-WebBinding -Name "Default Web Site" -IP "*" -Port 443 -Protocol https

$certObj = Get-Item $cloudCert.PSPath
New-Item IIS:SslBindings\0.0.0.0!443 -value $certObj

# Remove the HTTP binding
Remove-WebBinding -Name “Default Web Site” -BindingInformation *:80:

The new PowerShell 2.0 WebAdministration module way is much easier to deal with – but, it is PowerShell v2 – that is Server 2008 R2 and newer only.

VM to VM network communication in Azure

VM to VM communication for VMs running on Windows Azure is not just up and straightforward and wide open like it is when you run a VM on a hypervisor in the enterprise.

By design, Role Instances (these are VMs) in Azure have an outer security wrapper around them.  This establishes that final trust boundary for the VM container.  You can see this when you use VM Role.  Because beyond your Windows Advanced Firewall you have an additional level of block to your network connectivity.

This is where you must define your endpoints (and you must know your application and the required ports very well) within your Azure Service.

In the properties of any Azure Role is the Endpoints property.  these are little port specific pin-pricks in the Azure VM runtime container specific to network traffic.  You poke a hole, you allow incoming traffic on that particular port.  Don’t get hung up on the endpoint term – it is very much a developer type term for “something that I can connect to, or talk to.”

The Name is whatever you (or your developer) calls it.  the type is Input or Internal.  The Protocol is HTTP, TCP, or HTTPS.  Public ports apply to Input endpoints.  Private port applies to both Input and Internal endpoints. 

An Input endpoint is where your Azure Service is exposed to the wild world.  It is the public port that anyone can touch. 

An Internal endpoint is internal to your Azure Service and is where any Role Instance can touch another Role Instance over the network.  Here you set the Private Port number.

This opens a port in the envelope that allows incoming traffic to the role instance where this is defined.  And in the case of a public port, it maps that through a load balancer to a public IP.  Otherwise, your internal traffic is over an internal IP subnet (not a public range).

I envision this to look a little like this:

Now, the tricky part comes next.  The setup of your application in Azure can be exceedingly complex because of one rule that Azure has; your OS images are prepared with sysprep.  This is so that any Role can exists as multiple instances and be totally unique.

Under the hood, there are traditional Windows Setup and Deployment tricks happening all over the place.  Unattended setup, pre-tasks, post-tasks, user elevation, injection of applications, certificate installation – all kinds of things are happening or can happen when a Role Instance is provisioned.  This is all related to the Azure way of doing things.

Think about this.  You have an enterprise application.  Does it support being sysprep’d?  Many don’t.

Also, how does your application behave without any type of name resolution?

And, how do you discover your other role instances without connecting to and logging into each one?

For Web and Worker roles Microsoft wants you to use the AppFabric or Azure Storage (blobs, queues, tables) to enable this communication.  But what if re-writing is not an option?

Random Wait PowerShell Script

During my work with Azure I have run into cases where the provisioning of service instances causes flooding of some other component – such as the backend Azure SQL database as my instances register themselves into a configuration database.

To get around this very traditional “black hole” type of problem I have a very simple PowerShell script that I run in the proper sequence as a First Logon Command as my instances complete mini-setup.  (The OS is prepared with sysprep so everything has to go through mini-setup).

The command line in my unattend.xml looks like this:

<FirstLogonCommands>
  <SynchronousCommand wcm:action="add">
    <Order>1</Order>
    <CommandLine>%WINDIR%\System32\WindowsPowerShell\v1.0\PowerShell.exe -command set-executionpolicy remotesigned -force >> C:\Users\Public\Documents\setExecution.log</CommandLine>
    <Description>Set the ExecutionPolicy to RemoteSigned for the script to run</Description>
  </SynchronousCommand>
  <SynchronousCommand wcm:action="add">
    <Order>2</Order>
    <CommandLine>%WINDIR%\System32\WindowsPowerShell\v1.0\PowerShell.exe C:\Tanzanite\RandomWait.ps1 >> C:\Users\Public\Documents\RandomWait.log</CommandLine>
    <Description>A random wait time to prevent storming and corrupting the database</Description>
  </SynchronousCommand>
  </FirstLogonCommands>

You will notice that the first thing I have to do is to issue a command to set the ExecutionPolicy.  After mini-setup the ExecutionPolicy is reset to Restricted and my script will not run.

BTW – FirstLogonCommands are executed when an Administrator logs on to the machine for the first time.  So, prior to this I am using AutoAdminLogon to set the administrator to logon and then these scripts execute.

Now, the PowerShell script.  (RandomWait.ps1)

<#
.SYNOPSIS
    This is a frivolous script, strictly to be called to introduce a random wait period
    within a five minute window, based on a random number of seconds.
.DESCRIPTION
    This script is designed to facilitate VM instance application setup in the Azure Environment.
    To prevent database corruption during deployment this random wait
    is introduced after provisioning and called as an UNATTEND.xml FirstLogonCommand
    during mini-setup.
.LEGAL
    SCRIPT PROVIDED "AS IS" WITH NO WARRANTIES OR GUARANTEES OF ANY KIND, INCLUDING BUT NOT LIMITED TO
    MERCHANTABILITY AND/OR FITNESS FOR A PARTICULAR PURPOSE.  ALL RISKS OF DAMAGE REMAINS WITH THE USER, EVEN IF THE AUTHOR,
    SUPPLIER OR DISTRIBUTOR HAS BEEN ADVISED OF THE POSSIBILITY OF ANY SUCH DAMAGE.  IF YOUR STATE DOES NOT PERMIT THE COMPLETE
    LIMITATION OF LIABILITY, THEN DELETE THIS FILE SINCE YOU ARE NOW PROHIBITED TO HAVE IT.  TEST ON NON-PRODUCTION SERVERS.
.AUTHOR
    Brian Ehlert, Citrix Labs, Redmond, WA, USA
.REFERENCES
    Thank you TechNet. For examples.
#>#

# Add the Service Runtime snap-in to the standard Windows PowerShell command shell.
add-pssnapin microsoft.windowsazure.serviceruntime

# Take the VM Instance offline with Azure
Set-RoleInstanceStatus -Busy

# Get the random number generator object so as not to require PoSh v2.0
# PoSh v2 introduces Get-Random
$randomNo = new-object system.random

# Generate a random number between 1 and 300
# using seconds instead of minutes as chance for it to be not equal to other instances is higher.
$number = $randomNo.Next(1, 300)

Start-Sleep -Seconds $number

Creating the VM Role in an Azure Service

A VM Role is configured as a Role within a larger Hosted Service.  This is no different from a Web or Worker role except for the additional steps of creating and uploading the base VHD that the Role instances will be provisioned from.

The following example uses Visual Studio 2010 with the Azure SDK Installed to define and publish the service. It is important that the latest Azure SDK is installed, the VM Role capability is enabled, and Visual Studio is already configured to manage your subscription.

Create the VM Image

1. Begin by creating a VM using Hyper-V

   1. The VHD size cannot exceed 15 Gb for an Extra Small VM, 30 GB for a Small VM, or 64 GB for a Medium to Extra Large VM.

   2. http://msdn.microsoft.com/en-us/library/gg465391.aspx

2. Install Server 2008 R2

   1. Apply Windows Updates

   2. Add .Net Framework 3.5.1

3. Install and configure your application(s)

   1. Be aware that the image will be prepared with sysprep. When provisioned, the VM will be unique and will complete mini setup.  Therefore the application must be in a state that is compatible with sysprep.  Any additional configuration that must be performed after mini-setup is complete should be scripted using a custom service, or defining Syncronous Commands in the unattend.xml created by step 4.

4. Install the Azure VM Integration Components.

   1. http://msdn.microsoft.com/en-us/library/gg465409.aspx

   2. This generates an unattend.xml at the root of the system drive (with the Administrator password prompted for).  This unattend.xml is very specific to the VM successfully provisioning on Azure.  Sections can be added to the XML file, but not deleted.

5. Install Azure Connect endpoint agent if it will be used.

6. Apply any modifications to the unattend.xml to support installing or configuring your application.

   1. A combination of scripts or commands and the FirstLogonCommands and SynchronousCommand keys can be combined with the AutoLogon to orchestrate the actions.

7. Run sysprep to generalize the image

   1. http://msdn.microsoft.com/en-us/library/gg465407.aspx

   2. sysprep /generalize /oobe /shutdown /unattend:"C:\unattend.xml"

Upload the VM (vhd) image

1. Upload the VHD using the csupload tool that is included with the SDK.  This is accessed by launching the SDK Command prompt using "Run As Administrator".

   1. http://msdn.microsoft.com/en-us/library/gg465385.aspx

Author and Deploy an Azure Service

1. http://msdn.microsoft.com/en-us/library/gg465379.aspx

   1. External endpoints are ports that are available publically in front of the load balancers and mapped to a particular role.

   2. Internal endpoints are available behind the load balancers and can be used to define specific endpoints (open TCP ports) for machine to machine communication. 

   3. Be aware that you cannot use FQDNs or DNS or the like – the intercommunication involves enumeration of role instances using Azure Service Runtime components combined with proper configuration using IP addresses.

The Azure VM Role reboot reset – understanding the pale blue cloud

1/5/2011 update:
Only one day has gone by since I originally posted this – and I must say that this has been a very interesting adventure.  The detiled discussion is in the comments.  However, this is a real and valid scenario that a developer should plan for.

Here is a bit more insight into the behavior of VMs in Azure – and one more point that VM Role is NOT a solution for Infrastructure as a Service.
Lest begin with a very simple, one VM scenario:
With a VM Role VM – you (the developer, the person that wants to run your VM on Azure) uploads a VHD into what is now VHD specific storage, in a specific datacenter.
You then create an application in the tool of your choice and define the VHD with the service settings – this links your VHD to a VM definition, firewall configuration, load balancer configuration, etc.
You deploy your VM Role centric service and sit back and wait – then test and voila! it works.
You do stuff with your VM, the VM life changes and all is happy – or is it.
Now, you – being a curious individual – click the “Reboot” button in the Azure portal.  You think, cool, I am rebooting my VM – but you aren’t you are actually resetting your service deployment.  You return to your VM Role VM to find changes missing.
This takes us into behaviors of the Azure Fabric.  On a quick note – if you wanted some type of persistence, you need to use Azure storage for that.  Back to the issue at had – your rolled back VM.  Lets explore a possibility for why this is.
BTW - This behavior is the same for Web Roles, and Worker Roles as well – but it is the Azure base OS image, not yours.
Basically what happened was no different than a revert to a previous snapshot using Hyper-V, or the old VirtualPC rollback mode.  When a VM is deployed there is a base VHD (this can be a base image – or your VM Role VHD) and there is a new differencing disk that is spawned off.
You selecting reboot actually tossed out the differencing disk which contains your latest changes and created a new one, thus reverting your VM Role VM.  This is all fine and dandy, however my biggest question is:  What are the implications upon authentication mechanism such as Active Directory – AD does not deal with rollbacks of itself or of domain joined machines very well at all.
My scenario is that you are using Azure Connect Services connecting back to a domain controller in your environment – you join the domain, someone clicks reboot and your machine is no longer domain joined or you have a mess of authentication errors.  Again, this is not the Azure model.
The Azure model I in this case is that your VM reboots at the fabric layer back to the base image (they recommend that you prepare with sysprep) and it re-joins your domain as a new machine – with all of the pre-installed software.
This is all about persistence and where that persistence resides.  In the VMs of your service there is no persistence, the persistence resides within your application and its interaction with Azure storage or writing back to some element within the enterprise.
This is important to understand, especially if you think of Azure as IaaS  - which you need to stop doing.  It is a platform.  It is similar to a hypervisor but it is not a hypervisor in your interaction with it as a developer or ITPro. 
In a nutshell what happened is that during the reboot of my VM the Azure Fabric considered the VM unhealthy and thus provisioned a new one.  It could be that the differencing disk could not be written back to the root VHD, it could be that “something” in my VM is not as the fabric wants it so it considered it bad and provisioned a new one.
Regardless – this is valid behavior – it is behavior to understand and plan for – and again emphasizes that if you want persistence you must design it in by writing your application state out to Azure Storage (or enterprise storage using Azure Connect) in some way in order to guarantee persistence.
All very interesting.