Passing and parsing @ServiceVMComputerNames@

In my past post I mentioned the undocumented Service Settings that SCVMM will automatically fill-in for you and pass to your Application Scripts.

But, how can we pass these?

Some are easy, you just pass them like any other setting since they are relatively short strings.

For example, in testing things for this article I passed @ServiceVMComputerNames@, @ComputerName@, @ServiceName@, @VMID@, and @ServiceID@.  I had no idea how long these might be or what they might look like.

My Test Service had two tiers.  One tier with two VMs, one tier with one VM.  It looks like this:

When I deployed the Service I named it “blah”, I have a tier named “The Tier” and another named “The Other Tier”, and three VMs named “xVM01”, “xVM02”, and “yVM01” (SCVMM applied the numbers using the ## notation). 

Within VM xVM01 I sent and captured all of the settings I have mentioned.

What I got out was:

• @ServiceVMComputerNames@ = “The Tier[xVM01,xVM02]The Other Tier[yVM01]”
• @ComputerName@ = “xVM01”
• @ServiceName@ = “blah”
• @VMID@ = “26fd4a55-a707-4fba-89b5-c6955e4e05a2”
• @ServiceID@ = “741fbf99-e676-4a8b-9df7-096c0be1fd3e”

These short service names you can safely pass using:  myscript.ps1 –paramName @VMID@ or myscrpt.cmd @VMID@

There are lots of examples about that.

It is @ServiceVMComputerNames@ that could get really long and in turn make the command line too long to execute.  So, this one I pass in a bit differently.  To accommodate the length I pipe the setting to my PowerShell script (as I blogged about here).

In the Service Template designer this looks like:

My script receives the object as a pipeline and writes it out.

Within the script:

Param
(
    [parameter(Mandatory = $false, ValueFromPipeline = $true)]
    $serviceNames = ""
)

$logPath = "$env:ProgramData\testPath\"
# write out the string for debugging
$serviceNames | Out-File ($logPath + $startTime.Second + $startTime.Millisecond + ".txt")

Then I can just keep reusing this snip to see what I end up with.

What I in turn do with this input is I parse it into an XML document that I later re-use with other SCVMM Application Scripts.

Param
(
    [parameter(Mandatory = $false, ValueFromPipeline = $true)]
    $serviceNames = ""
)

$logPath = "$env:ProgramData\MyService\"

# make the path so Out-File will be happy
If (!(Test-Path -Path $logPath -PathType Container)){
    New-Item -Path $logPath -ItemType Container -Force
    Start-Sleep 10  # so the OS can register the path
}

# Parse the Service information from SCVMM
$tiers = $serviceNames.Split(']')

$service = New-Object System.Xml.XmlDocument
$root = $service.CreateElement("ServiceTemplate")
$root.SetAttribute("version","1.0")
$root.SetAttribute("createon",(Get-Date))
$root.SetAttribute("createdby","brianeh")

foreach($tierString in $tiers) {

    if($tierString){  #ignore any empties
        $tier = $service.CreateElement("Tier")

        $e = $tierString.Split('[')

            $tier.SetAttribute("Name",$e[0])

            $VMs = $e[1].Split(",")

            foreach ($vmString in $VMs){
                if($vmString){
                     $vm = $service.CreateElement("VM")
                     $vm.SetAttribute("Name",$vmString)
                 }
                $tier.AppendChild($vm)
            }

    }
    $root.AppendChild($tier)
}
$service.AppendChild($root)

$service.Save(($logPath + "ServiceNames.xml"))

There you have it, nice and neat XML.  And your scripts have a clue about themselves.

SCVMM hidden system Service Settings

Lately I have been spending a great deal of time with the System Center Virtual Machine Manger Service model and building Service Templates.

Just understanding what the Service model is, and how it takes the SCVMM VM composition model and brings it up to the enterprise / distributed application level helps you understand where MSFT is headed.  To continue this evolution, take a look at the v1 of Desired State Configuration that has been announced in Server 2012 R2.  But that was a bit off topic.

In the title I mention “system” Service Settings.

If you have spent any time with building SCVMM Service Templates you know that you can define many settings by using the “@MySetting@” notation.  When you deploy your Template and it becomes a running Service whomever is deploying is prompted to fill in the Settings you have defined.

This is excellent, now I have the ability to build a template, give it to you and you can personalize it for your environment.  I have one example back here where I built a template for the first domain controller in a domain.

Now, that is all fine and dandy. But what if you need to know ‘something’ about all the other Tiers in the Service.  Such as; “who are the machines in Tier Q?”  Why; '”Because I need to configure an application on this machine to talk to them.”

Before going farther, understand what when SCVMM executes the application scripts in a Service Template as it is being deployed, the scripts within a single machine all run inside the bubble of that machine.  They only know about themselves.  But most enterprise applications need to know about something other than them.  These are big, scaled-out, distributed applications with multiple server roles that all need to talk to each other.

Well, this is where (courtesy of a MSFT PM, who gave approval for me to write this) I ran across some undocumented Service Settings.  These are Service settings that SCVMM fills in during the “Refresh Preview,” instead of prompting your customer; and will process during the deployment.

In my last post I wrote about turning a string into XML.  My sample string was from the Service Setting; @ServiceVMComputerNames@.

• @ServiceVMComputerNames@ – Each Tier in the Service and the ComputerName within them.  In my mind, this is the handiest.  And you can get very similar information from the Azure Service.Runtime within a PaaS VM.

This gives you a string with the name of each Tier in your Service and the ComputerNames within it.  It is handy information.  In the follow-up post I will give my entire solution to passing and parsing this.

Other system Service settings that SCVMM can directly pass include: 

• @ComputerName@ – the computer name assigned to the VM.
• @ServiceName@ – the name of the deployed Service the VM is a member of
• @VMID@ – the GUID of the VM in SCVMM
• @ServiceID@ – the GUID of the Service in SCVMM

There may be more.  As far as I can tell, these are not documented at this time.  And it takes a bit of work to discover them and the formatting.

Turning a string into an XML document with PowerShell

This is one of those things that always seems like it should be really easy, and straightforward.

Well, if you are deeply familiar with XML it might be.  And If you can pry your string apart into an array using split or regex you have half of the problem tackled.

Lets begin by looking at my string.

$s = "Machine Tier - ScaleOut[Machine03,Machine04,Machine05]Machine Tier[Machine02]"

Ugly thing.  That actually contains three elements of data.

The entire string represents a Service.  The data outside the brackets represents a Tier.  The data inside the brackets is a list of VMs in that Tier.

First, I need to break it all apart.

$a = $s.Split(']');
foreach($b in $a) {
    $c = $b.Split('[');
    "Tier = " + $c[0];
    "VMs = " + $c[1];
}

Tier = Machine Tier - ScaleOut
VMs = Machine03,Machine04,Machine05
Tier = Machine Tier
VMs = Machine02
Tier =
VMs =

Okay, that is close, but not quite there. And still not usable.

First, lets do something about that empty last item in the array.

$a = $s.Split(']');
foreach($b in $a) {
    if($b){
        $c = $b.Split('[');
        "Tier = " + $c[0];
        "VMs = " + $c[1];
    }
}

And now, break apart the VM name string

$a = $s.Split(']');
foreach($b in $a) {
    if($b){
        $c = $b.Split('[');
        "Tier = " + $c[0];
        $VMs = $c[1].Split(",")
        foreach ($vm in $VMs){
            "VM = " + $vm
        }
    }
}

Tier = Machine Tier - ScaleOut
VM = Machine03
VM = Machine04
VM = Machine05
Tier = Machine Tier
VM = Machine02

Okay, now I have something that is usable.  And now I want to turn that into XML.

I have spent a great deal of time searching on PowerShell and XML.  Trying to figure out how to build an XML using PowerShell on the fly in my script.  All the examples always start with a TXT based framework of some type that is in turn manipulated by the author.  Or a file is read, or objects are queried.

I am sorry, but this is not an example of generating an XML using PowerShell as the title suggests.  It is a huge formatted text object that is manipulated.  Such a frustrating example to hit over and over again.

Well, I just have this silly string I parsed apart.  And I want that to be XML.  It already has meaning to the pieces, I just need to make them tags and whatnot.

I mentioned early on that I don’t know much about XML.  I read it, transpose it, consume the XML of others’, I never made my own.  So, I had to visit the wisdom of a developer friend and make sure that i was doing it ‘correctly’ and it was ‘proper’.

In it simplest sense, to create an empty XML document in PowerShell do this: “$service = New-Object System.Xml.XmlDocument” and you have an XML document.  But how do you put things into it.

Okay, this is all about objects, and object manipulation.  You don’t just add tags in.  You create objects that are of the type $service and you add them back to $service in the correct order.

I began with this:

$service = New-Object System.Xml.XmlDocument
$tier = $service.CreateElement("Tier")
$tier.SetAttribute("Name","My Test Tier")

$vm = $service.CreateElement("VM")
$vm.SetAttribute("Name","My VM Name")

$tier.AppendChild($vm)
$service.AppendChild($tier)

I create the XML Document $service.  Then I create an Element of type $service and define a Name and value as an Attribute of that Element.  I repeat this and create a $vm element as well. 

If you query $service, you find that these things aren’t there.  They are three separate objects at this point.  They are all share the same object type of $service.  But nothing more.  Now I assemble them together.

I take the $tier object and I add the $vm object to it as a child.  This nests <vm> under <tier> in the XML.  I then repeat this adding this updated $tier object to $service as a child.

The above is fine enough.  However, I was informed that I was missing a root element.  To define the Document.

$service = New-Object System.Xml.XmlDocument
$root = $service.CreateElement("RootElement")

$tier = $service.CreateElement("Tier")
$tier.SetAttribute("Name","My Test Tier")

$vm = $service.CreateElement("VM")
$vm.SetAttribute("Name","My VM Name")

$tier.AppendChild($vm)
$root.AppendChild($tier)
$service.AppendChild($root)

I have been informed that simply doing what I just showed above is still not quite good enough.  It meets the requirement of a root element but totally missed on the intent or spirit.  We will get back to that.

So, what does this XML document look like?  Well, you can step through $service and try to imagine it in your head our you can send it out to a file and open it in notepad.

$service.Save(".\service.xml")

Open that up and you have:

<RootElement>
  <Tier Name="My Test Tier">
    <VM Name="My VM Name" />
  </Tier>
</RootElement>

Now.  I have some XML.  And I am feeling pretty proud of myself.

Why did I ever do this in the first place?  So I could do this:

PS C:\Users\Public\Documents> $service.GetElementsByTagName("Tier")

Name                                                     VM
----                                                     --
My Test Tier                                             VM

PS C:\Users\Public\Documents> $service.GetElementsByTagName("VM")

Name
----
My VM Name

Now I have associations that I can look for and query against.

Now, the fun part, meshing those two different activities together as one.  And I have the following:

$tiers = $s.Split(']')

$service = New-Object System.Xml.XmlDocument
$root = $service.CreateElement("RootElement")

foreach($tierString in $tiers) {

    if($tierString){  #ignore any empties
        $tier = $service.CreateElement("Tier")

        $e = $tierString.Split('[')

            $tier.SetAttribute("Name",$e[0])

            $VMs = $e[1].Split(",")

            foreach ($vmString in $VMs){
                if($vmString){
                     $vm = $service.CreateElement("VM")
                     $vm.SetAttribute("Name",$vmString)
                 }
                $tier.AppendChild($vm)
            }

    }
    $root.AppendChild($tier)
}
$service.AppendChild($root)

Now, back to that really lazy root element I created.  In practice, that should be some meta information about the XML document itself.  If you look at lots of XML you will see things like creation dates, versions, authors, and a name that is somehow descriptive.

After I create the $root object (with a better name) I just update it with a few attributes and I am good to go.

$root.SetAttribute("version","1.0")
$root.SetAttribute("createon",(Get-Date))
$root.SetAttribute("createdby","brianeh")

Now, a really short example of what I can now do with this information.

# query on a specific VM element with a Name equal to "Machine02"
# The Item(0) returns the XML object itself instead of a reference to it.  $me = $me.ItemOf(0)

$me = ($service.SelectNodes("//VM[@Name='Machine02']")).Item(0)

# What Tier do I belong to?
$me.ParentNode.Name

# Do I have Siblings or am I an only Child?
$me.NextSibling
$me.PreviousSibling
$me.ParentNode.ChildNodes.Count

Note:  be careful.  These queries are in XPath format, and they are case sensitive.

You can also simply walk the XML as PowerShell supports that as a ‘.’ notation path.

Handling freakishly long strings from CMD to PowerShell

This is one of those “quick, blog it before you forget it all” type of posts.  As well as “hey, I think I get it” and “the things you learn”.

I can thank a particular Microsoft PM for even causing me to look into this.  Never would I have expected that I would need to be prepared to handle an “extremely long” string.  What do I mean by extremely long?  Over 8200 characters, that is what I mean.

To quote the individual that caused me to spend time figuring this out: “I don't recommend this .. because <string in question> can expand to a very large string that will overflow the max # of chars you can pass from cmd.”

That is my entire point here.  I am not just in the PowerShell state of euphoria.  I have some process that is invoking PowerShell as a command calling my script and feeding it parameters.  Open a command prompt and type “PowerShell.exe /?” for a flavor of this world.

Well, what is that maximum number?  How big could this be?

Open a command window and start typing.  When you think you are done, keep going.  You will hit a limit, it is just below 8200 characters.  If you try to pass a string to a script and output it, you will be there.  I did this:

.\SampleScript.ps1 my incredibly long string where I kept typing and copy and pasting until I ran out of characters.

Within the script I captured the input in a special way and output the length and the string back to a file.  Note that param property “ValueFromRemainingArguments” (below) if this is not there, each space in that string gets treated like a new argument and within the script you end up with $args as an array.

Param (
   [parameter(Mandatory = $false, ValueFromRemainingArguments = $true)]
   $ComputerNames = ""
)
$ComputerNames.getType() | Out-File C:\users\Public\Documents\ComputerNames.txt
$ComputerNames.length | Out-File C:\users\Public\Documents\ComputerNames.txt -Append
$ComputerNames | Out-File C:\users\Public\Documents\ComputerNames.txt -Append

Also buried in this correspondence with this individual was a suggestion to write an executable and capture the string using stdIn, parse it, and then invoke PowerShell from there. 

Well, I am becoming a bit of a PowerShell junkie, and I am a lot more comfortable there than with C#.  And, I have to think about the poor sole that has to take care of this project when I finish.  Why give them some hack of an exe to take care of?

Lets back up.  StdIn.  What in the world is that?  I have never run across that with PowerShell. I am definitely not a developer either.

The best way that I can describe StdIn is a read stream.  Instead of passing in a string a stream is passed in and parsed when the end of stream is received.  After talking to a developer cohort, I learned that I actually use StdIn in PowerShell quite frequently.  Pipelining “in” uses StdIn.

So doing this:

$someString | .\ComputerNames.ps1

Uses the StdIn method for inputting the data.

But wait.  Okay, a bit of noise about StdIn and cmd limits.  what about the long strings?

Okay, pipeline. 

My test was a 24,000+ character string.

$someString = "However you want to make this really, incredibly long.  Keep going."
$someString.Length
$someString | .\SampleScript.ps1

But you need to change that param line so that it takes the pipeline.

Param
(
[parameter(Mandatory = $false, ValueFromPipeline = $true)]
$ComputerNames = ""
)

If you want to see what happens the other way, go back and try.  If you keep it all in PowerShell it works.  But if you call the PowerShell script from cmd it truncates due to the original issue I was warned about.

So, in the end, I did not have to write some exe that only parses this input, I actually used PowerShell instead.

Here is how I invoke:

%WINDIR%\System32\WindowsPowerShell\v1.0\PowerShell.exe -command "'%ComputerNames' | .\SampleScript.ps1 -otherArg 'gibberish'"

Here is my script:

Param
(
[parameter(Mandatory = $false, ValueFromPipeline = $true, ValueFromRemainingArguments = $true)]
$ComputerNames = "",
[parameter(Mandatory = $true)]
[string]$otherArg = ""
)

$otherArg | Out-File C:\users\Public\Documents\ComputerNames.txt

$ComputerNames.getType() | Out-File C:\users\Public\Documents\ComputerNames.txt -Append
$ComputerNames.length | Out-File C:\users\Public\Documents\ComputerNames.txt -Append
$ComputerNames | Out-File C:\users\Public\Documents\ComputerNames.txt -Append


Thanks for reading. If you ask why? My only answer is because I have to, I have no other option. We don't always have interactive sessions at our disposal. Sometimes we are headless scripts running under some workflow engine.

==

My developer friend just uncovered the following this dies not support additional parameters to the string and may be able to handle strings that are longer yet:

File  Blah.ps1
$c = [Console]::In.ReadToEnd()
"Here"
$c

In use you still take your long input and pipe to the script, but the use of ‘reading in’ the input is more literal (for lack of a better description).

Discovering and initializing a data volume at VM provision

A few posts back I wrote about using PowerShell to find the DVD drive that a particular installer was attached to and then running that command line installer.

To take that a bit further I have a VM that I am deploying, and that VM has an empty VHDX attached to it.

This VHDX is on the second IDE controller (it needs to be available early in the boot process of the OS).  When I provision this VM the first time, I want to find, online, initialize, and format that virtual disk.

$disk = Get-Disk | where {($_.BusType -eq "ATA") -and ($_.IsBoot -eq $false)}

(You could also find the disk if it was on the SCSI controller)

$disk = Get-Disk | where {$_.BusType -eq "SCSI"}

And now for mounting, and formatting the disk.

Set-Disk -InputObject $disk -IsOffline $false

Set-Disk -InputObject $disk -IsReadOnly $false

Initialize-Disk -InputObject $disk -ErrorAction SilentlyContinue -WarningAction SilentlyContinue

$partition = New-Partition -InputObject $disk -UseMaximumSize –AssignDriveLetter

$partition | Format-Volume -FileSystem NTFS -NewFileSystemLabel "Cache" -Force:$true -Confirm:$false

And now you simply continue along with your scripting.

The reason that I capture the new partition to $partition is that there are lots of useful stuff in there for configuring things moving on.  Little things like: $partition.DriveLetter are highly useful.