Showing posts with label cloud. Show all posts
Showing posts with label cloud. Show all posts

Thursday, June 13, 2013

Turning a string into an XML document with PowerShell

This is one of those things that always seems like it should be really easy, and straightforward.

Well, if you are deeply familiar with XML it might be.  And If you can pry your string apart into an array using split or regex you have half of the problem tackled.

Lets begin by looking at my string.

$s = "Machine Tier - ScaleOut[Machine03,Machine04,Machine05]Machine Tier[Machine02]"

Ugly thing.  That actually contains three elements of data.

The entire string represents a Service.  The data outside the brackets represents a Tier.  The data inside the brackets is a list of VMs in that Tier.

First, I need to break it all apart.

$a = $s.Split(']');
foreach($b in $a) {
    $c = $b.Split('[');
    "Tier = " + $c[0];
    "VMs = " + $c[1];
}

Tier = Machine Tier - ScaleOut
VMs = Machine03,Machine04,Machine05
Tier = Machine Tier
VMs = Machine02
Tier =
VMs =

Okay, that is close, but not quite there. And still not usable.

First, lets do something about that empty last item in the array.

$a = $s.Split(']');
foreach($b in $a) {
    if($b){
        $c = $b.Split('[');
        "Tier = " + $c[0];
        "VMs = " + $c[1];
    }
}

And now, break apart the VM name string

$a = $s.Split(']');
foreach($b in $a) {
    if($b){
        $c = $b.Split('[');
        "Tier = " + $c[0];
        $VMs = $c[1].Split(",")
        foreach ($vm in $VMs){
            "VM = " + $vm
        }
    }
}

Tier = Machine Tier - ScaleOut
VM = Machine03
VM = Machine04
VM = Machine05
Tier = Machine Tier
VM = Machine02

Okay, now I have something that is usable.  And now I want to turn that into XML.

I have spent a great deal of time searching on PowerShell and XML.  Trying to figure out how to build an XML using PowerShell on the fly in my script.  All the examples always start with a TXT based framework of some type that is in turn manipulated by the author.  Or a file is read, or objects are queried.

I am sorry, but this is not an example of generating an XML using PowerShell as the title suggests.  It is a huge formatted text object that is manipulated.  Such a frustrating example to hit over and over again.

Well, I just have this silly string I parsed apart.  And I want that to be XML.  It already has meaning to the pieces, I just need to make them tags and whatnot.

I mentioned early on that I don’t know much about XML.  I read it, transpose it, consume the XML of others’, I never made my own.  So, I had to visit the wisdom of a developer friend and make sure that i was doing it ‘correctly’ and it was ‘proper’.

In it simplest sense, to create an empty XML document in PowerShell do this: “$service = New-Object System.Xml.XmlDocument” and you have an XML document.  But how do you put things into it.

Okay, this is all about objects, and object manipulation.  You don’t just add tags in.  You create objects that are of the type $service and you add them back to $service in the correct order.

I began with this:

$service = New-Object System.Xml.XmlDocument
$tier = $service.CreateElement("Tier")
$tier.SetAttribute("Name","My Test Tier")

$vm = $service.CreateElement("VM")
$vm.SetAttribute("Name","My VM Name")

$tier.AppendChild($vm)
$service.AppendChild($tier)

I create the XML Document $service.  Then I create an Element of type $service and define a Name and value as an Attribute of that Element.  I repeat this and create a $vm element as well. 

If you query $service, you find that these things aren’t there.  They are three separate objects at this point.  They are all share the same object type of $service.  But nothing more.  Now I assemble them together.

I take the $tier object and I add the $vm object to it as a child.  This nests <vm> under <tier> in the XML.  I then repeat this adding this updated $tier object to $service as a child.

The above is fine enough.  However, I was informed that I was missing a root element.  To define the Document.

$service = New-Object System.Xml.XmlDocument
$root = $service.CreateElement("RootElement")

$tier = $service.CreateElement("Tier")
$tier.SetAttribute("Name","My Test Tier")

$vm = $service.CreateElement("VM")
$vm.SetAttribute("Name","My VM Name")

$tier.AppendChild($vm)
$root.AppendChild($tier)
$service.AppendChild($root)

I have been informed that simply doing what I just showed above is still not quite good enough.  It meets the requirement of a root element but totally missed on the intent or spirit.  We will get back to that.

So, what does this XML document look like?  Well, you can step through $service and try to imagine it in your head our you can send it out to a file and open it in notepad.

$service.Save(".\service.xml")

Open that up and you have:

<RootElement>
  <Tier Name="My Test Tier">
    <VM Name="My VM Name" />
  </Tier>
</RootElement>

Now.  I have some XML.  And I am feeling pretty proud of myself.

Why did I ever do this in the first place?  So I could do this:

PS C:\Users\Public\Documents> $service.GetElementsByTagName("Tier")

Name                                                     VM
----                                                     --
My Test Tier                                             VM

PS C:\Users\Public\Documents> $service.GetElementsByTagName("VM")

Name
----
My VM Name

Now I have associations that I can look for and query against.

Now, the fun part, meshing those two different activities together as one.  And I have the following:

$tiers = $s.Split(']')

$service = New-Object System.Xml.XmlDocument
$root = $service.CreateElement("RootElement")

foreach($tierString in $tiers) {

    if($tierString){  #ignore any empties
        $tier = $service.CreateElement("Tier")

        $e = $tierString.Split('[')

            $tier.SetAttribute("Name",$e[0])

            $VMs = $e[1].Split(",")

            foreach ($vmString in $VMs){
                if($vmString){
                     $vm = $service.CreateElement("VM")
                     $vm.SetAttribute("Name",$vmString)
                 }
                $tier.AppendChild($vm)
            }

    }
    $root.AppendChild($tier)
}
$service.AppendChild($root)

Now, back to that really lazy root element I created.  In practice, that should be some meta information about the XML document itself.  If you look at lots of XML you will see things like creation dates, versions, authors, and a name that is somehow descriptive.

After I create the $root object (with a better name) I just update it with a few attributes and I am good to go.

$root.SetAttribute("version","1.0")
$root.SetAttribute("createon",(Get-Date))
$root.SetAttribute("createdby","brianeh")

Now, a really short example of what I can now do with this information.

# query on a specific VM element with a Name equal to "Machine02"
# The Item(0) returns the XML object itself instead of a reference to it.  $me = $me.ItemOf(0)

$me = ($service.SelectNodes("//VM[@Name='Machine02']")).Item(0)

# What Tier do I belong to?
$me.ParentNode.Name

# Do I have Siblings or am I an only Child?
$me.NextSibling
$me.PreviousSibling
$me.ParentNode.ChildNodes.Count

Note:  be careful.  These queries are in XPath format, and they are case sensitive.

You can also simply walk the XML as PowerShell supports that as a ‘.’ notation path.

Thursday, September 8, 2011

The Cloud–a place for my stuff

I keep trying to come up with new ways to describe clouds and what it means to run things in the cloud and to do things in the cloud.

To quote a really old George Carlin skit; a place for my stuff.  “That is the whole meaning of life, to find a place for my stuff”.

In the cloud world there are private places and public places.  A private cloud is a cloud that you are responsible for.  A public cloud is a cloud that someone else is responsible for.

The “location” of your cloud infers the physical location and who owns and manages that physical location.  If it is private, then you own or manage that physical location.  If it is public then someone else does.

In the end.  A cloud is not a new thing, but a new word describing an old thing.  It describes infrastructure.  The new thing is how you interact with that infrastructure – this is where underwear gets all knotted up.

When moving to clouds – someone loses control.  And it is that change in control that things someone other than me is messing with my stuff.

Personally, I love my SkyDrive.  Cloud based storage that I can interact with from anywhere.  A place for my document and picture (files) stuff.

I also like Gmail and Hotmail.  Most folks don’t think of these as cloud either, but they are.  Because they are off, somewhere else.  And, they are a place for my email stuff.

I like Azure.  It is my cloud based infrastructure.  I don’t have to worry about controlling it, because Azure does that.  I just tell it to run my stuff and it does.  A place for my workload stuff.

So, for me.  I have a place for my stuff.  I know where that physical place is.  Coming form the IT Pro background I have a need to know that physical place to link my stuff with a location, even if that location is a service.

But, I just feel good that I have a place for my stuff.  And that I can get to my stuff.  From any of my other stuff.  And I don’t have to manage all aspects of my stuff.

Wednesday, March 23, 2011

SCVMM 2012 beta with XenServer support

The SCVMM 2012 beta was announced at MMS.  Go pick it up here:  http://www.microsoft.com/downloads/en/details.aspx?FamilyID=e0fbb298-8f02-47e7-88be-0614bc44ee32

Now, I will warn you.  This is a big update. Not just some little incremental update.  It includes some new ways of thinking about your data center.  And it includes a lot of “cloud” type words and ways of thinking.

I consider this update a paradigm shift in the VMM world.  A big change in thinking, automation, abstraction, packaging, and delivering.

It also includes support for managing XenServer.  This is a feature that is near and dear to me.  There is a supplimental pack that is required for your XenServers which can be obtained from here: http://www.citrix.com/xenserver/microsoft-beta

This gives the ability to deploy too and manage XenServer.  You still need to setup your environment using xe or XenCenter (no different than Hyper-V) – but you can control and manage VMs with SCVMM.

Other MSFT folks have mentioned the other big paradigm changes, so I won’t cover them here.  http://blogs.technet.com/b/scvmm/archive/2011/03/22/system-center-virtual-machine-2012-beta-available-now.aspx

Friday, December 24, 2010

Fabric is infrastructure

Azure is this platform. It is a bunch of VMs but it isn't a bunch of VMs. There are roles and instance of roles (which are technically VMs).

Then there is this mysterious thing called 'fabric'. The fabric is the secret sauce, it is the thing that makes it all work.

For those of us who have been in enterprise IT - we know all about fabric. We have built it, we have managed it, we have fixed it.

This is a term that is not Azure specific by the way - you will see it appearing in more and more places - fabric is infrastructure. Azure fabric is a bit more - but you will be seeing this fabric term more frequently in the future.

In the simplest sense, fabric is infrastructure. It is the server, the hypervisor that runs on it. It is the storage and storage management. It is the network; the load balancers, the subnet, the VLANs. It is also the management layer that brings all of these things together.

It is this management layer that provides the real keys - the orchestration of events. Provisioning of storage, a VM on a hypervisor, booting that VM, configuring its firewall and networking, installing / instantiating / injecting / inserting / configuring the application, setting the virtual interfaces of the load balancers, and verifying the "health" of your application.

All of this is what the fabric does and it is really valuable, important, and useful.

If you want to geek out (in an IT Pro sort of way) and want to learn more - here are some presentations by Mark Russinovich describing the entire thing in deep detail at PCD 2010

http://player.microsoftpdc.com/Session/18a38105-520f-486a-9e04-d956736e506d

Thursday, November 18, 2010

What is the Private Cloud?

What is this “Private Cloud” thing anyway?

The simple term is that it is a pool of compute resources that runs on hardware that you own (it is in your datacenter). 

But, that just means that I am running virtual machines, right?

Not exactly.  It means that there is some type of automation framework in front of the hypervisors and the virtual machines that is providing some type of orchestration.  This could be XenServer + VMLogix or Hyper-V + SCVMM.

At the same time, virtual machines are not required – but it is easier with virtual machines.

HPC from Microsoft, the old fashioned mainframe, the Cray – these are examples of private clouds that have been around, performing massive calculations, they are orchestrated; they meet the definition.  But they require custom written applications so they work well in very specific situations.

The recent “Private Cloud” term tossed out by Microsoft I think can be summarized in this way:

  • Private Cloud is owned by you.
  • Private Cloud is dynamic (in some way)
  • Private Cloud is at least Hyper-V (but it can be more as you need it to be).
  • Private Cloud has OEM reference hardware designs (these are tested and known to work, but they are not the only option)
  • Private Cloud is NOT all boxed up and sitting on a shelf somewhere. (it is not a SKU that you purchase)
  • Private Cloud is not defined by a specific set of prescribed technologies.
  • Private Cloud must be pieced together by you (the architect, the enterprise, the planner – you must know what your requirements are).
  • Private Cloud is NOT Microsoft only technologies.

Private Cloud is your enterprise datacenter.  Enabled to be nimble and adaptive through technologies that make it dynamic.  Some of these technologies are:  Hyper-V, SCVMM, SSP, Operations Manager, Configuration Manager (basically any of the System Center group added on top of your infrastructure that happens to be running as virtual machines).

In the Citrix world I could describe the same, or I could have a mash-up of Citrix products with Systems Center products.  The end result would be the same – a Private Cloud.

This is where the big disconnect is happening and all of the confusion is currently happening.  Private Cloud is a term, a term that summarizes a conglomeration of technologies that are working together to make an IT shop more efficient, more nimble, more self-service.

Wednesday, November 17, 2010

Cloudy in the Private Public and Hosted worlds

Today there was a thread in the forums that brought up the new buzz word: “Private Cloud”

There is currently a lot of hype, questions, and confusion about this term – it is not a new term by the way – since it was used at TechEd 2010 in Berlin.

The original poster appears just as confused as the majority of folks as to what this term might mean.  And in the industry so far this seems to be the current state of things.  The question:  “What is a cloud?” 

My take on “cloud” is that it is a pool of compute resources that works in some way and it is in some place.  That seems to be the cross vendor understanding of cloud.

Today there are a few “cloudy” terms floating around.  Private, public, and hosted.

Private Cloud is a cloud that your (as an enterprise) own and run.  It is your traditional datacenter.  But set-up and managed in a way where it does not matter where the application runs as long as the application runs.

Public Cloud is a cloud that is “out there” and runs at a facility owned by someone other than you.  And the key is that you purchase compute resources on that cloud.  Azure is a really good example of this.  Amazon EC2 as well.

Hosted Cloud is a cloud that is “out there” and runs at a facility owned by someone other than you as well.  They key here is that it is provisioned and billed more like a co-location facility.  You have “machines” that you purchase – not highly generic capacity.  This is more like the Rackspace or upcoming Azure VM Role models.

 

The most interesting part of the entire thread that caused me to begin this post in the first place is that HPC was mentioned by the original poster.  You know, you could think of HPC as a private cloud.  Just as a good ‘ole mainframe could be a private cloud.

Here is how I responded to the thread – simply to broaden the thinking beyond a simple hypervisor:

I wanted to jump in to this thread as there are two distinctly different technologies being discussed. And since the OP is coming from the education industry both or neither could apply.

The term "Private Cloud" is simply a set of processing resources that you own and therefore control. It is not remote. That is all that we really know today. Beyond that, it is not clear if this is a combination of technologies, a SKU bundle, a pre-installed hardware package, etc.

HPC, Azure Appliance, and Hyper-V + SCVMM are all options that would give you something that you could refer to as a private cloud.

HPC is a platform designed for distributed processing - many individual nodes that provide CPU and contribute back to the primary job. This can be thought of as very mainframe-ish. I over-simply the concept and think of it as the SETI@Home or Folding@Home model - as it is a large compute. This requires custom written applications.

Azure is a platform - that is making breaks in the education realm right now - but it is not private. It is a pool of processing resources that can be scaled out and back on demand. It is HPC-like in concept, but Hyper-V-like in large scale implementation. MSFT owns and manages the hardware, you own the application - you purchase capacity during a slice of time on the Azure platform. This requires custom written applications.

Azure Appliance - this is the cloud version of Azure, but you own it, it is your datacenter, you house the physical hardware - all the other Azure rules apply. You simply don't pay MSFT for time as you own the hardware.

Hyper-V + SCVMM - this is the current model of using virtual machines to run enterprise applications. This allows you to run off-the-shelf applications installed on to operating systems - just as you would with physical hardware. Hyper-V is the hypervisor, SCVMM is the management stack for the virtual machines. Adding the VMM SSP 2.0 in front of SCVMM provides additional functionality.

Each one of these fits in different places and for different reasons. And they could all be referred to as "cloud" in some way or another. As it is very unclear what "cloud" really means beyond being a pool of compute resources.

In the end, it really comes down to what you require. Large complex calculations would gravitate to HPC or Azure. More traditional workloads would gravitate toward Hyper-V + SCVMM.