Monday, May 7, 2018

AWS exposes route tables so I can recover from their bug

The cloud.  It is a wondrous thing.  When it works.
But I think that one thing that most all can agree on, it should 'just work'.  All aspects of your cloud experience should 'just work'.

In the past weeks I have shifted my attention to AWS. 
It has taken some time to get used to referring to a virtual network under the marketing term Virtual Private Cloud. Or a virtual machine as an 'EC2 instance' or any number of other marketing term focused things.
My preference here, call it what it is, not the marketing name for the feature.
Enough of that.

Back to the title.  One of the first things that I thought interesting with AWS is that within a VPC, the route table is exposed to me.
Why would I want to / need to muck around with a route table in the cloud?  I have port level firewalls rules (Security Groups), I have to stand up an Internet Gateway to enable outgoing traffic.  Why would I ever need to muck with something as low level as a route table?

Well, I can tell you from being burned on this multiple times now, it is so I can fix what the AWS portal screws up for me.

Back to my original statement - this is the cloud, it should 'just work'.  Networking is a pretty fundamental thing here.  It needs to be solid, resilient, and always functional.
But yet, at least three times now, in less than three months I have lost days due to multiple route table entries in a VPC that I created through the portal.

And usually the kicker is that one of the route table entries is correct and there is a second one that is empty.  A route table entry, but no defined route.

The really strange things happen when things work say manually or with one VM, and then you start automating and suddenly what you automate does not work.
As if one deployment uses the correct route table and the other doesn't.

It is one of those obscure things, that as a customer, I expect to work - all the time.  I should not have to think about this, I should not have to remember (in the recovery from frustration) that I had been burned by this in the past and go looking, only to discover that I have this strange named but empty route table entry.

My call to AWS, fix it.  Don't let it be mucked up in the first place.  Give me success here.  Don't frustrate me, don't waste my time.

But then, you did get paid for extra days of running compute while i tried to figure it all out.  So I guess that you might not have my interest at heart.

Thursday, January 4, 2018

Finding likes between two arrays

In a recent interview I was asked to write a function to sole the following problem:

You have a SQL query and two arrays, find the like elements between the two arrays.

(I am assuming that a few of you have your own opinions on how to solve this, and that is excellent - please enlighten me)

The first thought that popped into my head was that this was not a coding interview, the second was that i really needed a laptop to work through this, and the third was - doesn't PowerShell have a cmdlet that will do this for me?

Back at home, I thought I would investigate the PowerShell angle.  Since I had access to a laptop.
I am assuming a small dataset

First, lets build a couple arrays to validate with:

$array1 = "Elmer", "hunter", "Bugs", "rabbit", "Tweety", "bird", "Sylvester", "feline"
$array2 = "Eddie", "investigator", "Roger", "rabbit", "Jessica", "rabbit", "Judge", "doom", "tweety", "bird"

Now, I did add some variation for fun.  Because why might this be an important skill?
It might be important because I might need to mine a bunch of data or logs to investigate a pattern.

At this point I have arbitrarily added an additional requirement on myself; I want to see all of the matched data items, not just the matched values.  Equivalents vs. equals if you will.
Why? Again, in an investigation I will probably want to use a fuzzy match instead of a literal match.  And then I will want to work through the resulting set again.  (most likely both with eyes and with code).

In regards to PowerShell doing this for me, I discovered this:

$compared = Compare-Object -ReferenceObject $array1 -DifferenceObject $array2 -IncludeEqual

But when looking at the output, working with this is not very intuitive and it hides my fuzzy match and detail output.

PS C:\Users\Brian> $compared

InputObject  SideIndicator
-----------  -------------
rabbit       ==
Tweety       ==
bird         ==
Eddie        =>
investigator =>
Roger        =>
Jessica      =>
rabbit       =>
Judge        =>
doom         =>
Elmer        <=
hunter       <=
Bugs         <=
Sylvester    <=
feline       <=

I actually ended up going back to my original idea; 

$likes = @()

foreach ($element1 in $array1){
    foreach ($element2 in $array2){
        if ($element1 -like $element2){
            $likes += $element1;
            $likes += $element2;
        }
    }
}

while not highly efficient and I am sure not wonderful with large data sets, it does give me the results that I wanted for further analysis:

PS C:\Users\Brian> $likes
rabbit
rabbit
rabbit
rabbit
Tweety
tweety
bird
bird

It is easy to see from this output where I might want to go next; counts, deeper analysis, trends, etc.  It all depends on the details in the records used.

If you have different ideas that meet my requirements, please share.


Tuesday, November 7, 2017

Service Principals and Azure AD

There are a few scenarios with Azure AD that folks commonly run in to.

I have not blogged about the relationships of an Azure Tenant and Azure AD.  So let me briefly mention the relationship between these two.
(I will be repeating this all in future posts, as this relationship is important to grok)

An Azure Tenant is the Account that you or your company has in Azure.  It is divided into Subscriptions. 
Each Subscription is where you 'consume' in Azure; and it also serves as an isolation boundary (as in, resources in different services can only talk to each other through public entry points - they cannot directly touch each other).
Below that you have Resource Groups, which are management containers (not isolation).
And then you have the actual resources that you get billed for consuming.

All of this together is an Azure Tenant - or phrased another way, you are a Tenant of Azure and this is your playground and thus billing entity.
I will keep using the phrase Azure Tenant in this way.
(I run across MSFT folks that use the word Subscription when referring to the Tenant, and it is just plain incorrect and thus confusing as to the ramifications)

Azure AD is an entirely separate thing.  It is this huge multi-tenant cloud based identity provider, with a number of cool features and touch points.
An Azure Tenant must have an associated Azure AD, but an Azure AD has no dependency on an Azure Tenant (or an Office365 tenant - which is yet another entity).

A single company can have multiple Azure AD's (which is highly likely), they could also have multiple Azure Tenants (which is not very likely, but possible).

A single Azure Tenant can only be associated with one Azure AD.  Nuance here; the tenant has one Azure AD, but people from other Azure ADs can be granted access.  But the invited accounts are foreign accounts.

I bring all of this up since it is this multi-pronged association that usually gets folks into a pickle.

The "Service Principal" is a term that has been within the IT world for a long time.  It describes the user account that a particular application or service runs under.  If that application needs to access resources on the network, the Service Principal user account for that particular service is used.

This has worked well for decades in the enterprise with Active Directory.  And now we need to take this to the cloud.  And it works a bit differently with Azure Active Directory.

Within Azure Active Directory the service principal is called an "App Registration".  And just like with the enterprise, you would have a unique app registration for each application.
This is different than a user account that you simply grant access to Azure resources.  While it really appears to be the same thing, it actually isn't.

Now, as we put things together a number of questions arise.  Which Azure AD should my app registration reside in?  Is there any limitation on the user account?  What resources does the app registration need to access?

Most likely you are using the app registration to interact with your Azure tenant resources.  For example, the app registration is used to provision new machines, or to power machines on and off.  Essentially to perform lifecycle events on your Azure resources.

This means that the app registration must be "native" to the Azure Tenant Azure AD - regardless of what Azure AD your user accounts reside in.  You cannot simply use a random user account.

If you manually create your app registration, you need to be a user administrator in your Azure AD.  And then you can grant the permissions to the Subscription or Resource Groups and you are golden.

https://itproctology.blogspot.com/2017/04/manually-creating-service-principal-for.html

If you are using some other app to create the app registration programmatically things immediately get unique.  All of a sudden your user account matters.
And to programmatically create an app registration, your user account needs to be native to the 
Azure AD.
This is a special kind of native.  The Azure AD account that you use must be an "onMicrosoft" account within that Azure AD.  You cannot use a user account that is synchronized with your on-premises Active Directory.  It does not even matter if your user is a domain admin.

Each Azure AD begins life as an "onMicrosoft" entity.  If you look at the properties of the Azure AD, you will see the information about its original creation, such as brianeh.onmicrosoft.com.  And this remains even if you add a vanity domain, such as ITProctology.com to the Azure AD.

I mention all of this because you might be in a position where you need to create a special Azure AD account, so that your app can create its own app registration (Citrix Cloud does this, RDMI does this).

In your Azure AD, create a new cloud user, using name@domain.onmicrosoft.com  This will create a cloud user that is native only to the Azure AD, and this user will have the full permissions to the API that it needs to create an app registration.

In your app you need to then use the credentials of this new user account so that it can access the API and self create its app registration.

A lot of explanation just to get you to the answer, but the why is often as important as the what.


Monday, October 9, 2017

The gotchas of Azure AD Domain Services in ARM

Not too awful long ago Azure Active Directory Domain Services moved over to the ARM portal from the Azure Classic portal.

Yea! the world said.  And there was much rejoicing.

Now, the real world impacts of this.

There are a few scenarios with Azure AD that folks commonly run in to.

I have not blogged about the relationships of an Azure Tenant and Azure AD.  So let me briefly mention the relationship between these two.
(I will be repeating this all in future posts, as this relationship is important to grok)

An Azure Tenant is the Account that you or your company has in Azure.  It is divided into Subscriptions. 
Each Subscription is where you 'consume' in Azure; and it also serves as an isolation boundary (as in, resources in different services can only talk to each other through public entry points - they cannot directly touch each other).
Below that you have Resource Groups, which are management containers (not isolation).
And then you have the actual resources that you get billed for consuming.

All of this together is an Azure Tenant - your phrased another way, you are a Tenant of Azure and this is your playground and thus billing entity.
I will keep using the phrase Azure Tenant in this way.
(I run across MSFT folks that use the word Subscription when referring to the Tenant, and it is just plain incorrect and thus confusing as to the ramifications)

Azure AD is an entirely separate thing.  It is this huge multi-tenant cloud based identity provider, with a number of cool features and touch points.
An Azure Tenant must have an associated Azure AD, but an Azure AD has no dependency on an Azure Tenant (or an Office365 tenant - which is yet another entity).

A single company can have multiple Azure AD's (which is highly likely), they could also have multiple Azure Tenants (which is not very likely, but possible).

A single Azure Tenant can only be associated with one Azure AD.  Nuance here; the tenant has one Azure AD, but people from other Azure ADs can be granted access.  But the invited accounts are foreign accounts.

Now.  Some background into the processes that get us into the strange places that folks end up in.

When an Azure Tenant is created an Azure AD is created for it.
So you end up with some Azure AD such as you@tenantName.onmicrosoft.com

This is fine.  It gets you up and running and then you add your admins which might be your.admin@yourcomapny.com  and they get invited.  Everything in the Azure Portal works.  Now, lets get into the cases that won't work in this scenario.

This actually puts you in a very common scenario, the scenario where the Azure AD associated with your Azure Tenant is not the same Azure AD where your corporate user accounts reside.

Now, if you only want to use Azure AD RBAC to add your IT folks to the Azure Tenant for administrative purposes, this is fine.  And thus, a more common scenario than most folks want to realize.

Now, lets get to Azure AD Domain Services in ARM.  There is a security boundary here.  The Azure Tenant.  Therefore, when you enable AAD DS in ARM you are restricted to the Azure AD that is associated with the Azure Tenant.

Oh, your Azure Tenant Azure AD is not the one with your users?  Oh my!  How do we resolve this?
(trust me, the sarcasm is real here, I can't tell you how many times I have spoken to folks about this and it takes a while for all the dots to connect before they realize the dilemma).

There are three ways to resolve this:
  1. 're-parent' the Azure Tenant.  What does this mean?  It means that you make some other Azure AD the primary Azure AD for your Azure Tenant.  There is an option in the Azure Portal "Move to another Directory".  The impacts:  If you had any RBAC set up, you will break it, and therefore need to set it up all over again.
  2. Use a vNet.  When logged on to the Azure AD where the users are, create a new Azure Tenant and subscription.  Turn on AAD DS there.  Then set up a gateway between the vNet in this subscription where your AAD DS and user accounts are, and a vNet in a subscription of the Azure Tenant where your workloads are that require the domain services provided by AAD DS.
  3. Don't use AAD DS.  Stand up a Windows Server Domain Service VM (more than one for a proper deployment) and use Azure AD Connect to sync the users with the AD domain.
In the end, this is about your user accounts, the reasons why you wanted AAD DS in the first place (you need NTLM or Kerberos for any reason).
Yes, AAD DS is convenient, but the security models that forced it into this particular assumption is not always in line with reality.

Friday, October 6, 2017

Day two as a free agent - looking sideways

If you missed the tale, I was RIF'd and I thought I would spend some time blogging about the experience and the whatever I decide to do next.

All of this went down on Wednesday, October 4, 2017 and pretty much wrapped up by noon.  Out of the building. 
My user account was gone by 2pm.  (the things a calendar invite that you included your personal email account as an attendee can show you - when that other attendee is an object that can't be resolved)

Talking last night, the wife was lamenting that she could not ping me on Skype any longer.  I mentioned that I do still have a Skype for Business account thanks to the MVP program.  And I proudly stated, the domain is blocked by China (ITProctology.com).  She simply looked at me quizzically and said she 'will think about it'

The day started like any other day.  I got up at the 'usual' time, cleaned up, scooped the cat litter, got the kid out of bed.

Kissed the lovely wife as she went off to work, got the kid to the bus stop, checked in the morning HAM radio net from the car.  Then drove all the way...  back home.

7:30am
Well, now what?  I have had a list of all kinds of things I have needed to get done.

Cold morning, checked the air in the tires, a bit low.  Grabbed the air compressor and resolved that.

Kind of chilly in the workshop, fired up the stove.
Looked at the wood scraps and thought of the cat tree the wife wants me to build.  Not feeling it.

7:45am
Watched an MVP PGI from yesterday that I missed. 
Got distracted by;
dirty dishes, cats fighting, HAM hobby antenna research, smelly garbage can, full recycling can, making list of items for another project from the hardware store, checking email (each time a notification popped up), checking LinkedIn, checking Twitter, checking Facebook.
I listened to the recording and viewed the visuals for the part I really wanted to see.

8:45am
Could really use a latte, warm, frothy, 15 minute drive - um, no.
Eventually feeling parched.  Looking for something cold and carbonated.  Hmm, no cold beverage fridge.
Tea it is I guess.

9:30am
Made notes of other follow up items I needed to do with HR.
Returned call to HR from yesterday (they called while I was in the middle of the only thing I had scheduled all day) - ring, ring, ring, disconnect.
I think, that was curious - I will try again later, she may have been on the phone with someone.

9:45 am
Phone rings - its Cabo.  (Mexico) they want to know when I can come visit.  Um, no...

10:00am
Start typing this

10:13am
All caught up
Checked email, read an article the wife sent, investigated LinkedIn Premium, looked around thinking "what next"

10:30am
Yea! spam to delete

And a little after that, my entire attempt at writing a humorous blog post was ruined.
(the following falls into a category that I won't name.  The comments are not disparagement to my former employer.)

I finally got hold of my HR representative, all was fine, my questions were answered.
Then I was asked a few questions (that I had already answered).
They wanted to know the accounts that I was using to access particular resources so they could be deactivated.  No problem.  I gave the account names.
Then I was asked for the passwords to those accounts.  Um.  No.  Just no.
My GitHub account,  They wanted the password to my GitHub account.  No, I never had a 'corporate' GitHub account.  They can just remove my account from the 'company' in GitHub.
And besides, I have access to other repositories that have nothing to do with my former employer. How can I trust the people that I am giving access to _my_ account?

Right there, any groove I had at writing humor was ruined.

So, I simply stepped away from everything for the remainder of the day.  And focused on other projects around the house.
Now, I still can't get this incident out of my head.  It is 5pm and I have to finish this post.

It is Friday.  Monday will bring a new adventure.  And a new post.
That one will actually be technical, and very useful to many folks.