Tuesday, April 7, 2020

Using Ansible to deploy a demo environment to AWS

I promised a few months ago that I would be back at it, and well....  Things got in the way.  Lots of things.

For a while I am going to focus on one method that I use with Ansible to set up a demo environment in AWS.

The first playbook I am going to talk about can be found here: https://github.com/brianehlert/ansible-nginx-controller-examples/blob/master/controller_environment_deploy_aws.yaml

The set-up


Before we begin, let me describe the first assumption:
I place the variable file and the playbook in the same folder and I import the variables when I execute the playbook.

The second is how I establish the VPC, I do not use the default VPC

I use the Cloud Formation Template: https://github.com/brianehlert/ansible-nginx-controller-examples/blob/master/demo-vpc.json
to establish a Virtual Private Cloud (aka network), 2 public and 2 private subnets, and the proper routing and gateways for communication between the subnets and to the outside world from each.

When I provision my machines for the demo, I will be referencing the same region, the vpc from above, and the subnets.  We will get to that in a bit.

Third; I run my Ansible host in the VPC, attached to one of the public subnets.  I lock down its public interface with a security group and I ssh to it to perform the tasks at hand.

Fourth: I have Ansible all set up with taking to the AWS API.
This was the first blog that really gave me success with getting that set up; https://tomoconnor.eu/blogish/part-3-ansible-and-amazon-web-services/#.XozLZHJ7l3j


The workflow


Having established the VPC, we will need to copy the vpc-id and the subnet-id(s) and paste them into the variable files.
This is the variable file for the machine provisioning: https://github.com/brianehlert/ansible-nginx-controller-examples/blob/master/controller_aws_vars.yaml

Let me describe that a bit:
---
region: "the AWS region your VPC is in from above"
vpc_id: "the ID of the VPC - something like vpc-0no9th755is5fake1"
vpc_cidr_ip: "the CIDR of the VPC, such as:" 10.240.0.0/16
loadbalancer_public_subnet: "the ID of one of the public subnets where loadbalancer machines will be placed - such as: subnet-0b03bad1228you93r"
webserver_private_subnet: "the ID of one of the private subnets for the webservers"
image_id: "the AMI image ID for the region - I use the Ubuntu 18.04 LTS minimal image myself"
key_name: "your ssh access key for AWS to embed in the image"
controller_public_subnet: "the other public subnet for the controller machine"
database_private_subnet: "the other private pubnet for the postgreSQL db server"
prepend_name: "a string to prepend to the machine names, such as 'mine'"

Now that the variables are all set up, and the vpc is in place, lets build some machines.

You can run the playbook with the following:
ansible-playbook controller_environment_deploy_aws.yaml -e "@controller_aws_vars.yaml"  

If I read that command line back it would speak like this:  Run the Ansible playbook controller_environment_deploy_aws.yaml including the variable from the file controller_aws_vars.yaml.
The '-e' means to include 'extra variables' to the execution of the playbook.  The '@' at the beginning of the variable means to reference a file on the Ansible host.

What all does that build for me

What you will now end up with is multi-fold.  Part of what I will describe is to make dealing with inventory a little easier.

If you read through the playbook you can probably understand that some security groups are created for each machine type.
The AMI that is referenced is used as the base image for all of the machines.  But the machines are created as different sizes.
The network attachments happen against the subnets defined in the vars file.

One key thing that happens is that individual inventory files are written out in the same folder where the playbook is being run from.
You will have an inventory file for each machine type.  These are how Ansible will reference each machine type later on.
We will be referencing '-i loadbalancers' when installing nginx, and '-i webservers' when installing nginx and the web site files. 
The '-i' refers to a specific inventory file.
I did it this way because it gives me the flexibility to quickly stand something up with inventory (and reset) without getting into all of details of Ansible inventory and the many ways to handle it.

You want to reset; delete the machines in AWS and delete the corresponding inventory files and start over.

Tuesday, November 12, 2019

my take on infrastructure as code

Lately I have been working through a number of automation tasks to create environments and deal with various customer scenarios.

I will say right now, I am all about 'cattle, not sacred cows' as in my configurations are always separate from the machine.  All that configuration state can always come from some place else.

15 years ago, I was doing this without automation.  We rebuilt our servers at every upgrade of the primary application running on them.  And I also updated all the firmware, etc.  So I have been in this school of thought for a long time.  We just didn't use source control back then, it was documents and settings files and a vault.

The primary concept behind 'infrastructure as code' is that you can run some set of automation.  Then bundle up all of the artifacts that drove that automation as a documented source of truth.

Some folks think of the 'infrastructure as code' part as just the settings files.  Just the code, and a set of variables.  But I challenge that it is much larger than that.

For example: with an Ansible playbook.  You would have the source playbook, you have the environment variables passed in.   And don't stop there.
You might also have some Jinja2 templates that the playbook used as a transform, you might have had temporary variables in flight, maybe files needed to be purged to harden the machine in production, etc.

All of that is part of the infrastructure as code.  Not just what is fed in, but also the scripts and automation that drive the result.  All of it.
I should be able to take an archive, open it, play it, and get the same result.
Which means that entire archive is your source of truth at that moment in time.

It is the moment in time part that has some type of source control all involved.
But in reality, your truth might not be in GitLab, it might be an archive in Artifactory.  As it might include binaries and other things that don't source control well.

So think about your pipelines and the artifacts make them up and move through them and the end results.
Think about the view of; 'can you replay that and get the same result'  or 'can you replay that offline'

I know that way back when we started to look at out rebuild process and combined that with a regular disaster recovery exercise, we really started to refine things and get a handle on the entire process and the dependencies across processes.
Details that are really easy to overlook in the daily grind of making it all just work.


Monday, November 11, 2019

Sorry for the huge silence

Sorry for the huge silence all.
As some of you may recall, the entire Redmond location that I was at with Citrix was let go.  RIF'd we call it.
After that I landed at F5 in Seattle.

The work at F5 was a pretty wild and constantly fast ride.
With the acquisition of NGINX by F5, I became part of the NGINX business.

That covers over 18 months in about as lightly as I can.

What you will find from me going forward is automation.  Probably interesting to DevOps and SRE types more so than what I used to write about.
And probably a lot more Linux than Windows.

I talk to customers a lot more than I used to. 
What I find interesting, is that the problems in IT that I was dealing with 20 years ago are still present in the industry today.
Yes, the tools have changed, the scope has changed, and the impacts have changed - but many of the problems still remain. 
It is just something that I find really interesting.

Part of me finds it disturbing as well.  Specifically that the core problems remain, but shift and change ever so slightly, but they are still present.
Is it that tools have come and gone?
The problems are solved, then the tools get re-written and they just surface again?
Is it that IT changes and keeps bringing everything back around with each generation?

Or, it is just that folks shift from one infrastructure to another and all that baggage just comes along for the ride?  To the new, not fully complete new platform.....
I speculate the latter more than not.

Anyway.  Back at it.  Hopefully posting things that are useful to the community, and hoping to gather some insights as well.

Monday, May 7, 2018

AWS exposes route tables so I can recover from their bug

The cloud.  It is a wondrous thing.  When it works.
But I think that one thing that most all can agree on, it should 'just work'.  All aspects of your cloud experience should 'just work'.

In the past weeks I have shifted my attention to AWS. 
It has taken some time to get used to referring to a virtual network under the marketing term Virtual Private Cloud. Or a virtual machine as an 'EC2 instance' or any number of other marketing term focused things.
My preference here, call it what it is, not the marketing name for the feature.
Enough of that.

Back to the title.  One of the first things that I thought interesting with AWS is that within a VPC, the route table is exposed to me.
Why would I want to / need to muck around with a route table in the cloud?  I have port level firewalls rules (Security Groups), I have to stand up an Internet Gateway to enable outgoing traffic.  Why would I ever need to muck with something as low level as a route table?

Well, I can tell you from being burned on this multiple times now, it is so I can fix what the AWS portal screws up for me.

Back to my original statement - this is the cloud, it should 'just work'.  Networking is a pretty fundamental thing here.  It needs to be solid, resilient, and always functional.
But yet, at least three times now, in less than three months I have lost days due to multiple route table entries in a VPC that I created through the portal.

And usually the kicker is that one of the route table entries is correct and there is a second one that is empty.  A route table entry, but no defined route.

The really strange things happen when things work say manually or with one VM, and then you start automating and suddenly what you automate does not work.
As if one deployment uses the correct route table and the other doesn't.

It is one of those obscure things, that as a customer, I expect to work - all the time.  I should not have to think about this, I should not have to remember (in the recovery from frustration) that I had been burned by this in the past and go looking, only to discover that I have this strange named but empty route table entry.

My call to AWS, fix it.  Don't let it be mucked up in the first place.  Give me success here.  Don't frustrate me, don't waste my time.

But then, you did get paid for extra days of running compute while i tried to figure it all out.  So I guess that you might not have my interest at heart.

Thursday, January 4, 2018

Finding likes between two arrays

In a recent interview I was asked to write a function to sole the following problem:

You have a SQL query and two arrays, find the like elements between the two arrays.

(I am assuming that a few of you have your own opinions on how to solve this, and that is excellent - please enlighten me)

The first thought that popped into my head was that this was not a coding interview, the second was that i really needed a laptop to work through this, and the third was - doesn't PowerShell have a cmdlet that will do this for me?

Back at home, I thought I would investigate the PowerShell angle.  Since I had access to a laptop.
I am assuming a small dataset

First, lets build a couple arrays to validate with:

$array1 = "Elmer", "hunter", "Bugs", "rabbit", "Tweety", "bird", "Sylvester", "feline"
$array2 = "Eddie", "investigator", "Roger", "rabbit", "Jessica", "rabbit", "Judge", "doom", "tweety", "bird"

Now, I did add some variation for fun.  Because why might this be an important skill?
It might be important because I might need to mine a bunch of data or logs to investigate a pattern.

At this point I have arbitrarily added an additional requirement on myself; I want to see all of the matched data items, not just the matched values.  Equivalents vs. equals if you will.
Why? Again, in an investigation I will probably want to use a fuzzy match instead of a literal match.  And then I will want to work through the resulting set again.  (most likely both with eyes and with code).

In regards to PowerShell doing this for me, I discovered this:

$compared = Compare-Object -ReferenceObject $array1 -DifferenceObject $array2 -IncludeEqual

But when looking at the output, working with this is not very intuitive and it hides my fuzzy match and detail output.

PS C:\Users\Brian> $compared

InputObject  SideIndicator
-----------  -------------
rabbit       ==
Tweety       ==
bird         ==
Eddie        =>
investigator =>
Roger        =>
Jessica      =>
rabbit       =>
Judge        =>
doom         =>
Elmer        <=
hunter       <=
Bugs         <=
Sylvester    <=
feline       <=

I actually ended up going back to my original idea; 

$likes = @()

foreach ($element1 in $array1){
    foreach ($element2 in $array2){
        if ($element1 -like $element2){
            $likes += $element1;
            $likes += $element2;
        }
    }
}

while not highly efficient and I am sure not wonderful with large data sets, it does give me the results that I wanted for further analysis:

PS C:\Users\Brian> $likes
rabbit
rabbit
rabbit
rabbit
Tweety
tweety
bird
bird

It is easy to see from this output where I might want to go next; counts, deeper analysis, trends, etc.  It all depends on the details in the records used.

If you have different ideas that meet my requirements, please share.