Tuesday, April 21, 2020

Installing NGINX OSS webserver with Ansible - Part 2

This is part three of me walking through a simple demo that I set up using Ansible.

This installment installs the web server machines using NGINX OSS.

Please refer back to the first two installments to get an understanding of the assumptions: local file system inventory files, Ansible host deployed to the same VPC as the remote machines, variable files, run the playbooks from the same folder as the inventory and variable files.

The playbook

In this example I will focus on the playbook: https://github.com/brianehlert/ansible-nginx-examples/blob/master/nginx_web_demo_oss.yaml

As the previous blog, this uses the nginx Ansible Role.

This time, I have included the 'extra_vars' - extra variables specific to this playbook, within the playbook itself instead of using an external file.

The reason is did this was to follow a different pattern and that this is a static playbook for me.  The only substitution that I am doing involves the individual webserver machines that this particular configuration is applied to.

The play uses the default.conf Jinja2 template for an http server from the Ansible Role.
I am placing the configuration in the default location, setting an error page location.
Instructing it to respond to traffic from any IP address, on port 80.
And lastly I am setting the path and file for the page to be demo_index.html

(you will also find demo_index.html in the example repository along side this file).

Running the playbook:
ansible_playbook nginx_web_demo_oss.yaml -i webservers

The result

Like my previous post, we aren't done yet.  So another error message.
This time the error message that you should receive is no longer 'an error occurred' but rather '403 forbidden'

Why '403 forbidden' you might ask?  Because we configured a file path for the HTML file.
The path we set is: /usr/share/nginx/html/demo_index.html
But there is not a file at that path.  As far as the web server knows, you can't access whatever you are requesting.  So the 403 because it is a configured path instead of a 404 which you would get against a path that does not exist.

Fixing the 403

I thought I would be nice and fix the 403 in this blog post, so here is the solution.

ansible-playbook update_demo_index.html.yaml -i webservers

This is another simple playbook; https://github.com/brianehlert/ansible-nginx-examples/blob/master/update_demo_index_html.yaml

It simply copies the file demo_index.html to the correct path on each webserver.

Now, if you refresh your browser that you are using to test the loadbalancer, you should get an NGINX Demo page.

Why so granular

One reason for being so granular with the playbooks is to separate the tasks that are being performed.
The benefit is that the playbooks can be reused in other playbooks, like functions.  Another benefit is that they align with how Roles should be created, as reusable tasks.

If you get into using Ansible Tower, you can start linking together playbooks into a workflow - branching off for success and failure conditions.  Even adding pauses for approval workflows.

Tuesday, April 14, 2020

Installing NGINX OSS loadbalancer with Ansible - part 1

In the previous post I provisioned a number of machines for a demo environment into AWS.

Background

Remember, that to make ssh access easy between the Ansible server and the target machines, my Ansible host runs in the same VPC in AWS as the demo environment machines.

if you inspect the inventory files, that explains why I write in the internal DNS names of the instances.

Starting in this post I will be using those machines and deploying out a very simple topology of a load balancer in front of two webservers.
For this entire setup, we will be using NGINX.

If you recall back to the previous blog, the playbook created machines in AWS and wrote some inventory files to the file system where the playbook was run from.

So now you should see 4 additional files without extensions for controller, dbserver, loadbalancers, and webservers.  We will be using the loadbalancers inventory file in this exercise.

In this blog I am focused on this playbook: https://github.com/brianehlert/ansible-nginx-examples/blob/master/nginx_lb_oss.yaml

The playbook uses the NGINX Role from Ansible Galaxy, you can find that here: https://galaxy.ansible.com/nginxinc/nginx

Installing NGINX OSS on the loadbalancer


To run the playbook execute the command:
ansible_playbook nginx_lb_oss.yml -i loadbalancers

This playbook uses both the loadbalancers and webservers file as it executes, let me explain.

The webservers file is read into memory.

Then at the loadbalancers machines the variable file nginx_lb_vars.yaml is read from the playbook folder and the variables 'webserver1' and 'webserver2' are replaced with the values from the webservers inventory file - so that any traffic being forwarded is to real machines.

The the role is invoked with all of the required 'extra_vars' in memory.

The Ansible Role for NGINX installs NGINX OSS (aka open source version) and adds a reverse proxy (aka load balancer) configuration based on my yaml variables file.  This points to the DNS names of the two web server instances.

Testing the configuration

Now, you probably ran that, and jumped straight to testing traffic by hitting the public endpoint of the loadbalancer machine.  And then you probably thought to yourself; 'what kind of hack is this guy, this doesn't work'

Well, you are right.  You should be receiving the response 'an error occurred'.  This is good, this is right and proper for where we are in the deployment.
If the URL can't be reached, then something else happened.  But if your error is: 'an error occurred' we are in a good place.

And might you guess why you have that error message?
Because neither of the two webservers are responding.  They don't yet have NGINX installed nor configured.
So everything is working as it should be.
It just isn't working, yet.


Tuesday, April 7, 2020

Using Ansible to deploy a demo environment to AWS

I promised a few months ago that I would be back at it, and well....  Things got in the way.  Lots of things.

For a while I am going to focus on one method that I use with Ansible to set up a demo environment in AWS.

The first playbook I am going to talk about can be found here: https://github.com/brianehlert/ansible-nginx-controller-examples/blob/master/controller_environment_deploy_aws.yaml

The set-up


Before we begin, let me describe the first assumption:
I place the variable file and the playbook in the same folder and I import the variables when I execute the playbook.

The second is how I establish the VPC, I do not use the default VPC

I use the Cloud Formation Template: https://github.com/brianehlert/ansible-nginx-controller-examples/blob/master/demo-vpc.json
to establish a Virtual Private Cloud (aka network), 2 public and 2 private subnets, and the proper routing and gateways for communication between the subnets and to the outside world from each.

When I provision my machines for the demo, I will be referencing the same region, the vpc from above, and the subnets.  We will get to that in a bit.

Third; I run my Ansible host in the VPC, attached to one of the public subnets.  I lock down its public interface with a security group and I ssh to it to perform the tasks at hand.

Fourth: I have Ansible all set up with taking to the AWS API.
This was the first blog that really gave me success with getting that set up; https://tomoconnor.eu/blogish/part-3-ansible-and-amazon-web-services/#.XozLZHJ7l3j


The workflow


Having established the VPC, we will need to copy the vpc-id and the subnet-id(s) and paste them into the variable files.
This is the variable file for the machine provisioning: https://github.com/brianehlert/ansible-nginx-controller-examples/blob/master/controller_aws_vars.yaml

Let me describe that a bit:
---
region: "the AWS region your VPC is in from above"
vpc_id: "the ID of the VPC - something like vpc-0no9th755is5fake1"
vpc_cidr_ip: "the CIDR of the VPC, such as:" 10.240.0.0/16
loadbalancer_public_subnet: "the ID of one of the public subnets where loadbalancer machines will be placed - such as: subnet-0b03bad1228you93r"
webserver_private_subnet: "the ID of one of the private subnets for the webservers"
image_id: "the AMI image ID for the region - I use the Ubuntu 18.04 LTS minimal image myself"
key_name: "your ssh access key for AWS to embed in the image"
controller_public_subnet: "the other public subnet for the controller machine"
database_private_subnet: "the other private pubnet for the postgreSQL db server"
prepend_name: "a string to prepend to the machine names, such as 'mine'"

Now that the variables are all set up, and the vpc is in place, lets build some machines.

You can run the playbook with the following:
ansible-playbook controller_environment_deploy_aws.yaml -e "@controller_aws_vars.yaml"  

If I read that command line back it would speak like this:  Run the Ansible playbook controller_environment_deploy_aws.yaml including the variable from the file controller_aws_vars.yaml.
The '-e' means to include 'extra variables' to the execution of the playbook.  The '@' at the beginning of the variable means to reference a file on the Ansible host.

What all does that build for me

What you will now end up with is multi-fold.  Part of what I will describe is to make dealing with inventory a little easier.

If you read through the playbook you can probably understand that some security groups are created for each machine type.
The AMI that is referenced is used as the base image for all of the machines.  But the machines are created as different sizes.
The network attachments happen against the subnets defined in the vars file.

One key thing that happens is that individual inventory files are written out in the same folder where the playbook is being run from.
You will have an inventory file for each machine type.  These are how Ansible will reference each machine type later on.
We will be referencing '-i loadbalancers' when installing nginx, and '-i webservers' when installing nginx and the web site files. 
The '-i' refers to a specific inventory file.
I did it this way because it gives me the flexibility to quickly stand something up with inventory (and reset) without getting into all of details of Ansible inventory and the many ways to handle it.

You want to reset; delete the machines in AWS and delete the corresponding inventory files and start over.

Tuesday, November 12, 2019

my take on infrastructure as code

Lately I have been working through a number of automation tasks to create environments and deal with various customer scenarios.

I will say right now, I am all about 'cattle, not sacred cows' as in my configurations are always separate from the machine.  All that configuration state can always come from some place else.

15 years ago, I was doing this without automation.  We rebuilt our servers at every upgrade of the primary application running on them.  And I also updated all the firmware, etc.  So I have been in this school of thought for a long time.  We just didn't use source control back then, it was documents and settings files and a vault.

The primary concept behind 'infrastructure as code' is that you can run some set of automation.  Then bundle up all of the artifacts that drove that automation as a documented source of truth.

Some folks think of the 'infrastructure as code' part as just the settings files.  Just the code, and a set of variables.  But I challenge that it is much larger than that.

For example: with an Ansible playbook.  You would have the source playbook, you have the environment variables passed in.   And don't stop there.
You might also have some Jinja2 templates that the playbook used as a transform, you might have had temporary variables in flight, maybe files needed to be purged to harden the machine in production, etc.

All of that is part of the infrastructure as code.  Not just what is fed in, but also the scripts and automation that drive the result.  All of it.
I should be able to take an archive, open it, play it, and get the same result.
Which means that entire archive is your source of truth at that moment in time.

It is the moment in time part that has some type of source control all involved.
But in reality, your truth might not be in GitLab, it might be an archive in Artifactory.  As it might include binaries and other things that don't source control well.

So think about your pipelines and the artifacts make them up and move through them and the end results.
Think about the view of; 'can you replay that and get the same result'  or 'can you replay that offline'

I know that way back when we started to look at out rebuild process and combined that with a regular disaster recovery exercise, we really started to refine things and get a handle on the entire process and the dependencies across processes.
Details that are really easy to overlook in the daily grind of making it all just work.


Monday, November 11, 2019

Sorry for the huge silence

Sorry for the huge silence all.
As some of you may recall, the entire Redmond location that I was at with Citrix was let go.  RIF'd we call it.
After that I landed at F5 in Seattle.

The work at F5 was a pretty wild and constantly fast ride.
With the acquisition of NGINX by F5, I became part of the NGINX business.

That covers over 18 months in about as lightly as I can.

What you will find from me going forward is automation.  Probably interesting to DevOps and SRE types more so than what I used to write about.
And probably a lot more Linux than Windows.

I talk to customers a lot more than I used to. 
What I find interesting, is that the problems in IT that I was dealing with 20 years ago are still present in the industry today.
Yes, the tools have changed, the scope has changed, and the impacts have changed - but many of the problems still remain. 
It is just something that I find really interesting.

Part of me finds it disturbing as well.  Specifically that the core problems remain, but shift and change ever so slightly, but they are still present.
Is it that tools have come and gone?
The problems are solved, then the tools get re-written and they just surface again?
Is it that IT changes and keeps bringing everything back around with each generation?

Or, it is just that folks shift from one infrastructure to another and all that baggage just comes along for the ride?  To the new, not fully complete new platform.....
I speculate the latter more than not.

Anyway.  Back at it.  Hopefully posting things that are useful to the community, and hoping to gather some insights as well.