Automation in an Infrastructure as Code world

With IaaS on Azure or AWS

Jon Spriggs, FDE – jon.sprig.gs

Reading along at home? Press “S” to also see my notes!

Image is “Engine” by “Paul Swansen” and is released under a CC-BY-ND license

Introduction

Who (is the speaker)?
Why (talk about this)?
What (is this about)?

Tom Hanks Hello via GIPHY

* Who am I – Former Wintel Engineer, Former Firewall Engineer, Former SOC analyst, Current cloud automation consultant, specialising in Security Products.
* Why is this interesting to me – I’m lazy. I don’t really like doing things more than once by hand.
* Infrastructure As Code is an Automated set of instructions to build a network of components, like Virtual Networks, Subnets, Virtual Machines and other supporting assets. While the Infrastructure parts of this can be built by hand, we’ll talk about why we might automate them later.
* That said, it’s not an exhaustive view of the world! I don’t go into Puppet, Chef, Salt, bash scripts, anything to do with Windows (so no DSC). It’s also really IAAS focused.
* The missing “Where” and “When” in the 5W’s is Here and Now! :) (For those reading along after, it’s actually October 2019, at the FDE Conference in Berlin)

Why Automate?

harry potter shrug via GIPHY

Topics

Terminology
Cloud Formation
ARM templates
Ansible
Terraform
Combinations

read texas am GIF by Texas A&M University GIF from Giphy

read texas am by Texas A&M University via GIPHY

Terminology?

"Library" by "Stephanie Overton" on Flickr

Image is “Library” by “Stephanie Overton” and is released under a CC-BY-ND license

Terminology

Infrastructure as Code

Image is “Scaffold” by “Alan Myers” and is released under a CC-BY license

In “The Cloud” many of your familiar concepts are available to you.

Machines (sometimes called VMs or Instances) run your “compute” “workloads”. These machines are part of a “Virtual Network” (VNet) or “Virtual Private Cloud” (VPC) that is divided into Subnets. The subnets all route to one-another, and may, if there’s a route there, get to the Internet, or to your on-site environments, using a dedicated connection (ExpressRoute or Direct Connect) or via an IPsec VPN.

We also have Firewalls, Load Balancers and WAF devices that mostly translate to the physical on-prem world.

The one, key difference, about compute in the cloud, is that there are Security Groups applied at the Interfaces that connect the VMs to the Subnets. These security groups are small firewall policies which only permit traffic on specific ports or from specific sources.

Terminology

Infrastructure as Code

“code crunching” by “Ruben Molina” on Flickr

Image is “code crunching” by “Ruben Molina” and is released under a CC-BY-ND license

* Parameters – Something passed from *outside* to *inside*, e.g. defining the prefix for the resources created by your script on the command line, specifying the region to deploy something in
* Variables – Any value that is *NOT* a fixed part of the script that is run (e.g. CIDR, Host prefixes, usernames and passwords)
* Templates. A template is a specially formatted “normal” file or string, with a way to inject variables and values into it, for example, the string “Hello {{ yourname }}” might be transformed into “Hello Lovely Audience”.
* File Formats
* DSL (Domain Specific Language) uses native terminology from that application vendor to describe what the environment looks like.
* YAML (YAML Ain’t Markup Language, previously called “Yet Another Markup Language”) is a readable generic text file format. There is a bit of skill in understanding how these files make anything which can be read by the computer, but on the whole it’s more human readable than the other file formats.
* JSON (Javascript Object Notation) is another text format used to pass data structures around. While YAML uses spaces, hyphens and newlines to indicate array and dictionary formats, JSON uses curled and square brackets. It’s still readable, but might be harder.

Terminology

Infrastructure as Code

"IMG_1361 copy" by "adogcalledstray" on Flickr

Image is “IMG_1361 copy” by “adogcalledstray” and is released under a CC-BY license

A moment to talk about methodology

Stepped

"Checklist" by "Xavier Vergés" on Flickr
Image is “Checklist” by “Xavier Vergés” and is released under a CC-BY license

Computed

"Blueprints" by "Cameron Degelia" on Flickr
Image is “Blueprints” by “Cameron Degelia” and is released under a CC-BY license

About My Code Sample Links

Fear Not! A QR link to the samples will be towards the end of the slide deck!

The Desired State at the end of each of the code samples

Let’s Talk Tooling

Ready Lets Go GIF by Leroy Patterson GIF from Giphy

Ready Lets Go via GIPHY

Cloud Formation

Documentation
Reference Guide
My Code Samples

A screen grab of the AWS Documentation taken 2019-09-25

### Context
* **About:** AWS native & Computed
* **Terminology:** Stacks (Parameters, Resources, Mappings)
* **File Type:** JSON or YAML formatted.
* **Run:** AWS cli tool, API or Web console (search for “Cloud Formation”)

### Pro’s
* CF is native to AWS. It tends to track the capabilities of the platform quite closely.
* It’s fast and there’s a web UI for dragging and dropping assets.
* There’s [extensive documentation ](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-template-resource-type-ref.html) which defines all your options

### Con’s
* It’s a monolithic file.
* Removing a CF Stack, by default, removes all the assets associated with it.
* Like all things AWS, it’s zone specific too.
* The WebUI doesn’t prompt you for what it *can* do with each of the assets either. Renaming objects breaks links between objects.

Azure Resource Manager (ARM) Templates

Documentation
My Code Samples

A screen grab of the Azure Documentation taken 2019-09-25

### Context
* **About:** Azure native & Computed
* **Terminology:** Template file and Parameters file – Deployment (Resource, Parameters, Variables and Outputs)
* **File Type:** JSON formatted.
* **Run:** Azure cli tool, API or Web console (you have to go looking for it! – Search for “Deploy a Custom Template”)

### Pro’s
* ARM is native to Azure. It generally tends to track the capabilities of the platform quite closely.
* It’s (relatively) fast
* It’s easy to point-and-click to create your ARM templates (go into the Resource Group, go into “Automation” and download the template. Customize).

### Con’s
* It’s a monolithic file.
* It can’t deliver a Resource Group. One must be created in advance!
* You can’t delete a Deployment in one go (the result of running the Template) – you have to remove the RG (which might have more content than was in your deployment).

Ansible

Ansible Logo (reversed colours)
Documentation
My Code Samples:
AWS | Azure

### Context
* **About:** Multi-Cloud (including AWS, Azure, GCP, VMWare) as well as local operations & Stepped
* **Terminology:** Playbooks contain Variables, Tasks and Handlers, and can also include values from other files. Values may contain Jinja2 templates. Playbook can also call Roles (basically collections of variables, tasks and handlers) – typically used for applying a role to a machine, not to deliver a cloud config.
* **File Type:** YAML formatted with Jinja2 templates.
* **Run:** Linux (or WSL) command line – NOT WINDOWS. Or, Ansible Tower (or AWX) in a web browser

### Pro’s
* Cross-platform, and will deliver post-config as well as infrastructure
* Can combine with all the other tools being discussed (but we’ll talk about that later!)
* Multi-file, with dynamic includes and templating from resources in the context of the cloud and external resources (e.g. “Hashicorp Vault”)

### Con’s
* Much slower than the other platforms, can’t build cloud things in parallel. Can put in work-arounds to solve that, but they’re messy and don’t always work right.
* Creates individual assets, can’t remove “everything” in one go.
* With AWS, doesn’t always perform Idempotent actions (e.g. creates two hosts with the same name, etc.)

Terraform

Terraform Logo
Documentation
My Code Samples:
AWS | Azure

### Context
* **About:** Multi-Cloud (list of providers: [core](https://www.terraform.io/docs/providers/index.html) and [community](https://www.terraform.io/docs/providers/type/community-index.html)) & Computed
* **Terminology:** Configuration files contain Resources or Data Sources which are fulfilled by a Provider (e.g. AWS, Azure). Parameters can be passed from CLI or a separate file. All are available as Variables. These variables can be exposed as an Output at the end. A collection of configuration files is known as a “module”. Once the Configuration File is completed, the automation engine is called to Plan what will happen next, then Apply that plan to the providers mentioned. Once you’ve finished working with that environment, you can Destroy it.
* **File Type:** HCL (HashiCorp Configuration Language), with some common naming structures for files (variables.tf, main.tf, output.tf)
* **Run:** Linux, Mac, Windows CLI or via Terraform Cloud Web Service (untested)

### Pro’s
* Cross-platform and multi-cloud
* It’s often faster at adding automation pieces to the terraform providers than the cloud vendors are themselves at adding features to CF/ARM.
* Multi-File, with some templating and some file includes
* You can destroy a whole stack in one go… provided…. the state file is valid.

### Con’s
* Terraform only really likes working with resources *it* created. Anything it doesn’t create, you need to tell it to go and find out about first (which adds it to the state file). Lose the state file, and you need to import everything all over again.

Combinations?

Combinations!

X triggers Y
My code samples:

AWS

Azure

Ansible → TF → AWS API

Ansible → TF → Azure API

Ansible from user/custom data
Examples:

AWS CF	AWS TF	AWS Ansible
Azure ARM	Azure TF	Azure Ansible

"Zwei Grand Gattai" by "finalfusion" on Flickr

Image is “Zwei Grand Gattai” by “finalfusion” and is released under a CC-BY license

Beyond that, Terraform and Ansible can both trigger the execution of an ARM template or a Cloud Formation stack, while Terraform and Ansible both have the capability to call each other, although I’d be tempted to say that trying to execute an Ansible job from Terraform locally probably won’t work very well!

Taking this further, if you find that Terraform is faster to deploy a CF stack, but you find Ansible is better at creating that CF file, then make the file with Ansible, then use Ansible to call Terraform, which uses the CF file Ansible created!

The reverse is also true. You can include in your user data (a blob of code that’s executed on initial boot) an Ansible play book, and an instruction to download Ansible and run it locally. If you include that in an Ansible, Terraform, CF or ARM instruction to create a virtual machine, then you can build your environment too. Alternatively, you can call out from your VM to an Ansible Tower instance, to ask it to provision your machine. These are all options you have.

Take Away

Azure Resource Manager ❤️ Ansible ❤️ Terraform ❤️ Cloud Formation	There’s no right or wrong answer. Each tool has it’s place; it’s strengths and weaknesses.
Jon Spriggs, @JonTheNiceGuy \| https://jon.sprig.gs

Your Next Steps?

"baby walking" by "Philippe Put" on Flickr
Image is “baby walking” by “Philippe Put” and is released under a CC-BY license

Code Samples

All available from: ⇒
https://github.com/JonTheNiceGuy
/cautious-octo-palm-tree

nobodys perfect nbc GIF by The Good Place from Giphy

nobodys perfect nbc GIF by The Good Place via GIPHY

QR Link to https://github.com/JonTheNiceGuy/cautious-octo-palm-tree

Jon Spriggs, FDE

Twitter: @JonTheNiceGuy
Linked In: LinkedIn.com/in/JonTheNiceGuy
Blog: jon.sprig.gs
Github: JonTheNiceGuy

Any questions?

Code Samples

All available from: ⇒
https://github.com/JonTheNiceGuy
/cautious-octo-palm-tree

nobodys perfect nbc GIF by The Good Place via GIPHY

Automation in an Infrastructure as Code world

With IaaS on Azure or AWS

Jon Spriggs, FDE – jon.sprig.gs

Reading along at home? Press “S” to also see my notes!

Introduction

Why Automate?

Topics

Terminology?

Terminology

Infrastructure as Code

Terminology

Infrastructure as Code

Terminology

Infrastructure as Code

A moment to talk about methodology

Stepped

Computed

About My Code Sample Links

Let’s Talk Tooling

Cloud Formation

Azure Resource Manager (ARM) Templates

Ansible

Terraform

Combinations?

Combinations!

Take Away

Azure Resource Manager ❤️ Ansible ❤️ Terraform ❤️ Cloud Formation

Your Next Steps?

Code Samples

All available from: ⇒ https://github.com/JonTheNiceGuy/cautious-octo-palm-tree

Jon Spriggs, FDE

Any questions?

Code Samples

All available from: ⇒ https://github.com/JonTheNiceGuy/cautious-octo-palm-tree

Azure Resource Manager
❤️
Ansible
❤️
Terraform
❤️
Cloud Formation

All available from: ⇒
https://github.com/JonTheNiceGuy
/cautious-octo-palm-tree

All available from: ⇒
https://github.com/JonTheNiceGuy
/cautious-octo-palm-tree