Computers – Page 12 – A nice guy's view on life

Run an Ansible Playbook against a Check Point Gaia node running R80+

2019-06-052019-11-05 JonTheNiceGuy Leave a comment

Late Edit – 2019-11-05: Ansible 2.9 has some Check Point modules for interacting with the Check Point Manager API which are actually Idempotent, and if you’re running Ansible <=2.8, there are some non-idempotent modules available directly from Check Point. This post is about interacting with the OS. The OS might now be much more addressable using ansible_connection=ssh!

In Check Point Gaia R77, if you wanted to run Ansible against this node, you were completely out of luck. The version of Python on the host was broken, modules were missing and … well, it just wouldn’t work.

Today, I’m looking at running some Ansible playbooks against Check Point R80 nodes. Here’s some steps you need to get through to make it work.

Make sure the user that Ansible is going to be using has the shell /bin/bash. If you don’t have this set up, the command is: set user ansible shell /bin/bash.
If you want a separate user account to do ansible actions, run these commands:
add user ansible uid 9999 homedir /home/ansible
set user ansible password-hash $1$D3caF9$B4db4Ddecafbadnogoood (note this hash is not valid!)
add rba user ansible roles adminRole
set user ansible shell /bin/bash
Make sure your inventory specifies the right path for your Python binary. In the next code block you’ll see my inventory for three separate Check Point R80+ nodes. Note that I’ll only be targetting the “checkpoint” group, but that I’m using the r80_10, r80_20 and r80_30 groups to load the variables into there. I could, alternatively, add these in as values in group_vars/r80_10.yml and so on, but I find keeping everything to do with my connection in one place much cleaner. The python interpreter is in a separate path for each version time, and if you don’t specify ansible_ssh_transfer_method=piped you’ll get a message like this: [WARNING]: sftp transfer mechanism failed on [cpr80-30]. Use ANSIBLE_DEBUG=1 to see detailed information (fix from Add pipeline-ish method using dd for file transfer over SSH (#18642) on the Ansible git repo)

[checkpoint]
cpr80-10        ansible_user=admin      ansible_password=Sup3rS3cr3t-
cpr80-20        ansible_user=admin      ansible_password=Sup3rS3cr3t-
cpr80-30        ansible_user=admin      ansible_password=Sup3rS3cr3t-

[r80_10]
cpr80-10

[r80_20]
cpr80-20

[r80_30]
cpr80-30

[r80_10:vars]
ansible_ssh_transfer_method=piped
ansible_python_interpreter=/opt/CPsuite-R80/fw1/Python/bin/python

[r80_20:vars]
ansible_ssh_transfer_method=piped
ansible_python_interpreter=/opt/CPsuite-R80.20/fw1/Python/bin/python

[r80_30:vars]
ansible_ssh_transfer_method=piped
ansible_python_interpreter=/opt/CPsuite-R80.30/fw1/Python/bin/python

And there you have it, one quick “ping” check later…

$ ansible -m 'ping' -i hosts checkpoint
cpr80-10 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
cpr80-30 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
cpr80-20 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

One quick word of warning though, don’t use gather_facts: true or the setup: module. Both of these still rely on missing libraries on the Check Point nodes, and won’t work… But then again, you can get whatever you need from shell commands….. right? ;)

Featured image is “Untitled” by “Ryan Dickey” on Flickr and is released under a CC-BY license.

Building a Gitlab and Ansible Tower (AWX) Demo in Vagrant with Ansible

2019-05-162019-05-16 JonTheNiceGuy Leave a comment

TL;DR – I created a repository on GitHub ‌ containing a Vagrantfile and an Ansible Playbook to build a VM running Docker. That VM hosts AWX (Ansible Tower’s upstream open-source project) and Gitlab.

A couple of years ago, a colleague created (and I enhanced) a Vagrant and Ansible playbook called “Project X” which would run an AWX instance in a Virtual Machine. It’s a bit heavy, and did a lot of things to do with persistence that I really didn’t need, so I parked my changes and kept an eye on his playbook…

Fast-forward to a week-or-so ago. I needed to explain what a Git/Ansible Workflow would look like, and so I went back to look at ProjectX. Oh my, it looks very complex and consumed a lot of roles that, historically, I’ve not been that impressed with… I just needed the basics to run AWX. Oh, and I also needed a Gitlab environment.

I knew that Gitlab had a docker-based install, and so does AWX, so I trundled off to find some install guides. These are listed in the playbook I eventually created (hence not listing them here). Not all the choices I made were inspired by those guides – I wanted to make quite a bit of this stuff “build itself”… this meant I wanted users, groups and projects to be created in Gitlab, and users, projects, organisations, inventories and credentials to be created in AWX.

I knew that you can create Docker Containers in Ansible, so after I’d got my pre-requisites built (full upgrade, docker installed, pip libraries installed), I add the gitlab-ce :latest docker image, and expose some ports. Even now, I’m not getting the SSH port mapped that I was expecting, but … it’s no disaster.

I did notice that the Gitlab service takes ages to start once the container is marked as running, so I did some more digging, and found that the uri module can be used to poll a URL. It wasn’t documented well how you can make it keep polling until you get the response you want, so … I added a PR on the Ansible project’s github repo for that one (and I also wrote a blog post about that earlier too).

Once I had a working Gitlab service, I needed to customize it. There are a bunch of Gitlab modules in Ansible but since a few releases back of Gitlab, these don’t work any more, so I had to find a different way. That different way was to run an internal command called “gitlab-rails”. It’s not perfect (so it doesn’t create repos in your projects) but it’s pretty good at giving you just enough to build your demo environment. So that’s getting Gitlab up…

Now I need to build AWX. There’s lots of build guides for this, but actually I had most luck using the README in their repository (I know, who’d have thought it!??!) There are some “Secrets” that should be changed in production that I’m changing in my script, but on the whole, it’s pretty much a vanilla install.

Unlike the Gitlab modules, the Ansible Tower modules all work, so I use these to create the users, credentials and so-on. Like the gitlab-rails commands, however, the documentation for using the tower modules is pretty ropey, and I still don’t have things like “getting your users to have access to your organisation” working from the get-go, but for the bulk of the administration, it does “just work”.

Like all my playbooks, I use group_vars to define the stuff I don’t want to keep repeating. In this demo, I’ve set all the passwords to “Passw0rd”, and I’ve created 3 users in both AWX and Gitlab – csa, ops and release – indicative of the sorts of people this demo I ran was aimed at – Architects, Operations and Release Managers.

Maybe, one day, I’ll even be able to release the presentation that went with the demo ;)

On a more productive note, if you’re doing things with the tower_ modules and want to tell me what I need to fix up, or if you’re doing awesome things with the gitlab-rails tool, please visit the repo with this automation code in, and take a look at some of my “todo” items! Thanks!!

Featured image is “Tower” by “Yijun Chen” on Flickr and is released under a CC-BY-SA license.

"funfair action" by "Jon Bunting" on Flickr

Improving the speed of Azure deployments in Ansible with Async

2019-05-152019-05-15 JonTheNiceGuy Leave a comment

Recently I was building a few environments in Azure using Ansible, and found this stanza which helped me to speed things up.

  - name: "Schedule UDR Creation"
    azure_rm_routetable:
      resource_group: "{{ resource_group }}"
      name: "{{ item.key }}_udr"
    loop: "{{ routetables | dict2items }}"
    loop_control:
        label: "{{ item.key }}_udr"
    async: 1000
    poll: 0
    changed_when: False
    register: sleeper

  - name: "Check UDRs Created"
    async_status:
      jid: "{{ item.ansible_job_id }}"
    register: sleeper_status
    until: sleeper_status.finished
    retries: 500
    delay: 4
    loop: "{{ sleeper.results|flatten(levels=1) }}"
    when: item.ansible_job_id is defined
    loop_control:
      label: "{{ item._ansible_item_label }}"

What we do here is to start an action with an “async” time (to give the Schedule an opportunity to register itself) and a “poll” time of 0 (to prevent the Schedule from waiting to be finished). We then tell it that it’s “never changed” (changed_when: False) because otherwise it always shows as changed, and to register the scheduled item itself as a “sleeper”.

After all the async jobs get queued, we then check the status of all the scheduled items with the async_status module, passing it the registered job ID. This lets me spin up a lot more items in parallel, and then “just” confirm afterwards that they’ve been run properly.

It’s not perfect, and it can make for rather messy code. But, it does work, and it’s well worth giving it the once over, particularly if you’ve got some slow-to-run tasks in your playbook!

Featured image is “funfair action” by “Jon Bunting” on Flickr and is released under a CC-BY license.

A web browser with the example.com web page loaded

Working around the fact that Ansible’s URI module doesn’t honour the no_proxy variable…

2019-05-142019-05-14 JonTheNiceGuy Leave a comment

An Ansible project I’ve been working on has tripped me up this week. I’m working with some HTTP APIs and I need to check early whether I can reach the host. To do this, I used a simple Ansible Core Module which lets you call an HTTP URI.

- uri:
    follow_redirects: none
    validate_certs: False
    timeout: 5
    url: "http{% if ansible_https | default(True) %}s{% endif %}://{{ ansible_host }}/login"
  register: uri_data
  failed_when: False
  changed_when: False

This all seems pretty simple. One of the environments I’m working in uses the following values in their environment:

http_proxy="http://192.0.2.1:8080"
https_proxy="http://192.0.2.1:8080"
no_proxy="10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24"

And this breaks the uri module, because it tries to punt everything through the proxy if the “no_proxy” contains CIDR values (like 192.0.2.0/24) (there’s a bug raised for this)… So here’s my fix!

- set_fact:
    no_proxy_match: |
      {
        {% for no_proxy in (lookup('env', 'no_proxy') | replace(',', '') ).split() %}
          {% if no_proxy| ipaddr | type_debug != 'NoneType' %}
            {% if ansible_host | ipaddr(no_proxy) | type_debug != 'NoneType' %}
              "match": "True"
            {% endif %}
          {% endif %}
        {% endfor %}
      }

- uri:
    follow_redirects: none
    validate_certs: False
    timeout: 5
    url: "http{% if ansible_https | default(True) %}s{% endif %}://{{ ansible_host }}/login"
  register: uri_data
  failed_when: False
  changed_when: False
  environment: "{ {% if no_proxy_match.match | default(False) %}'no_proxy': '{{ ansible_host }}'{% endif %} }"

So, let’s break this down.

The key part to this script is that we need to override the no_proxy environment variable with the IP address that we’re trying to address (so that we’re not putting 16M addresses for 10.0.0.0/8 into no_proxy, for example). To do that, we use the exact same URI block, except for the environment line at the end.

In turn, the set_fact block steps through the no_proxy values, looking for IP Addresses to check ({% if no_proxy | ipaddr ... %}‌ says “if the no_proxy value is an IP Address, return it, but if it isn’t, return a ‘None’ value”) and if it’s an IP address or subnet mask, it checks to see whether the IP address of the host you’re trying to reach falls inside that IP Address or Subnet Mask ({% if ansible_host | ipaddr(no_proxy) ... %} says “if the ansible_host address falls inside the no_proxy range, then return it, otherwise return a ‘None’ value”). Both of these checks say “If this previous check returns anything other than a ‘None’ value, do the next thing”, and on the last check, the “next” thing is to set the flag ‘match’ to ‘true’. When we get to the environment variable, we say “if match is not true, it’s false, so don’t put a value in there”.

So that’s that! Yes, I could merge the set_fact block into the environment variable, but I do end up using that a fair amount. And really, if it was merged, that would be even MORE complicated to pick through.

I have raised a pull request on the Ansible project to update the documentation, so we’ll see whether we end up with people over here looking for ways around this issue. If so, let me know in the comments below! Thanks!!

Idempotent Dynamic Content in Ansible

2019-04-302019-05-19 JonTheNiceGuy Leave a comment

One of my colleagues recently sent out a link to a post about generating idempotent random numbers for Ansible. As I was reading it, I realised that there are other ways of doing the same thing (but not quite as pretty).

See, one of the things I (mis-)use Ansible for is to build Azure, AWS and OpenStack environments (instead of, perhaps, using Terraform, Cloud Formations or Heat Stacks). As a result, I frequently want to set complex passwords that are unique to *that environment* but that aren’t new for each build. My way of doing this is to run a delegated task to generate files in host_vars. Here’s a version of the playbook I use for that!

--- - hosts: all # Don't gather facts - for this test, we don't have access to any actual # hosts. This replicates an "infrastructure build" environment where you # might not have the hosts actually created at this point! gather_facts: false vars: # This next block looks up or generates the passwords for each host. # - ansible_password is defined as part of the inventory. # - machine_password is the generated password, stored in host_vars. # - PASSWORD is an environment variable, which can be specified on the # command line. # - project_password is a playbook-wide password, if all the machines # need to have a common password. # - Otherwise, generate a 12 character long password using A-Za-z0-9, # and suffix it with _ and the common suffix (default "password"). password: | {%- if ansible_password is defined -%} {{ ansible_password }} {%- elif machine_password is defined -%} {{ machine_password }} {%- elif lookup('env', 'PASSWORD') != '' -%} {{ lookup('env', 'PASSWORD') }} {%- else -%} {{ project_password | default( lookup ('password', '/dev/null length=12 chars=lowercase,uppercase,digits') + '_' + project_password_suffix | default('Password') ) }} {%- endif -%} tasks: # Initially we need to ensure that the host_vars directory is created. - name: Create host_vars directory delegate_to: localhost run_once: true file: path: "host_vars" state: directory mode: 0755 # And that each host has it's own directory (but only if we're generating # a new password for that host). - name: Create Directory for Stored Generated Values delegate_to: localhost file: path: "host_vars/{{ inventory_hostname }}" state: directory mode: 0755 when: - ansible_password is not defined - machine_password is not defined - project_password is not defined # And lastly, we store the generated password. Ideally, we would use a vault # at this point, but I don't know how to issue this statement from an ansible # playbook! - name: Store Generated Values delegate_to: localhost lineinfile: path: "host_vars/{{ inventory_hostname }}/generated.yml" regexp: '^machine_password:' line: 'machine_password: {{ password }}' state: present create: yes when: - ansible_password is not defined - machine_password is not defined - project_password is not defined # And just because this *is* a test, print it out :) - name: Print password delegate_to: localhost debug: var=password

In the same gist as that block has been sourced from I have some example output from “20 hosts” – one of which has a pre-defined password in the inventory, and the rest of which are generated by the script.

I hope this is useful to someone!

Late Edit – 2019-05-19: Encrypting the values you generate

Following this post, a friend of mine – Jeremy mentioned on Linked In that I should have a look at Ansible Vault. Well, *ideally*, yes, however, when I looked at this code, I couldn’t work out a way of forcing the session to run Vault against a value I’ve just created, short of running something a raw or a shell module like “ansible-vault encrypt {{ file_containing_password }}“. Realistically, if you’re doing a lot with these passwords, you should probably use an external password vault, such as HashiCorp’s Vault or PasswordStore.org’s Pass. Neither of which I tend to use, because it’s just not part of my life yet – but I’ve heard good things about both!

Featured image is “Matrix” by “Paul Downey” on Flickr and is released under a CC-BY license.

One to listen to: “Salary Negotiation for DevOps with Josh Doody”

2019-04-102019-04-10 JonTheNiceGuy Leave a comment

A few weeks ago, during a podcast binge, I came across this podcast (Salary Negotiations for DevOps with Josh Doody on the Real World DevOps Podcast). I noted at the time that it was really good content with great advice… and then forgot about it. (Oh, and it’s not just for DevOps people!)

Fast forward to today, when one of the Admin Admin Podcast Listeners (in our Telegram Channel) announces that he’s just gone for a new job, had been offered it, and was thinking of taking the job… but that they’d offered him a package lower than he was hoping to receive. My response “Say you wanted more, see if they can meet you halfway!” The main thing I took away from this podcast was that by the time you’re in the interview stage, the company you’re being interviewed by is *likely* to have already paid several thousand pounds/dollars/euros to have you sat in front of them, so if they want you, they’ll probably pay that bit more to not have to go through that process again!

Anyway, this is a great podcast for anyone who works for an employer, is thinking of asking for a pay rise‌ or is looking for a new job, and it’s well worth a listen!

Featured image is “Money” by “Images Money” on Flickr and is released under a CC-BY license.

"Wifi Here on a Blackboard" by "Jem Stone" on Flickr

Free Wi-Fi does not need to be password-less!

2019-03-282019-03-28 JonTheNiceGuy 2 Comments

Recently a friend of mine forwarded an email to me about a Wi-fi service he wanted to use from a firm, but he raised some technical questions with them which they seemed to completely misunderstand!

So, let’s talk about the misconceptions of Wi-fi passwords.

Many people assume that when you log into a system, it means that system is secure. For example, logging into a website makes sure that your data is secure and protected, right? Not necessarily – the password you entered could be on a web page that is not secured by TLS, or perhaps the web server doesn’t properly transfer it’s contents to a database. Maybe the website was badly written, and means it’s vulnerable to one of a handful of common attacks (with fun names like “Cross Site Scripting” or “SQL Injection Attacks”)…

People also assume the same thing about Wi-fi. You reached a log in page, so it must be secure, right? It depends. If you didn’t put in a password to access the Wi-fi in the first place (like in the image of the Windows 10 screen, or on my KDE Desktop) then you’re probably using Unsecured Wi-fi.

An example of a secured Wi-fi sign-in box on Windows 10

People like to compare network traffic to “sending things through the post”, notablycomparing E-Mail to “sending a postcard”, versus PGP encrypted E-Mail being compared to “sending a sealed letter”. Unencrypted Wi-fi is like using CB. Anyone who can hear your signal can understand what you are saying… but if you visit a website which uses HTTPS, then it’s like listening to someone saying random numbers over the radio.

And, if you’re using Unencrypted Wi-fi, it’s also possible for an attacker to see what website you visited, because the request for the address to reach on the Internet (e.g. “Google.com” = 172.217.23.14) is sent in the clear. Also because of the way that DNS works (that name to address matching thing) means that if someone knows you’re visiting a “site of interest” (like, perhaps a bank website), they can reply *before* the real DNS server, and tell you that the server on their machine is actually your bank’s website.

So, many of these things can be protected against by using a simple method, that many people who provide Wi-fi don’t do.

Turn on WPA2 (the authentication bit). Even if *everyone* uses the same password (which they’d have to for WPA2), the fact you’re logging into the Access Point means it creates a unique shared secret for your session.

“But hang on”, I hear the guy at the back cry, “you used the same password – how does that work?”

OK, so this is where the fun stuff starts. The password is just part of how you negotiate to get on to the network. There’s a complex beast of a method that explains how get a shared unique secret when you’re passing stuff around “in the clear”, and so as a result, when you first connect to that Wi-fi access point, and you hand over your password, it “Authorises” you on to the network, but then hands you over to the encryption part, where you generate a key and then use that to talk to each other. The encryption is the bit like “HTTPS”, where you make it so that people can’t see what you’re looking at.

“I got told that if everyone used the same password” said a hipster in the front row, “I wouldn’t be able to tell them apart.” Aha, not true. You can have a separate passphrase to access the Wi-fi from the Login page, after all, you’ve got to make sure that people aren’t breaking the rules (which they *TOTALLY* read, before clicking “I agree, just get me on the damn Wi-fi already”) by using your network.

“OK”, says the lady over on the right, “but when I connected to the Wi-fi, they asked me to log in using Facebook – that’s secure, right?”

Um, no. Well, maybe. See, if they gave you a WPA2 password to log into the Wi-fi, and then the first thing you got to was that login screen, then yep, it’s all good! {*} You can browse with (relative) impunity. But if they didn’t… well, not only are they asking you to shout your secrets on the radio, but if you’re really unlucky, the page asking you to log into Facebook might *also* not actually be Facebook, but another website that just looks like Facebook… after all, I’m sure that page you went to complained that it wasn’t Google or Facebook when you tried to open it…

{*} Except for the fact they’re asking you to tell them not only who you are, but who you’re also friends with, where you went to school, what your hobbies are, what groups you’re in, your date of birth and so on.

But anyway. I understand why those login screens are there. They’re asserting that not only do you understand that you mustn’t use their network for bad things, but that if the police come and ask them who used their network to do something naughty, they can say “He said his name was ‘Bob Smith’ and his email address was ‘bob@example.com’, Officer”…

It also means that the “free” service they provide to you, usually at some great expense (*eye roll*) can get them some return on investment (like, they just got your totally-real-and-not-at-all-made-up-email-address… honest, and they also know what websites you visited while you were there, which they can sell on).

So… What to do the next time you “need” Wi-fi, and there’s a free service there? Always use a VPN when you’re not using a network you trust. If the Wi-fi isn’t using WPA2 encryption (even something as simple as “Buy a drink first” is a great passphrase to use!) point them to this page, and tell them it’s virtually pain free (as long as the passphrase is easy to remember, easy to type and doesn’t have too many weird symbols in) and makes their service more safe and secure for their customers…

Featured image is “Wifi Here on a Blackboard” by “Jem Stone” on Flickr and is released under a CC-BY license.

"Centralized, Decentralized, Distributed" by "Amber Case" on Flickr

A brief summary of Terminology about non-centralised applications

2019-02-212019-02-21 JonTheNiceGuy Leave a comment

I hang out in the #redecentralize matrix group, and yesterday one of the group asked a question about getting clarification on the terminology. Here’s what I wrote:

Self Hosted: An application (usually running on a server) that you run in your own environment.
Examples include: Ethercalc, Sandstorm, WordPress.
[Note, Self Hosted services may still be classed as self-hosted, even if you don’t manage the environment yourself, for example, if you use a Virtual Machine, a Virtual Private Server, or pay someone like modular.im to build and run it for you – provided you can migrate your hosted application to your own environment if you want to]
P2P (Peer to Peer): A locally running application (or client) which predominantly talks to other clients (referred to as a peer), not to a server. There may be a central server which helps facilitate the initial connection between applications, but this is typically only used for that introduction. There may also be a semi-fixed list of “seed nodes” used to discover other nodes in the network.

Examples include: Bittorrent, Secure Scuttlebutt
[Many VoIP systems will have some sort of federated connection between “signalling” nodes, but have a P2P connection for the Audio/Visual streams.]
Federated: A server-based application that can talk to other server applications. (Federation can also refer to the method by which they find each other – either by responses to specific HTTP(s) requests or from particular DNS records).

Examples include: Matrix.org, Mastodon
Distributed: This is more how data is processed – if it’s centralised but distributed (e.g. Facebook, Netflix) then a central server instructs other servers how to act, and the nodes perform actions on behalf of the server. When talking about Decentralised, this means that you could have several nodes cooperating on an activity.

Examples include: BOINC, DNS
Blockchain: A distributed, secure, append-only database. May be P2P or Federated.
Examples include: Bitcoin
Decentralised: Any application which does not require a central service to function. Usually implies Self hosted.

Examples include: Collabora Online, “Internet Mail” (the original decentralised service!)
Enhanced from a message sent to the #redentralize:matrix.org chat, following advice from participants in the group

I hope you find this list of definitions useful!

(Edited 2019-02-21 to address comments from Ben in the Binary Times Telegram group, also others from mylo5ha5 in the Redecentralize group. Typo fixed, thanks to uhoreg)

Featured image is “Centralized, Decentralized, Distributed” by “Amber Case” on Flickr and is released under a CC-BY-NC license.

"Juniper NetScreen 25 Firewall front" by "jackthegag" on Flickr

Standard Firewall Rules

2019-02-012019-02-01 JonTheNiceGuy Leave a comment

One of the things I like to do is to explain how I set things up, but a firewall is one of those things that’s a bit complicated, because it depends on your situation, and what you’re trying to do in your environment. That said, there’s a template that you can probably get away with deploying, and see if it works for your content, and then you’ll see where to add the extra stuff from there. Firewall policies typically work from the top down.

This document will assume you have a simple boundary firewall. This simple firewall has two interfaces, the first being an “Outside” interface, connected to your ISP, with an IPv4 address of 192.0.2.2/24 and a default gateway of 192.0.2.1, it also has a IPv6 address of 2001:db8:123c:abd::2/64 and a default gateway address of 2001:db8:123c:abd::1. The second “Inside” interface, where your protected network is attached, has an IPv4 address of 198.51.100.1/24 and an IPv6 address of 2001:db8:123d:abc::1/64. On this inside interface, the firewall is the default gateway for the inside network.

I’ll be using simple text rules to describe firewall policies, following this format:

Source Interface: <outside | inside>
Source IP Address: <x.x.x.x/x | "any">
NAT Source IP Address: <x.x.x.x/x | no>
Destination Interface: <outside | inside>
Destination IP Address: <x.x.x.x/x | "any">
NAT Destination IP Address: <x.x.x.x/x | no>
Destination Port: <tcp | udp | icmp | ip>/<x>
Action: <allow | deny | reject>
Log: <yes | no>
Notes: <some commentary if required>

In this model, if you want to describe HTTP access to a web server, you might write the following policy:

Source Interface: outside
Source IP Address: 0.0.0.0/0 (Any IP)
NAT Source IP Address: no
Destination Interface: inside
Destination IP Address: 192.0.2.2 (External IP)
NAT Destination IP Address: 198.51.100.2 (Internal IP)
Destination Port: tcp/80
Action: allow
Log: yes

So, without further waffling, let’s build a policy. By default all traffic will be logged. In high-traffic environments, you may wish to prevent certain traffic from being logged, but on the whole, I think you shouldn’t really lose firewall logs unless you need to!

Allowing established, related and same-host traffic

This rule is only really needed on iptables based firewalls, as all the commercial vendors (as far as I can tell, at least) already cover this as “standard”. If you’re using UFW (a wrapper to iptables), this rule is covered off already, but essentially it goes a bit like this:

Source Interface: lo (short for "local", where the traffic never leaves the device)
Source IP Address: any
NAT Source IP Address: no
Destination Interface: lo
Destination IP Address: any
NAT Destination IP Address: no
Destination Port: any
Action: allow
Log: no
Notes: This above rule permits traffic between localhost addresses (127.0.0.0/8) or between public addresses on the same host, for example, between two processes without being blocked.

flags: Established OR Related
Action: allow
Log: no
Notes: This above rule is somewhat special, as it looks for specific flags on the packet, that says "If we've already got a session open, let it carry on talking".

Dropping Noisy Traffic

In a network, some proportion of the traffic is going to be “noisy”. Whether it’s broadcast traffic from your application that uses mDNS, or the Windows File Share trying to find like-minded hosts to exchange data… these can fill up your logs, so lets drop the broadcast and multicast IPv4 traffic, and not log them.

Source Interface: any
Source IP Address: 0.0.0.0/0
NAT Source IP Address: no
Destination Interface: any
Destination IP Address: 255.255.255.255 (global broadcast), 192.0.2.255 ("outside" broadcast), 198.51.100.255 ("inside" broadcast) and 224.0.0.0/4 (multicast)
NAT Destination IP Address: no
Destination Port: any
Action: deny
Log: no
Notes: The global and local broadcast addresses are used to "find" other hosts in a network, whether that's a DHCP server or something like mDNS. Dropping this prevents the traffic from appearing in your logs later.

Permitting Management Traffic

Typically you want to trust certain machines to access or be accessed by this host – whether it’s your SYSLOG collector, or the box that can manage the firewall policy, so here we’ll create a policy that lets these in.

Source Interface: inside
Source IP Address: 198.51.100.2 and 2001:db8:123d:abc::2 (Management IP)
NAT Source IP Address: no
Destination Interface: inside
Destination IP Address: 198.51.100.1 and 2001:db8:123d:abc::1 (Firewall IP)
NAT Destination IP Address: no
Destination Port: SSH (tcp/22)
Action: permit
Log: yes
Notes: Allow inbound SSH access. You're unlikely to need more inbound ports, but if you do - customise them here.

Source Interface: inside
Source IP Address: 198.51.100.1 and 2001:db8:123d:abc::1 (Firewall IP)
NAT Source IP Address: no
Destination Interface: inside
Destination IP Address: 198.51.100.2 and 2001:db8:123d:abc::2 (Management IP)
NAT Destination IP Address: no
Destination Port: SYSLOG (udp/514)
Action: permit
Log: yes
Notes: Allow outbound SYSLOG access. Tailor this to outbound ports you need.

Allowing Control Traffic

ICMP is a protocol that is fundamental to IPv4 and IPv6. Commonly used for Traceroute and Ping, but also used to perform REJECT responses and that sort of thing. We’re only going to let it be initiated *out* not in. Some people won’t allow this rule, or tailor it to more specific destinations.

Source Interface: inside
Source IP Address: any
NAT Source IP Address: 192.0.2.2 (The firewall IP address which may be replaced with 0.0.0.0 indicating "whatever IP address is bound to the outbound interface")
Destination Interface: outside
Destination IP Address: any
NAT Destination IP Address: no
Destination Port: icmp
Action: allow
Log: yes
Notes: ICMPv4 and ICMPv6 are different things. This is just the ICMPv4 version. IPv4 does require NAT, hence the difference from the IPv6 version below.

Source Interface: inside
Source IP Address: any
NAT Source IP Address: no
Destination Interface: outside
Destination IP Address: any
NAT Destination IP Address: no
Destination Port: icmpv6
Action: allow
Log: yes
Notes: ICMPv4 and ICMPv6 may be treated as different things. This is just the ICMPv6 version. IPv6 does not require NAT.

Protect the Firewall

There should be no other traffic going to the Firewall, so let’s drop everything. There are two types of “Deny” message – a “Reject” and a “Drop”. A Reject sends a message back from the host which is refusing the connection – usually the end server to say that the service didn’t want to reply to you, but if there’s a box in the middle – like a firewall – this reject (actually an ICMP packet) comes from the firewall instead. In this case it’s identifying that the firewall was refusing the connection for the node, so it advertises the fact the end server is protected by a security box. Instead, firewall administrators tend to use Drop, which just silently discards the initial request, leaving the initiating end to “Time Out”. You’re free to either “Reject” or “Drop” whenever we show “Deny” in the below policies, but bear it in mind that it’s less secure to use Reject than it is to Drop.

Source Interface: any
Source IP Address: any
NAT Source IP Address: no
Destination Interface: any
Destination IP Address: 192.0.2.2, 2001:db8:123c:abd::2, 198.51.100.1 and 2001:db8:123d:abc::1 (may also be represented as :: or 0.0.0.0 depending on the platform)
NAT Destination IP Address: no
Destination Port: any
Action: deny
Log: no
Notes: Drop everything targetted at the firewall IPs. If you have more NICs or additional IP addresses on the firewall, these will also need blocking.

“Normal” Inbound Traffic

After you’ve got your firewall protected, now you can sort out your “normal” traffic flows. I’m going to add a single inbound policy to represent the sort of traffic you might want to configure (in this case a simple web server), but bear in mind some environments don’t have any “inbound” rules (for example, most homes would be in this case), and some might need lots and lots of inbound rules. This is just to give you a flavour on what you might see here.

Source Interface: outside
Source IP Address: any
NAT Source IP Address: no
Destination Interface: inside
Destination IP Address: 192.0.2.2 (External IP)
NAT Destination IP Address: 198.51.100.2 (Internal IP)
Destination Port: tcp/80 (HTTP), tcp/443 (HTTPS)
Action: allow
Log: yes
Notes: This is the IPv4-only rule. Note a NAT MUST be applied here.

Source Interface: outside
Source IP Address: any
NAT Source IP Address: no
Destination Interface: inside
Destination IP Address: 2001:db8:123d:abc::2
NAT Destination IP Address: no
Destination Port: tcp/80 (HTTP), tcp/443 (HTTPS)
Action: allow
Log: yes
Notes: This is the IPv6-only rule. Note that NO NAT is required (but, you may wish to perform NAT, depending on your environment).

“Normal” Outbound Traffic

If you’re used to a DSL router, that basically just allows all outbound traffic. We’re going to implement that here. If you want to be more specific about things, you’d define your outbound rules like the inbound rules in the block above… but if you’re not that worried, then this rule below is generally going to be all OK :)

Source Interface: inside
Source IP Address: any
NAT Source IP Address: 192.0.2.2 (The firewall IP address which may be replaced with 0.0.0.0 indicating "whatever IP address is bound to the outbound interface")
Destination Interface: outside
Destination IP Address: any
NAT Destination IP Address: no
Destination Port: any
Action: allow
Log: yes
Notes: This is just the IPv4 version. IPv4 does require NAT, hence the difference from the IPv6 version below.

Source Interface: inside
Source IP Address: any
NAT Source IP Address: no
Destination Interface: outside
Destination IP Address: any
NAT Destination IP Address: no
Destination Port: any
Action: allow
Log: yes
Notes: This is just the IPv6 version. IPv6 does not require NAT.

Drop Rule

Following your permit rules above, you now need to drop everything else. Fortunately, by now, you’ve “white-listed” all the permitted traffic, so now we can just drop “everything”. So, let’s do that!

Source Interface: any
Source IP Address: any
NAT Source IP Address: no
Destination Interface: any
Destination IP Address: any
NAT Destination IP Address: no
Destination Port: any
Action: deny
Log: yes

And so that is a basic firewall policy… or at least, it’s the template I tend to stick to! :)

Expanding XFS drives with LVM

2019-01-232019-01-23 JonTheNiceGuy Leave a comment

Say, for example, you’ve got a lovely CentOS VM (using XFS by default) which has a disk that isn’t quite big enough. Fair enough, your VM Hypervisor is sensible enough to resize that disk without question… How do you resize the XFS partition? Assuming you’ve got your disk mounted as /dev/sda, and you’ve got a boot volume as partition 1 and a root volume as partition 2 (the standard install model)

parted /dev/sda resizepart 2 100%
partprobe /dev/sda
pvresize /dev/sda2
lvextend /dev/centos/root /dev/sda2
xfs_growfs /dev/mapper/centos-root

The graphical version of the steps above

Research via:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Allowing established, related and same-host traffic

Dropping Noisy Traffic

Permitting Management Traffic

Allowing Control Traffic

Protect the Firewall

“Normal” Inbound Traffic

“Normal” Outbound Traffic

Drop Rule

Share this:

Share this: