One to read: “SKIP grep, use AWK” / ”Awk Tutorial, part {1,2,3,4}”

Do you use this pattern in your sh/bash/zsh/etc-sh scripts?

cat somefile | grep 'some string' | awk '{print $2}'

If so, you can replace that as follows:

cat somefile | awk '/some string/ {print $2}'

Or how about this?

grep -v 'something' < somefile | awk '{print $0}'

Try this:

awk '! /something/ {print $0}' < somefile

Ooo OK, how about if you want to get all actions performed by users when the ISO formatted dates (Y-m-d) match the first day of the month, but where you don’t want to also match January (unless you’re talking about the first of January)…

# echo 'BLOGGSF 2001-01-23 SOME_ACTION' | awk '$2 ~ /-01$/ {print $1, $3}'
(EMPTY LINE)
# echo 'BLOGGSF 2002-02-01 SOME_ACTION' | awk '$2 ~ /-01$/ {print $1, $3}'
BLOGGSF SOME_ACTION

This is so cool! Thanks to the tutorials “SKIP grep, use AWK” and the follow-up tutorials starting here…

One to read: “Test Driven Development (TDD) for networks, using Ansible”

Thanks to my colleague Simon (@sipart on Twitter), I spotted this post (and it’s companion Github Repository) which explains how to do test-driven development in Ansible.

Essentially, you create two roles – test (the author referred to it as “validate”) and one to actually do the thing you want it to do (in the author’s case “add_vlan”).

In the testing role, you’d have the following layout:

/path/to/roles/testing/tasks/main.yml
/path/to/roles/testing/tasks/SOMEFEATUREtest.yml

In the main.yml file, you have a simple stanza:

---
- name: Include all the test files
  include: "{{ outer_item }}"
  with_fileglob:"/path/to/roles/validate/tasks/*test.yml"
  loop_control: loop_var=outer_item

I’m sure that “with_fileglob” line could be improved to not actually need a full path… anyway

Then in your YourFeature_test.yml file, you do things like this:

---
- name: "Pseudocode in here. Use real modules for your testing!!"
  get_vlan_config: filter_for=needle_vlan
  register:haystack_var

- assert: that=" {{ needle_item }} in haystack_var "

When you run the play of the role the first time, the response will be “failed” (because “needle_vlan” doesn’t exist). Next do the “real” play of the role (so, in the author’s case, add_vlan) which creates the vlan. Then re-run the test role, your response should now be “ok”.

I’d probably script this so that it goes:

      reset-environment set_testing=true (maybe create a random little network)
      test
      run-action
      test
      reset-environment set_testing=false

The benefit to doing it that way is that you “know” your tests aren’t running if the environment doesn’t have the “set_testing” thing in place, you get to run all your tests in a “clean room”, and then you clear it back down again afterwards, leaving it clear for the next pass of your automated testing suite.

Fun!

One to read or watch: “Programming is Forgetting: Toward a New Hacker Ethic”

Here is a transcript of a talk by Allison Parrish at the Open Hardware Summit in Portland, OR. The talk “Programming is Forgetting: Toward a New Hacker Ethic” is a discussion about the failings of the book “Hackers” by Steven Levy. Essentially, that book proposed (in the 80’s) a set of ethics for Hackers (which is to say, creative programmers or engineers, not malicious operators). Allison suggests that many of the parables in the book do not truly reflect the “Hacker Ethic”, and revises them for today’s world.

Her new questions (not statements) are as follows:

  • Who gets to use what I make? Who am I leaving out? How does what I make facilitate or hinder access?
  • What data am I using? Whose labor produced it and what biases and assumptions are built into it? Why choose this particular phenomenon for digitization or transcription? And what do the data leave out?
  • What systems of authority am I enacting through what I make? What systems of support do I rely on? How does what I make support other people?
  • What kind of community am I assuming? What community do I invite through what I make? How are my own personal values reflected in what I make?

This is a significant re-work of the original “Hacker Ethic“, and you should really either watch or read the talk to see how she got to these from the original, especially as it’s not as punchy as the original.

I’d like to think I was thinking of things like these questions when I wrote CampFireManager and CCHits.

One to read: “You need to rethink that Jump Server”

One of my colleagues, Nick Cross, posts links to articles he thinks are worth reading. This is one of the ones I though was worth re-posting.
Essentially, the key suggestions I took from it is that the jump server should be application white-listed to the bare bones, and should be auto-rebuilt on a daily basis.
It suggests some other controls, but these are the two key ones which I think could be “easiest” to implement.