Tag: Remote Administration

One to read/watch: IPsec and IKE Tutorial

JonTheNiceGuy Leave a comment

Ever been told that IPsec is hard? Maybe you’ve seen it yourself? Well, Paul Wouters and Sowmini Varadhan recently co-delivered a talk at the NetDev conference, and it’s really good.

Sowmini’s and Paul’s slides are available here: https://www.files.netdevconf.org/d/a18e61e734714da59571/

A complete recording of the tutorial is here. Sowmini’s part of the tutorial (which starts first in the video) is quite technically complex, looking at specifically the way that Linux handles the packets through the kernel. I’ve focused more on Paul’s part of the tutorial (starting at 26m23s)… but my interest was piqued from 40m40s when he starts to actually show how “easy” configuration is. There are two quick run throughs of typical host-to-host IPsec and subnet-to-subnet IPsec tunnels.

A key message for me, which previously hadn’t been at all clear in IPsec using {free,libre,open}swan is that they refer to Left and Right as being one party and the other… but the node itself works out if it’s “left” or “right” so the *SAME CONFIG* can be used on both machines. GENIUS.

Also, when you’re looking at the config files, anything prefixed with an @ symbol is something that doesn’t need resolving to something else.

It’s well worth a check-out, and it’s inspired me to take another look at IPsec for my personal VPNs :)

I should note that towards the end, Paul tried to run a selection of demonstrations in Opportunistic Encryption (which basically is a way to enable encryption between two nodes, even if you don’t have a pre-established VPN with them). Because of issues with the conference wifi, plus the fact that what he’s demoing isn’t exactly production-grade yet, it doesn’t really work right, and much of the rest of the video (from around 1h10m) is him trying to show that working while attendees are running through the lab, and having conversations about those labs with the attendees.

Running Streisand to provide VPN services on my home server

JonTheNiceGuy Leave a comment

A few months ago I was a guest on The Ubuntu Podcast, where I mentioned that I use Streisand to terminate my VPN connections. I waffled and blathered a bit about how I set it up, but in the end it comes down to this:

  1. Install Virtualbox on my Ubuntu server. Include the “Ext Pack”.
  2. Install Vagrant on my Ubuntu server.
  3. Clone the Streisand Github repository to my Ubuntu server.
  4. Enter that cloned repository, and edit the Vagrantfile as follows:
    1. Add the line “config.vm.boot_timeout = 65535” after the one starting “config.vm.box”.
    2. Change the streisand.vm.hostname line to be an appropriate hostname for my network, and add on the following line (replace “eth0” with the attached interface on your network and “192.0.2.1” with an unallocated static IP address from your network): 
      streisand.vm.network "public_network", bridge: "eth0", ip: "192.0.2.1", :use_dhcp_assigned_default_route => false
    3. Add a “routing” line, as follows (replace 192.0.2.254 with your router IP address): 
      streisand.vm.provision "shell", run: "always", inline: "ip route add 0.0.0.0/1 via 192.0.2.254 ; ip route add 128.0.0.0/1 via 192.0.2.254"
    4. Comment out the line “streisand_client_test => true”
    5. Amend the line “streisand_ipv4_address” to reflect the IP address you’ve put above in 4.2.
    6. Remove the block starting “config.vm.define streisand-client do |client|”
  5. Run “vagrant up” in that directory to start the virtual machine. Once it’s finished starting, there will be a folder called “Generated Docs” – open the .html file to see what credentials you must use to access the server. Follow it’s instructions.
  6. Once it’s completed, you should open ports on your router to the IP address you’ve specified. Typically, at least, UDP/500 and UDP/4500 for the IPsec service, UDP/636 for the OpenVPN service and TCP/4443 for the OpenConnect service.

One to read: “SKIP grep, use AWK” / ”Awk Tutorial, part {1,2,3,4}”

JonTheNiceGuy 1 Comment

Do you use this pattern in your sh/bash/zsh/etc-sh scripts?

cat somefile | grep 'some string' | awk '{print $2}'

If so, you can replace that as follows:

cat somefile | awk '/some string/ {print $2}'

Or how about this?

grep -v 'something' < somefile | awk '{print $0}'

Try this:

awk '! /something/ {print $0}' < somefile

Ooo OK, how about if you want to get all actions performed by users when the ISO formatted dates (Y-m-d) match the first day of the month, but where you don’t want to also match January (unless you’re talking about the first of January)…

# echo 'BLOGGSF 2001-01-23 SOME_ACTION' | awk '$2 ~ /-01$/ {print $1, $3}'
(EMPTY LINE)
# echo 'BLOGGSF 2002-02-01 SOME_ACTION' | awk '$2 ~ /-01$/ {print $1, $3}'
BLOGGSF SOME_ACTION

This is so cool! Thanks to the tutorials “SKIP grep, use AWK” and the follow-up tutorials starting here…

Today I learned… Cloud-init doesn’t like you repeating the same things

JonTheNiceGuy Leave a comment

Because of templates I was building in my post “Today I learned… Ansible Include Templates”, I thought you could repeat the same sections over again. Here’s a snippet of something like what I’d built (after combining lots of templates together):

Note this is a non-working code sample!


#cloud-config
packages:
- iperf
- git

write_files:
- content: {% include 'files/public_key.j2' %}
  path: /root/.ssh/authorized_keys
  owner: root:root
  permission: '0600'
- content: {% include 'files/private_key.j2' %}
  path: /root/.ssh/id_rsa
  owner: root:root
  permission: '0600'

packages:
- byobu

write_files:
- content: |
    #!/bin/bash
    git clone {{ test_scripts }} /root/iperf_scripts
    bash /root/iperf_scripts/run_test.sh
  path: /root/run_test
  owner: root:root
  permission: '0700'

runcmd:
- /root/run_test

I’d get *bits* of it to run – basically, the last file, the last package and the last runcmd… but not all of it.

Turns out, cloud-init doesn’t like having to rebuild all the fragments together. Instead, you need to put them all together, so the write_files items, and the packages items all live in the same area.

Which, when you think about what it’s doing, which is that the parent lines are defining a variable called… well, whatever that line is, and if you replace it, it’s only going to keep the last one, then it all makes sense really!

Today I learned… that you can look at the “cloud-init” files on your target server…

JonTheNiceGuy Leave a comment

Today I have been debugging why my Cloud-init scripts weren’t triggering on my Openstack environment.

I realised that something was wrong when I tried to use the noVNC console[1] with a password I’d set… no luck. So, next I ran a command to review the console logs[2], and saw a message (now, sadly, long gone – so I can’t even include it here!) suggesting there was an issue parsing my YAML file. Uh oh!

I’m using Ansible’s os_server module, and using templates to complete the userdata field, which in turn gets populated as cloud-init scripts…. and so clearly I had two ways to debug this – prefix my ansible playbook with a few debug commands, but then that can get messy… OR SSH into the box, and look through the logs. I knew I could SSH in, so the cloud-init had partially fired, but it just wasn’t parsing what I’d submitted. I had a quick look around, and found a post which mentioned debugging cloud-init. This mentioned that there’s a path (/var/lib/cloud/instances/$UUID/) you can mess around in, to remove some files to “fool” cloud-init into thinking it’s not been run… but, I reasoned, why not just see what’s there.

And in there, was the motherlode – user-data.txt…. bingo.

In the jinja2 template I was using to populate the userdata, I’d referenced another file, again using a template. It looks like that template needs an extra line at the end, otherwise, it all runs together.

Whew!

This does concern me a little, as I had previously been using this stanza to “simply” change the default user password to something a little less complicated:


#cloud-config
ssh_pwauth: True
chpasswd:
  list: |
    ubuntu:{{ default_password }}
  expire: False

But now that I look at the documentation, I realise you can also specify that as a pre-hashed value (in which case, you would suffix that default_password item above with |password_hash('sha512')) which makes it all better again!

[1] If you run openstack --os-cloud cloud_a console url show servername gives you a URL to visit that has an HTML5 based VNC-ish client. Note the “cloud_a” and “servername” should be replaced by your clouds.yml reference and the server name or server ID you want to connect to.
[2] Like before, openstack --os-cloud cloud_a console log show servername gives you the output of the boot sequence (e.g. dmesg plus the normal startup commands, and finally, cloud-init). It can be useful. Equally, it’s logs… which means there’s a lot to wade through!

Using Expect to SFTP a file

JonTheNiceGuy Leave a comment

Because of technical limitations on a pair of platforms I’m using at work, I am unable to set-up key-based SFTP or SCP to transfer files between the pair of them, so I knocked together this short script using the TCL based Expect language.


#!/usr/bin/expect
set arg1 [lindex $argv 0]
set arg2 [lindex $argv 1]
set arg3 [lindex $argv 2]
set timeout 1000
spawn sftp "$arg2"
expect {
yes {
send "yes\r"
exp_continue
}
ass {
send "$arg3\r"
exp_continue
}
sftp {
send "put $arg1\r"
expect {
100% {
send "quit\r"
exp_continue
}
}
}
}

view raw

upload.exp

hosted with ❤ by GitHub

There’s no error checking here, which isn’t great, but as a quick-and-dirty script to SFTP files to a box which needs the password each run… it works! :)

Installing Symantec Endpoint Protection (SEP) on Ubuntu 14.04

JonTheNiceGuy Leave a comment

At work we use Symantec Endpoint Protection, and in a lab, I was asked to confirm whether we could install it on our Ubuntu 14.04 servers. This took a few hops to get it installed, so I figured, I’d publish how I got it working, to save some other poor soul the trouble :)

Firstly, add the webupd8team’s Java PPA and update the repository cache: sudo add-apt-repository ppa:webupd8team/java && sudo apt-get update

This gives you the ability to install the Java 8 installer: sudo apt-get install oracle-java8-installer

This should download the install files, but for some reason, I was struggling to download it (the install script seems to struggle with downloading the actual .tar.gz file from Oracle), so I manually followed the link to http://download.oracle.com/otn-pub/java/jdk/8u77-b03/jdk-8u77-linux-x64.tar.gz, accepted the license, and placed the file in /var/cache/oracle-jdk8-installer/ and then re-ran the above apt-get install line.

— Note: This above issue was because I was running a caching proxy, which somehow doesn’t play nicely with this script. Turn off your proxy – should be all good :)

Next I had to install the Java Cryptography Extension which I got from the Java SE page. I placed this file in /tmp/jce_policy-8.zip (the filename is the one Oracle use) and replaced the files in /usr/lib/jvm/java-8-oracle/jre/lib/security with the ones from the extracted archive with this line: cp -b /tmp/UnlimitedJCEPolicyJDK8/*.jar /usr/lib/jvm/java-8-oracle/jre/lib/security.

The SEP client also has a dependency on the 32bit version of GLibc. I installed this with sudo apt-get install libc6-i386

I was then, finally, able to install the SEP client by unpacking the installer zip file, and running sudo bash install.sh -i from the path I’d unpacked the zip file in.

Not very complicated, I guess!

— Sources:

  • https://ubuntuincident.wordpress.com/2011/04/14/install-the-java-cryptography-extension-jce/
  • http://www.linuxquestions.org/questions/linux-newbie-8/how-to-install-32-bit-glibc-2-9-or-later-on-64-bit-ubuntu-12-04-a-4175413667/
  • http://www.webupd8.org/2012/09/install-oracle-java-8-in-ubuntu-via-ppa.html

Starting EC2 instances using PHP

JonTheNiceGuy Leave a comment

I run a small podcast website called CCHits.net. It runs on Dreamhost because they offer unlimited storage and bandwidth, but while it’s a great service for storage, it’s not really useful for running a batch process because long running processes are killed regularly (in my case, building the cchits podcasts on a daily basis).

As a result, I built an EC2 instance which I trigger every day using a cronjob. Previously, I used the “AWS CLI tools”, but as this uses a Java Virtual Machine, it was taking an awful lot of resources just to spin up an instance, and Dreamhost kept killing the task off. As a result, I found the AWS PHP SDK, and coded up this little snippet to spin up the EC2 instance.

VNC in Ubuntu 11.04 with Unity

JonTheNiceGuy Leave a comment

I recently bought myself a new laptop. Sometimes though, I want to check something on it on a rare occasion when I’ve not taken it with me. In comes VNC. Under Ubuntu 11.04, turning on VNC support is pretty straight forward.

To turn on VNC, go to the power icon in the top right corner (I think they call it the “Session Menu”, but it looks like a power button to me) and select “System Settings”. Under the “Internet and Network” heading, is an option called “Remote Desktop”. Click on that. Tick the top two boxes “Allow other users to view your desktop” and “Allow other users to control your desktop”. Tick the box “Require the user to enter this password” (and enter a password) and “Configure network automatically to accept connections”. Untick “You must confirm each access to this machine” and select “Only display an icon when there is someone connected”. Close it.

Now, try connecting to your device, and see what happens. I had some issues with Compiz elements not rendering correctly, and found a few hints to fix it. The first says to turn on the “disable_xdamage” option. It says to use gconf-editor, but I’m SSHing in, so I need to use gconftool-2 as follows:

gconftool-2 --set "/desktop/gnome/remote_access/disable_xdamage" --type boolean "true"

Personally, I only want to ever connect over OpenVPN to this, so I added the following:

gconftool-2 --set "/desktop/gnome/remote_access/network_interface" --type string "tun0"

You may wish to only ever access it over SSH, in which case replace “tun0” with “lo”

Now, I next made a big mistake. I followed some duff guidance, and ended up killing my vino server (I’m still not sure if I was supposed to do this or not), but to get it back, I followed this instruction to restart it. I had to tweak it a little:

sudo x11vnc -rfbport 5901 -auth guess

Once you’ve started this, tunnel an extra port (5901) to your machine, start VNC to the tunnelled port, and then go back through the options above. Exit your VNC session to the new tunnelled port, and then hit Control+C on the SSH session to close that x11vnc service.