Podcast Summary – Ubuntu Podcase S09E37 and S09E38

I was very fortunate to have been invited to guest present on the Ubuntu Podcast, for the episodes S09E37 and S09E38.

I recorded the show on my normal laptop, with a specially built Ubuntu Mate 16.10 image, which just included the show’s recording tools (Mumble, Voice Recorded, Gobby)

Due to some issues with my local recording, the audio used in the podcast was from the Mumble recording, rather than the local recording I made.

I’m generally happy with the recording, although I could have done with reading the content in more depth before the show, but, as it happened on a work night, I didn’t really make the time to research properly.

It was an absolute pleasure to record the show, and I’ve offered to guest present again in the future, if they’ll have me!

Using Expect to SFTP a file

Because of technical limitations on a pair of platforms I’m using at work, I am unable to set-up key-based SFTP or SCP to transfer files between the pair of them, so I knocked together this short script using the TCL based Expect language.


#!/usr/bin/expect
set arg1 [lindex $argv 0]
set arg2 [lindex $argv 1]
set arg3 [lindex $argv 2]
set timeout 1000
spawn sftp "$arg2"
expect {
yes {
send "yes\r"
exp_continue
}
ass {
send "$arg3\r"
exp_continue
}
sftp {
send "put $arg1\r"
expect {
100% {
send "quit\r"
exp_continue
}
}
}
}

view raw

upload.exp

hosted with ❤ by GitHub

There’s no error checking here, which isn’t great, but as a quick-and-dirty script to SFTP files to a box which needs the password each run… it works! :)

Talk Summary – How the internet works

Format: Freeform speech based on slides (AV issue). Theatre layout chairs. 50 attendees.

Slides: None

Video: None

Slot: Day 1 (Saturday), Slot 7 (15:00-15:30)

Notes: Covered ARP resolution of MAC addresses, routing (IPv4), DHCP, DNS and UDP. Missed TCP and HTTP but out of time. Mentioned IPv6.

Talk Summary – Building Labs – a basic guide

Format: Slides with code examples. Theatre layout chairs. 30 attendees.

Slides: Hacker Slides format (Markdown based) in a gist

Video: No URL as yet. It was recorded.

Slot: Day 1 (Saturday), Slot 5 (14:00-14:30)

Notes: Covered virtual machines, physical appliances, Vagrant, Ansible, Firewalling, routing, vlans. Mentioned ser2net but out of time to fully expand on slides at the end.

Talk Summary – Sandstorm: The Future Of Self-Hosting?

Format: Website tabs. An ad-hoc walkthrough of installation documentation and using Sandstorm. (Partial repeat of a previous talk at OggCamp ’15). Beanbags and a small number of chairs. 20 attendees.

Slides: None

Video: None

Slot: Day 1 (Saturday), Slot 1 (10:30-11:00)

Notes: Covered installing from curl | bash and mentioned PGP signing of installer. Covered installing apps and creating your own (might have confused people on this point). Demonstrated using apps with my own server, and installing apps too. Several follow-ups about why sandstorm and constraints in what to install it on, also about updates.

GPG Encrypting files using a keyserver

Another “at work” post!

I’ve been generating files which need to be distributed via a file server, but need to be encrypted using GPG (the open source PGP application). Rather than managing keys for a large number of users, instead, I have a text file with the user names in, and a batch file. Please see the below gist for details :)

Installing Symantec Endpoint Protection (SEP) on Ubuntu 14.04

At work we use Symantec Endpoint Protection, and in a lab, I was asked to confirm whether we could install it on our Ubuntu 14.04 servers. This took a few hops to get it installed, so I figured, I’d publish how I got it working, to save some other poor soul the trouble :)

Firstly, add the webupd8team’s Java PPA and update the repository cache: sudo add-apt-repository ppa:webupd8team/java && sudo apt-get update

This gives you the ability to install the Java 8 installer: sudo apt-get install oracle-java8-installer

This should download the install files, but for some reason, I was struggling to download it (the install script seems to struggle with downloading the actual .tar.gz file from Oracle), so I manually followed the link to http://download.oracle.com/otn-pub/java/jdk/8u77-b03/jdk-8u77-linux-x64.tar.gz, accepted the license, and placed the file in /var/cache/oracle-jdk8-installer/ and then re-ran the above apt-get install line.

— Note: This above issue was because I was running a caching proxy, which somehow doesn’t play nicely with this script. Turn off your proxy – should be all good :)

Next I had to install the Java Cryptography Extension which I got from the Java SE page. I placed this file in /tmp/jce_policy-8.zip (the filename is the one Oracle use) and replaced the files in /usr/lib/jvm/java-8-oracle/jre/lib/security with the ones from the extracted archive with this line: cp -b /tmp/UnlimitedJCEPolicyJDK8/*.jar /usr/lib/jvm/java-8-oracle/jre/lib/security.

The SEP client also has a dependency on the 32bit version of GLibc. I installed this with sudo apt-get install libc6-i386

I was then, finally, able to install the SEP client by unpacking the installer zip file, and running sudo bash install.sh -i from the path I’d unpacked the zip file in.

Not very complicated, I guess!

— Sources:

  • https://ubuntuincident.wordpress.com/2011/04/14/install-the-java-cryptography-extension-jce/
  • http://www.linuxquestions.org/questions/linux-newbie-8/how-to-install-32-bit-glibc-2-9-or-later-on-64-bit-ubuntu-12-04-a-4175413667/
  • http://www.webupd8.org/2012/09/install-oracle-java-8-in-ubuntu-via-ppa.html

Setting up a Google Play Music uploader for Linux Servers

THIS POST HAS BEEN SUPERCEEDED.

I like having an online music server. At home, I use a Logitech Media Server (formerly Squeezebox Server) [1] and run my several O2 Jogglers around the house with the “Squeezeplay OS” [2] images to play music from that server, but when it comes to an Android tablet (or Android phone), there’s not that joined-up thinking from Logitech (although you can just about cobble it together using a few 3rd party apps [3]), but what I do like is the Google Play Music service.

Google Play Music [4] was the first product from the Google Play team after they rebranded to “Google Play”, and I pretty quickly got interested in it. I installed the Google Play Uploader [5] on my home server, and uploaded all my music (apparently, I’m up to 12,000 tracks, but I think there are some duplicates there!) but what to do about the rest of the family? Well, until just recently, it didn’t matter. Jules has no interest in playing music on her phone or tablet, and Daniel, well, he’s 3-and-a-bit.

Since pretty early on, he’s been all over our tech – initially just using whatever apps we had installed on Jules’ and my phones, then Jules’ tablet (I was, and still am, pretty cautious about him using my tablet as it’s an Asus TF300T [6] which cost quite a lot of money, and I keep toying with the idea of installing some other OS – like Firefox OS [7] or Ubuntu Touch [8] on there and can’t if it’s wrecked), and now he’s pretty comfortable with browsing YouTube or the Play Store, although he knows not to click on adverts and can’t install anything that costs anything.

In the last couple of months, since he’s been really learning how to spell, he’s been asking how to spell the names of his favourite films (“Oliver!”, “Frozen” and “The Polar Express” primarily) to get the film clips up in YouTube, or to play snippets from the soundtracks of the films… which got me thinking. I can’t really do much about the films (not yet at least!), but perhaps I could set up Google Play Music with the soundtracks he listens to… but I already have one account sync’ed with Google Play Music from my media server…. how do I get his stuff set up on there?

Essentially, Google Play Music Uploader is a GUI application that can’t be started in the CLI (which kinda makes sense from a simplicity perspective), but as I’m already running one instance of the application on my media server, I can’t start up a second one, so what do I do?

Well, as it turns out, there’s actually a python library for interacting with Google Play Music’s upload and download applications called “gmusicapi” [9], and this couples with a really nice wrapper gives me a CLI utility I can run in a CRON job on my media server.

The wrapper is called gmusicapi-scripts [10], which contains 3 utilities – gmdownload.py, gmsync.py and the key to this – gmupload.py.

You need to install a few libraries. On my Ubuntu 14.04 system, I needed to run:

sudo apt-get install python-pip avconv-tools
sudo pip install gmusicapi
sudo pip install docopt

Once you’ve got this, you can get the tools themselves like this:

git clone https://github.com/thebigmunch/gmusicapi-scripts.git

This will give you a folder called gmusicapi-scripts in which is gmupload.py – the first time you run it, it’ll ask you to visit a web page in order to register the client. Click “OK” to approve the library having access to Google Play Music. This is a pretty spartan page, and ends up with a grey text box containing a string. Copy the contents of that box back into your terminal, and hey-presto, you get it working…

Well, sort of. For me, because I’d not set Daniel up with Google Play Music yet, I needed to set up his account first. I didn’t realise this (I thought, just going to the web page the script points you to will get you access, but it doesn’t because they need to vet which country you’re accessing from…) and the script didn’t tell me that (it just kept saying “Not Subscribed” [11])

Anyway, once it’s done you run

~/gmusicapi-scripts/gmupload.py /path/to/file.mp3

( 1/1) Successfully uploaded /path/to/file.mp3

If you’ve got multiple users, you can rename ~/.local/share/gmusicapi/oauth.cred to ~/.local/share/gmusicapi/SomeOtherName.cred and then run

~/gmusicapi-scripts/gmupload.py -c SomeOtherName /path/to/file.mp3

Subsequent passes will prompt you to authenticate the next account as you go, and then you can rename them as appropriate.

[1] Logitech Media Server (formerly Squeezebox Server): http://www.mysqueezebox.com/download
[2] “Squeezeplay OS”: http://birdslikewires.co.uk/articles/squeezeplay-os
[3] a few 3rd party apps: I got it working on my TF300T by combining https://play.google.com/store/apps/details?id=de.bluegaspode.squeezeplayer and https://play.google.com/store/apps/details?id=de.cedata.android.squeezecommander
[4] Google Play Music: https://play.google.com/music
[5] Google Play Uploader: https://play.google.com/music/listen?u=0#/manager
[6] TF300T: http://en.wikipedia.org/wiki/Asus_Transformer_Pad_TF300T
[7] Firefox OS: https://www.mozilla.org/en-US/firefox/os/
[8] Ubuntu Touch: http://www.ubuntu.com/tablet
[9] “gmusicapi”: https://github.com/simon-weber/Unofficial-Google-Music-API
[10] gmusicapi-scripts: https://github.com/thebigmunch/gmusicapi-scripts
[11] “Not Subscribed”: I raised a bug, but the lead developer said it seemed obvious to him that you have to set it up first… not disputing that for the first pass, but a nice message would have been good :) https://github.com/thebigmunch/gmusicapi-scripts/issues/22

Some notes about Ethernet over Power

I messed around a bit with my network tonight, in order to set set up my Ethernet-over-power (AKA Powerline Networking), and I figured out some things which, while they may not be useful to many of you, this is a bit of a prompt for the next time around.

1) The manager application runs under Windows only (although apparently, there are github repositories where you can get and build a linux application which even lets you set QoS aka Quality Of Service and other such fun things – I’ve not tried them, so I can’t recommend them). If you’ve got more than a matched pair of these, then you’ll need to run the application. I didn’t try running it in a virtual machine – I kept the supplied Windows OS from when I bought this machine specifically for purposes like this.

2) Not all models reset in the same way. If you can’t get them all to reset, connect to them with a CAT5 cable, go to the “Privacy” tab, select “Public network” which will reset it to “PowerLineAV”, and then select “Local computer”. You should then be able to browse across them all.

3) Not all models come with a “password” (sometimes referred to as a DEK). In this case, you also have to plug into these devices to set up their security. If they do have a password, it’ll be entirely in upper case, and even though the application shows numeric characters, in the 4 devices I received, they were all alphabetic-only strings of 16 characters, separated by hyphens.

4) Once you’ve got them all set to “PowerLineAV”, typed the passwords in for the models which have them, you can now set a community wide network password. This could be used to set up several logical segments, but realistically, it’s going to be one flat network :)

I can’t think, offhand, of anything else I need to say right now, but it’s been pretty interesting setting this up, so… hope you enjoyed it!

Starting EC2 instances using PHP

I run a small podcast website called CCHits.net. It runs on Dreamhost because they offer unlimited storage and bandwidth, but while it’s a great service for storage, it’s not really useful for running a batch process because long running processes are killed regularly (in my case, building the cchits podcasts on a daily basis).

As a result, I built an EC2 instance which I trigger every day using a cronjob. Previously, I used the “AWS CLI tools”, but as this uses a Java Virtual Machine, it was taking an awful lot of resources just to spin up an instance, and Dreamhost kept killing the task off. As a result, I found the AWS PHP SDK, and coded up this little snippet to spin up the EC2 instance.