Some notes about Ethernet over Power

I messed around a bit with my network tonight, in order to set set up my Ethernet-over-power (AKA Powerline Networking), and I figured out some things which, while they may not be useful to many of you, this is a bit of a prompt for the next time around.

1) The manager application runs under Windows only (although apparently, there are github repositories where you can get and build a linux application which even lets you set QoS aka Quality Of Service and other such fun things – I’ve not tried them, so I can’t recommend them). If you’ve got more than a matched pair of these, then you’ll need to run the application. I didn’t try running it in a virtual machine – I kept the supplied Windows OS from when I bought this machine specifically for purposes like this.

2) Not all models reset in the same way. If you can’t get them all to reset, connect to them with a CAT5 cable, go to the “Privacy” tab, select “Public network” which will reset it to “PowerLineAV”, and then select “Local computer”. You should then be able to browse across them all.

3) Not all models come with a “password” (sometimes referred to as a DEK). In this case, you also have to plug into these devices to set up their security. If they do have a password, it’ll be entirely in upper case, and even though the application shows numeric characters, in the 4 devices I received, they were all alphabetic-only strings of 16 characters, separated by hyphens.

4) Once you’ve got them all set to “PowerLineAV”, typed the passwords in for the models which have them, you can now set a community wide network password. This could be used to set up several logical segments, but realistically, it’s going to be one flat network :)

I can’t think, offhand, of anything else I need to say right now, but it’s been pretty interesting setting this up, so… hope you enjoyed it!

Starting EC2 instances using PHP

I run a small podcast website called CCHits.net. It runs on Dreamhost because they offer unlimited storage and bandwidth, but while it’s a great service for storage, it’s not really useful for running a batch process because long running processes are killed regularly (in my case, building the cchits podcasts on a daily basis).

As a result, I built an EC2 instance which I trigger every day using a cronjob. Previously, I used the “AWS CLI tools”, but as this uses a Java Virtual Machine, it was taking an awful lot of resources just to spin up an instance, and Dreamhost kept killing the task off. As a result, I found the AWS PHP SDK, and coded up this little snippet to spin up the EC2 instance.

Development Environment Replication with Vagrant and Puppet

This week, I was fortunate enough to meet up with the Cheadle Geeks group. I got talking to a couple of people about Vagrant and Puppet, and explaining how it works, and I thought the best thing to do would be to also write that down here, so that I can point anyone who missed any of what I was saying to it.

Essentially, Vagrant is program to read a config file which defines how to initialize a pre-built virtual machine. It has several virtual machine engines which it can invoke (see [1] for more details on that), but the default virtual machine to use is VirtualBox.

To actually find a virtual box to load, there’s a big list over at vagrantbox.es which have most standard cloud servers available to you. Personally I use the Ubuntu Precise 32bit image from VagrantUp.com for my open source projects (which means more developers can get involved). Once you’ve picked an image, use the following command to get it installed on your development machine (you only need to do this step once per box!):

vagrant box add {YourBoxName} {BoxURL}

After you’ve done that, you need to set up the Vagrant configuration file.

cd /path/to/your/dev/environment
mkdir Vagrant
cd Vagrant
vagrant init {YourBoxName}

This will create a file called Vagrantfile in /path/to/your/dev/environment/Vagrant. It looks overwhelming at first, but if you trim out some of the notes (and tweak one or two of the lines), you’ll end up with a file which looks a bit like this:

Vagrant.configure("2") do |config|
  config.vm.box = "{YourBoxName}"
  config.vm.hostname = "{fqdn.of.your.host}"
  config.vm.box_url = "{BoxURL}"
  config.vm.network :forwarded_port, guest: 80, host: 8080
  # config.vm.network :public_network
  config.vm.synced_folder "../web", "/var/www"
  config.vm.provision :puppet do |puppet|
    puppet.manifests_path = "manifests"
    puppet.manifest_file  = "site.pp"
  end
end

This assumes you’ve replaced anything with {}’s in it with a real value, and that you want to forward TCP/8080 on your machine to TCP/80 on that box (there are other work arounds, using more Vagrant plugins, different network types, or other services such as pagekite, but this will do for now).

Once you’ve got this file, you could start up your machine and get a bare box, but that’s not much use to you, as you’d have to tell people how to configure your development environment every time they started up a new box. Instead, we’ll be using a Provisioning service, and we’re going to use Puppet for that.

Puppet was originally designed as a way of defining configuration across all an estate’s servers, and a lot of tutorials I’ve found online explain how to use it for that, but when we’re setting up Puppet for a development environment, we just need a simple file. This is the site.pp manifest, and in here we define the extra files and packages we need, plus any commands we need to run. So, let’s start with a basic manifest file:

node default {

}

Wow, isn’t that easy? :) We need some more detail than that though. First, let’s make sure the timezone is set. I live in the UK, so my timezone is “Europe/London”. Let’s put that in. We also need to make sure that any commands we run have the right path in them. So here’s our revised, debian based, manifest file.

node default {
    Exec {
        path => '/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/sbin:/usr/sbin'
    }

    package { "tzdata":
        ensure => "installed"
    }

    file { "/etc/timezone":
        content => "Europe/London\n",
        require => Package["tzdata"]
    }

    exec { "Set Timezone":
        unless => "diff /etc/localtime /usr/share/zoneinfo/`cat /etc/timezone`",
        command => "dpkg-reconfigure -f noninteractive tzdata",
        require => File["/etc/timezone"]
    }
}

OK, so we’ve got some pretty clear examples of code to run here. The first Exec statement must always be in there, otherwise it gets a bit confused, but after that, we’re making sure the package tzdata is installed, we then make sure that, once the tzdata package is installed, we create or update the /etc/timezone file with the value we want, and then we use the dpkg-reconfigure command to set the timezone, but only if the timezone isn’t already set to that.

Just to be clear, this file describes what the system should look like at the end of it running, not a step-by-step guide to getting it running, so you might find that some of these packages install out of sequence, or something else might run before or after when you were expecting it to run. As a result, you should make good use of the “require” and “unless” statements if you want a proper sequence of events to occur.

Now, so far, all this does is set the timezone for us, it doesn’t set up anything like Apache or MySQL… perhaps you want to install something like WordPress here? Well, let’s see how we get other packages installed.

In the following lines of code, we’ll assume you’re just adding this text above the last curled bracket (the “}” at the end).

First, we need to ensure our packages are up to date:

exec { "Update packages":
    command => "sudo apt-get update && sudo apt-get dist-upgrade -y",
}

Here’s Apache getting installed:

package { "apache2":
    ensure => "installed",
    require => Exec['Update packages']
}

And, maybe you’ll want to set up something that needs mod_rewrite and a custom site? Add this to your Vagrantfile

config.vm.synced_folder "../Apache_Site", "/etc/apache2/shared_config"

Create a directory called /path/to/your/dev/environment/Apache_Site which should contain your apache site configuration file called “default”. Then add this to your site.pp

exec { "Enable rewrite":
    command => 'a2enmod rewrite',
    onlyif => 'test ! -e /etc/apache2/mods-enabled/rewrite.load',
    require => Package['apache2']
}

file { "/etc/apache2/sites-enabled/default":
  ensure => link,
  target => "/etc/apache2/shared_config/default",
}

So, at the end of all this, we have the following file structure:

/path/to/your/dev/environment
+ -- /Apache_Site
|    + -- default
+ -- /web
|    + -- index.html
+ -- /Vagrant
     + -- /manifests
     |    + -- site.pp
     + -- Vagrantfile

And now, you can add all of this to your Git repository [2], and off you go! To bring up your Vagrant machine, type (from the Vagrant directory):

vagrant up

And then to connect into it:

vagrant ssh

And finally to halt it:

vagrant halt

Or if you just want to kill it off…

vagrant destroy

If you’re tweaking the provisioning code, you can run this instead of destroying it and bringing it back up again:

vagrant provision

You can do some funky stuff with running several machines, and using the same puppet file for all of those, but frankly, that’s a topic for another day.

[1] Vagrant is extended using plugins. There is a list of plugins on this Github Wiki Page. The plugins here can include additional virtual machine back ends (called Providers in Vagrant terminology), and methods of configuring the OS after bootup (called Provisioners), but also anything around defining where to find resources, to define network addresses, even to handle caches and proxies.

[2] If you’re not using Git, you should be! However, you might want to add some stuff to your .gitignore – in particular, Vagrant adds a directory called /path/to/your/dev/environment/Vagrant/.vagrant where it puts the VMs it creates.

Stripping a UK O2 Samsung Galaxy SIII Mini down to the bare essentials

The company I work for have recently issued all On Call engineers in my team a Samsung Galaxy SIII Mini to give us access to company e-mail and resources out of hours. Rather than shipping a customized image, we have received a stock O2 imaged mobile, and so this is my limited guide to bringing this to as close to “Stock” Android as I can manage (or want).

Most of what we need is provided to us using a commercial solution called Touchdown, so I won’t be covering that here, as whatever you get shipped to you will not include that. I’ve elected not to use the device for my personal systems, barring my Google calendar, which means I’ll stand a fighting chance of not booking overtime and other work things for personal appointments.

So, on power-on, I completed the post-install steps, including setting up my Google account. I decided not to keep the device in sync with my Google account, as I already have a few other Android devices, and I don’t want to get my work infrastructure mixed up with my home kit.

Next, I went into Settings, and from there into the Google Account. I clicked on my e-mail address and unselected the following options:

  • Sync App Data
  • Sync Contacts
  • Sync Gmail
  • Sync Internet
  • Sync People details

After that, I went into Application Manager (again, in Settings), and swapped to the “All” tab. Firstly, I needed to clear out the downloaded contacts, which I did by selecting Contacts Storage, and then pressing the “Clear Data” button.

Next, I disabled all the applications that I either don’t need, or don’t want on my work phone. I did this by selecting each in turn, and then selecting the option to disable them. Here’s the list:

  • Amazon MP3
  • eBay
  • Flipboard
  • Gallery
  • Game Hub
  • Gmail
  • Google Play Books
  • Google Play Magazines
  • Google Play Music
  • Google+
  • Music
  • O2 Space
  • S Planner
  • S Planner Widget
  • S Suggest
  • S Voice
  • Samsung Account
  • Samsung Apps
  • Samsung Backup Provider
  • Samsung Browser SyncAdapter
  • Samsung Calendar SyncAdapter
  • Samsung Cloud Data Relay
  • Samsung Contact SyncAdapter
  • Samsung Push Service
  • Samsung Syncadapters
  • Tags
  • Talk
  • Talkback
  • Video Hub
  • Yahoo! Finance
  • Yahoo! News
  • YouTube

Wow, isn’t that a list!

My next step was to hide some of the applications I don’t need. To do this, I went into the applications page, pressed the menu button, and selected “Hide applications”. This puts selection boxes next to all the applications on the page, and once you’ve done selecting options, press “Done” in the top right corner to hide them. Here’s my list:

  • Contacts
  • Downloads
  • E-Mail
  • FM Radio
  • Google Settings
  • Help
  • Memo
  • Music Player
  • My Files
  • Video Player
  • Voice Recorder
  • Voice Search

Lastly, installed a couple of applications from the Play Store:

Once I’d got Agenda Widget Plus, and Google Keyboard configured, I hid those applications in the applications pane too.

After all of that, I set up Touchdown… which you’ll need to follow up though your own instructions!

One final thing before I wrap this all up… even though I’m on-call, this doesn’t include being engaged via e-mail. As such, my e-mail doesn’t need to disturb me, and so I’ve disabled the touchdown application’s notifications for e-mail. To do this, go into Touchdown, make sure you’re at the “main” screen (not the default e-mail screen, but the one which also includes all your tasks and calendar options), and then press the menu button, press “Settings”, and select the “Advanced” tab. Scroll right to the bottom of the list, and press the “Email Alerts” button. Select “Customize settings” and then select appropriate options. If you leave nothing ticked, all you’ll get is a flag in the notifications tray showing an e-mail has appeared. Personally, I’ve turned on “Enable lights” and picked a colour, so I can quickly see whether I’ve had a mail just by checking the screen.

The noisy pub, a wifi parable

We’ve all been there – you’re the first in the pub, and you start a conversation with a mate and you’re both talking normally, then someone else comes in and you start having to talk louder, or you start missing what the other person says… it’s even worse if you’re in a weird part of the pub, you know, where you’re stood either side of a pillar and you can’t quite make out what your friend is saying because they keep moving just out of view. OK, so maybe the last bit is a bit of a stretch, but this is a bit like what happens with wifi. If you were in a pub and you couldn’t talk, you’d move to somewhere it’s easier to hear them, or you’d maybe even consider leaving the pub. With wifi you don’t have this option… but you can do some things to fix the problem.

This post was inspired by a short conversation I had yesterday, starting with this exchange:

@GizziErskine:  I hate the Internet in my house. I hate it I hate it I hate it. #hate (Link)

@jontheniceguy: @GizziErskine what’s wrong with the internet access in your house? Slow, unreliable or randomly rubbish? Happy to try to troubleshoot! (Link)

@GizziErskine: @jontheniceguy only works one side of my very tiny flat but does work the other side when it fancies it. (Link)

The solution? Well, first thing is first, you need to work out what exactly you’re fixing. You can install a WiFi analyser (this handy free android app is what I use at home) to see whether you’re getting a signal from your access point (sometimes it’s part of the router or modem you got from your ISP) at the place where you’re having issues.

Signal meter for android wifi analyser
The signal meter display in the wifi analyser application. It’s simple display makes it very simple to determine where you’re having wifi issues, and why.

A quick win might just be to move the router around a bit until you get a stronger signal, and it’s usually best to put the antennas on your access point (if you have them) to about a 45 degree angle to the floor, and as high up as possible. You might want to review this article for some pro tips on where to put it!

Aside from that, your next best bet is to get a wifi extender installed into your house. There are a few different models from a wide variety of vendors, but this one seems like the easiest I’ve found.

Image of the TP-Link TL-WPA4220KIT AV500 Powerline 300M Wi-Fi Booster/Extender with Two LAN Ports Kit
TP-Link TL-WPA4220KIT AV500 Powerline 300M Wi-Fi Booster/Extender with Two LAN Ports Kit as found at Amazon

Essentially, with this produce, there are two boxes, one that you plug a network cable into your router and into the wall, and the other you plug somewhere in the area where you’ve got little or no coverage. The network signals travel over your mains cabling, which you’ve got strung around your house anyway, and so can give you more coverage. Of course, it’s still a signal being broadcast (albeit just down a wire), so in a large block of flats you might get some interference from other devices on the same bit of wire, and some older models (with no encryption) reported that people a couple of doors away from where they were got to see some of that traffic, if they were looking… so turn on encryption!

I think I’ve covered everything here, but if you’ve got any questions, feel free to ask in the comments below, or send me an e-mail to jon@sprig.gs and I’ll see what I can do to help!

Broadcom BCM43228 and recent Linux support

I have an Acer V5-171 laptop, with a BCM43228 802.11a/b/g/n wireless network adaptor. In Ubuntu 12.04 and 12.10, I had absolutely no issues with my wireless connectivity. I upgraded to Ubuntu 13.04, and the wifi device dropped out.

I fixed the wifi by performing the following command (found via this forum post):

sudo apt-get install --reinstall bcmwl-kernel-source

I’d had a few issues with my Ubuntu install – mostly due to tinkering, so I thought I’d give a few other distributions a shot. Unfortunately, the state of the support of this driver was even worse on the others I installed.

Sabayon 13.04 (note, this is from memory!): You need to edit /etc/modprobe.d/blacklist.conf and uncomment the blacklisting of the b43 module. You need to comment the blacklisting of the 5 or so modules above it (mostly to enable the “wl” module). While this brought the NIC up, it didn’t survive an upgrade of packages, and by this point I’d spent about 2 days on it, so I was getting ratty, and wanted to try something else.

Fedora 18: Also didn’t work – I checked this distro because of my issues with Sabayon, but I figured that as it wasn’t working, perhaps there was something fundamental going on – probably either installing a package, or blacklisting a module would have solved this – I won’t know now!

OpenSUSE 12.3: I finally settled on installing OpenSUSE after I’d realised my issues were just with the module and not the distribution. I’d considered running OpenSUSE for some time and thought I’d give it a shot. I found a post (which I’ve subsequently lost) which showed that the package wasn’t installed by default to support this adaptor, so I found this page which listed both the relevant kernel module (in my case the x86_64 12.3 package) and the matching software package. As I was doing the install semi-offline (I can’t tether my phone to the laptop right now, and had no wired access) I transferred the relevant RPMs over, and installed them using rpm (the RedHat/Fedora/OpenSUSE/etc package manager). Wireless came up, but I’m missing certain APs – probably a configuration item that I’ve not yet fixed. It’s not disastrous, but is annoying :)

A quick update

So, my last post ended with me sacking it all in. Fortunately for open source, and my projects in general, a few people stepped up and reminded me why I do this stuff. So, CCHits is still going, and I don’t feel as alone any more with it (which is nice :) ) and CFM development is back on the cards. MOTP-AS is still a bit on hiatus until I get my head around a MVC framework (I’m currently using ZF2 in CFM3).

What is really nice about where I am right now is that I’m learning stuff about the tools I’m using, so it’s not all focusing on stuff which isn’t working, and is instead focusing on the shiny :)

I’ll try and get some posts about Vagrant, Puppet, and ZF2 out as I discover non-trivial stuff about it :)

The Apathy of the Lone Coder

I think I might be having a bit of a mid-life crisis. It’ll be my 35th birthday this year, and I’ve started to realise that I don’t really want to do much more of the Open Source’y stuff that I’ve been a part of for the past 10 or so years.

Don’t get me wrong. This isn’t me saying I want to hang up my linux user hat, put away the android phone, wipe the PHP manuals from my kindle or return an HTTP 410 code for everything I’ve ever published… but it’s getting close.

The rot has been setting in for some time.

November 2011 was the “first birthday” of CCHits.net – I’d planned to have my site-wide re-write of the whole code base ready for the birthday, but frankly, I’d massively underestimated the amount of work involved, so it wasn’t ready for November. As it was, a critical failure on my web host prompted me to “make” the rewrite work in April – nearly half a year after it was supposed to be in by. I’m not at all happy with the site layout, the way the tracks are build, the lack of adoption of the service by any other podcasters than the three who currently submit to the site (no criticisms there for anyone else, just a frustration really) and, well, the fact it never really achieved the vision I had for it.

In April, I helped to organise UCubed – a one day unconference about Linux and Open Source [1], held at MadLab, Manchester. We put less effort into organising it than we had the last few times, I pretty much wimped out on the day, taking my son to his swimming lesson (which meant leaving two hours after the event started, and returning an hour before it finished), and after the event, I felt like all I’d done was go to get the refreshments.

In July and August, I pulled a lot of 2 and 3AM finishes to get CampFireManager ready for OggCamp. I had some solid support from a guy called Jack who committed a load of great code to the project, plus loads of encouragement from the organisation team for OggCamp, the big day came, and, well, let’s just say there were issues. Quite a lot of issues really. I missed all of both mornings of talks because I was fire fighting those issues, and on the second day, I was held up as an example of “why not to code something instead of just doing it”. I had a top notch PHP engineer [2] sitting next to me while I was looking through issues, and even though I’ve gone through the theory of how the site works with her before, she couldn’t get her head around it. OK, I was skimming through the code pretty fast and I know most of it like the back of my hand so I knew roughly where code had gone and was going to next but still… code is code, right? Not if it’s crap code with unusual structure, insufficient testing, incomprehensible logic and, well, it’s just crap…

Before OggCamp, I inadvertently became the project lead for something I still don’t fully understand (although I’m a lot closer on it, to be fair): MOTP-AS. An implementation of the Mobile One Time PIN algorithm, written in PHP, tied up to a FreeRadius Server with a pretty web UI to give something a bit like RSA SecurID Authentication Manager server. Essentially, I made some suggestions on how to improve the code, and was told “Well, actually, we were pretty much going to kill off the project after the next release – do you want to take it over?” and I, in hindsight, stupidly said “Oh, OK”. I said that from October, I’d have “loads” of time, and was going to re-write the code base using Object Oriented principals, was going to roll in Unit Testing, PHPDocumentor and, theoretically, move to using a sensible framework to render the whole thing.

The hindsight thing I mentioned there? On the 28th August, my father passed away. I’ve not really talked about it much on Social Media. It’s a pretty hard thing to do, as it may mean airing an awful lot of dirty laundry as a result, but I guess the outcome of that was that I’ve been spending a lot more time away from my home, staying instead at my fathers home where I have been clearing it to sell it, and when I’ve not been away from home, I’ve wanted to spend more time with Jules and Daniel.

The first couple of trips down to my Dad’s house were on the train. I tried to break open a text editor and start turning out reusable PHP which I could form into something in MOTP-AS, but let’s be serious about this, it was like trying to read a book in the same circumstances – you just keep reading the same page over and over again, but nothing “right” comes out the other side. I’ve not had the enthusiasm to even start to look at that project since then.

Everyone I was working with – CCHits, CampFireManager, MOTP-AS – all knew I was offline, and would be “for some time”, but the funk that set in on that train hasn’t shifted yet, and I still can’t work out if it’s something to do with my Dad, or just the fact that I’m not really feeling the code right now.

At a recent PHPNW session, Lorna said (although I am paraphrasing) that most of my bad practices come from a lack of exposure to other PHP developers, and that working as part of a team towards something would help. My day job has nothing to do with coding (and there’s no scope to bring it into my role, and the few times I’ve tried to bring it in, it’s caused me more issues with my work than if I hadn’t) and 5% to do with open source software (the 5% is due to the OS that many of the devices we support are RedHat, BSD or Solaris based). I don’t want to, and can’t afford to make a career change now (aside from anything else, I still love my job, especially what I’m doing at the moment) to get that experience, and I’m getting closer and closer to burning out on the projects I’m involved in – just because there’s no one else who understands it like I do… which is sad.

When I do start to code in the evenings, what I tend to do is think of something I’d like to write (yep, starting a new project will fix *everything* Jon!), open my IDE, try and work out what I want to learn to use this time, and start reading the documentation for it… and not actually start working on the project. And then 2 hours have passed, I’ve done nothing, and frankly I could do with going to bed.

So, how do I beat this apathy folks. Is there anyone out there who can help?

I think if I’ve not sorted something out by June, I’ll close down CCHits.net. It’s been a great blast, but I’m so nervous of something going wrong with the system and it collapsing like a pack of cards… which is a real shame as HPR [3] have just said they’ll be running the daily shows in their Icecast server when “real” feeds aren’t being streamed, that and I love discovering, or re-discovering the music which is played through the system.

Likewise, I think I’ll probably try and find someone to hand CFM over to during OggCamp this year, and if I can’t find someone to hand it over to, I’ll shut it down. Again, it’s been fun, but I don’t need 2 months of sleepless nights and 2 days of sheer panic for something which ultimately could be replaced by a sheet of paper and some post-it notes.

Of all of the projects I’ve mentioned, the MOTP-AS part is most likely to be something of use to me in my day job (which was, in fact, how I came across it… for our lab network), so I might make more of an effort with that, but again, I really can’t see me being happy with it at the end of it all.

[1] It used to be about more than that, but frankly, it’s what it turned into.
[2] Plug for that top notch PHP engineer who, fortunately for me, was happy (or if not actually happy, appeared to be happy enough) to be an observer, a person to bounce ideas off, a muse and cheerleader (sort-of) for those two days of hell – http://LornaJane.net
[3] HackerPublicRadio.net – a podcast network made up from individual posts by the community.