Development Environment Replication with Vagrant and Puppet

This week, I was fortunate enough to meet up with the Cheadle Geeks group. I got talking to a couple of people about Vagrant and Puppet, and explaining how it works, and I thought the best thing to do would be to also write that down here, so that I can point anyone who missed any of what I was saying to it.

Essentially, Vagrant is program to read a config file which defines how to initialize a pre-built virtual machine. It has several virtual machine engines which it can invoke (see [1] for more details on that), but the default virtual machine to use is VirtualBox.

To actually find a virtual box to load, there’s a big list over at vagrantbox.es which have most standard cloud servers available to you. Personally I use the Ubuntu Precise 32bit image from VagrantUp.com for my open source projects (which means more developers can get involved). Once you’ve picked an image, use the following command to get it installed on your development machine (you only need to do this step once per box!):

vagrant box add {YourBoxName} {BoxURL}

After you’ve done that, you need to set up the Vagrant configuration file.

cd /path/to/your/dev/environment
mkdir Vagrant
cd Vagrant
vagrant init {YourBoxName}

This will create a file called Vagrantfile in /path/to/your/dev/environment/Vagrant. It looks overwhelming at first, but if you trim out some of the notes (and tweak one or two of the lines), you’ll end up with a file which looks a bit like this:

Vagrant.configure("2") do |config|
  config.vm.box = "{YourBoxName}"
  config.vm.hostname = "{fqdn.of.your.host}"
  config.vm.box_url = "{BoxURL}"
  config.vm.network :forwarded_port, guest: 80, host: 8080
  # config.vm.network :public_network
  config.vm.synced_folder "../web", "/var/www"
  config.vm.provision :puppet do |puppet|
    puppet.manifests_path = "manifests"
    puppet.manifest_file  = "site.pp"
  end
end

This assumes you’ve replaced anything with {}’s in it with a real value, and that you want to forward TCP/8080 on your machine to TCP/80 on that box (there are other work arounds, using more Vagrant plugins, different network types, or other services such as pagekite, but this will do for now).

Once you’ve got this file, you could start up your machine and get a bare box, but that’s not much use to you, as you’d have to tell people how to configure your development environment every time they started up a new box. Instead, we’ll be using a Provisioning service, and we’re going to use Puppet for that.

Puppet was originally designed as a way of defining configuration across all an estate’s servers, and a lot of tutorials I’ve found online explain how to use it for that, but when we’re setting up Puppet for a development environment, we just need a simple file. This is the site.pp manifest, and in here we define the extra files and packages we need, plus any commands we need to run. So, let’s start with a basic manifest file:

node default {

}

Wow, isn’t that easy? :) We need some more detail than that though. First, let’s make sure the timezone is set. I live in the UK, so my timezone is “Europe/London”. Let’s put that in. We also need to make sure that any commands we run have the right path in them. So here’s our revised, debian based, manifest file.

node default {
    Exec {
        path => '/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/sbin:/usr/sbin'
    }

    package { "tzdata":
        ensure => "installed"
    }

    file { "/etc/timezone":
        content => "Europe/London\n",
        require => Package["tzdata"]
    }

    exec { "Set Timezone":
        unless => "diff /etc/localtime /usr/share/zoneinfo/`cat /etc/timezone`",
        command => "dpkg-reconfigure -f noninteractive tzdata",
        require => File["/etc/timezone"]
    }
}

OK, so we’ve got some pretty clear examples of code to run here. The first Exec statement must always be in there, otherwise it gets a bit confused, but after that, we’re making sure the package tzdata is installed, we then make sure that, once the tzdata package is installed, we create or update the /etc/timezone file with the value we want, and then we use the dpkg-reconfigure command to set the timezone, but only if the timezone isn’t already set to that.

Just to be clear, this file describes what the system should look like at the end of it running, not a step-by-step guide to getting it running, so you might find that some of these packages install out of sequence, or something else might run before or after when you were expecting it to run. As a result, you should make good use of the “require” and “unless” statements if you want a proper sequence of events to occur.

Now, so far, all this does is set the timezone for us, it doesn’t set up anything like Apache or MySQL… perhaps you want to install something like WordPress here? Well, let’s see how we get other packages installed.

In the following lines of code, we’ll assume you’re just adding this text above the last curled bracket (the “}” at the end).

First, we need to ensure our packages are up to date:

exec { "Update packages":
    command => "sudo apt-get update && sudo apt-get dist-upgrade -y",
}

Here’s Apache getting installed:

package { "apache2":
    ensure => "installed",
    require => Exec['Update packages']
}

And, maybe you’ll want to set up something that needs mod_rewrite and a custom site? Add this to your Vagrantfile

config.vm.synced_folder "../Apache_Site", "/etc/apache2/shared_config"

Create a directory called /path/to/your/dev/environment/Apache_Site which should contain your apache site configuration file called “default”. Then add this to your site.pp

exec { "Enable rewrite":
    command => 'a2enmod rewrite',
    onlyif => 'test ! -e /etc/apache2/mods-enabled/rewrite.load',
    require => Package['apache2']
}

file { "/etc/apache2/sites-enabled/default":
  ensure => link,
  target => "/etc/apache2/shared_config/default",
}

So, at the end of all this, we have the following file structure:

/path/to/your/dev/environment
+ -- /Apache_Site
|    + -- default
+ -- /web
|    + -- index.html
+ -- /Vagrant
     + -- /manifests
     |    + -- site.pp
     + -- Vagrantfile

And now, you can add all of this to your Git repository [2], and off you go! To bring up your Vagrant machine, type (from the Vagrant directory):

vagrant up

And then to connect into it:

vagrant ssh

And finally to halt it:

vagrant halt

Or if you just want to kill it off…

vagrant destroy

If you’re tweaking the provisioning code, you can run this instead of destroying it and bringing it back up again:

vagrant provision

You can do some funky stuff with running several machines, and using the same puppet file for all of those, but frankly, that’s a topic for another day.

[1] Vagrant is extended using plugins. There is a list of plugins on this Github Wiki Page. The plugins here can include additional virtual machine back ends (called Providers in Vagrant terminology), and methods of configuring the OS after bootup (called Provisioners), but also anything around defining where to find resources, to define network addresses, even to handle caches and proxies.

[2] If you’re not using Git, you should be! However, you might want to add some stuff to your .gitignore – in particular, Vagrant adds a directory called /path/to/your/dev/environment/Vagrant/.vagrant where it puts the VMs it creates.

Stripping a UK O2 Samsung Galaxy SIII Mini down to the bare essentials

The company I work for have recently issued all On Call engineers in my team a Samsung Galaxy SIII Mini to give us access to company e-mail and resources out of hours. Rather than shipping a customized image, we have received a stock O2 imaged mobile, and so this is my limited guide to bringing this to as close to “Stock” Android as I can manage (or want).

Most of what we need is provided to us using a commercial solution called Touchdown, so I won’t be covering that here, as whatever you get shipped to you will not include that. I’ve elected not to use the device for my personal systems, barring my Google calendar, which means I’ll stand a fighting chance of not booking overtime and other work things for personal appointments.

So, on power-on, I completed the post-install steps, including setting up my Google account. I decided not to keep the device in sync with my Google account, as I already have a few other Android devices, and I don’t want to get my work infrastructure mixed up with my home kit.

Next, I went into Settings, and from there into the Google Account. I clicked on my e-mail address and unselected the following options:

  • Sync App Data
  • Sync Contacts
  • Sync Gmail
  • Sync Internet
  • Sync People details

After that, I went into Application Manager (again, in Settings), and swapped to the “All” tab. Firstly, I needed to clear out the downloaded contacts, which I did by selecting Contacts Storage, and then pressing the “Clear Data” button.

Next, I disabled all the applications that I either don’t need, or don’t want on my work phone. I did this by selecting each in turn, and then selecting the option to disable them. Here’s the list:

  • Amazon MP3
  • eBay
  • Flipboard
  • Gallery
  • Game Hub
  • Gmail
  • Google Play Books
  • Google Play Magazines
  • Google Play Music
  • Google+
  • Music
  • O2 Space
  • S Planner
  • S Planner Widget
  • S Suggest
  • S Voice
  • Samsung Account
  • Samsung Apps
  • Samsung Backup Provider
  • Samsung Browser SyncAdapter
  • Samsung Calendar SyncAdapter
  • Samsung Cloud Data Relay
  • Samsung Contact SyncAdapter
  • Samsung Push Service
  • Samsung Syncadapters
  • Tags
  • Talk
  • Talkback
  • Video Hub
  • Yahoo! Finance
  • Yahoo! News
  • YouTube

Wow, isn’t that a list!

My next step was to hide some of the applications I don’t need. To do this, I went into the applications page, pressed the menu button, and selected “Hide applications”. This puts selection boxes next to all the applications on the page, and once you’ve done selecting options, press “Done” in the top right corner to hide them. Here’s my list:

  • Contacts
  • Downloads
  • E-Mail
  • FM Radio
  • Google Settings
  • Help
  • Memo
  • Music Player
  • My Files
  • Video Player
  • Voice Recorder
  • Voice Search

Lastly, installed a couple of applications from the Play Store:

Once I’d got Agenda Widget Plus, and Google Keyboard configured, I hid those applications in the applications pane too.

After all of that, I set up Touchdown… which you’ll need to follow up though your own instructions!

One final thing before I wrap this all up… even though I’m on-call, this doesn’t include being engaged via e-mail. As such, my e-mail doesn’t need to disturb me, and so I’ve disabled the touchdown application’s notifications for e-mail. To do this, go into Touchdown, make sure you’re at the “main” screen (not the default e-mail screen, but the one which also includes all your tasks and calendar options), and then press the menu button, press “Settings”, and select the “Advanced” tab. Scroll right to the bottom of the list, and press the “Email Alerts” button. Select “Customize settings” and then select appropriate options. If you leave nothing ticked, all you’ll get is a flag in the notifications tray showing an e-mail has appeared. Personally, I’ve turned on “Enable lights” and picked a colour, so I can quickly see whether I’ve had a mail just by checking the screen.

The noisy pub, a wifi parable

We’ve all been there – you’re the first in the pub, and you start a conversation with a mate and you’re both talking normally, then someone else comes in and you start having to talk louder, or you start missing what the other person says… it’s even worse if you’re in a weird part of the pub, you know, where you’re stood either side of a pillar and you can’t quite make out what your friend is saying because they keep moving just out of view. OK, so maybe the last bit is a bit of a stretch, but this is a bit like what happens with wifi. If you were in a pub and you couldn’t talk, you’d move to somewhere it’s easier to hear them, or you’d maybe even consider leaving the pub. With wifi you don’t have this option… but you can do some things to fix the problem.

This post was inspired by a short conversation I had yesterday, starting with this exchange:

@GizziErskine:  I hate the Internet in my house. I hate it I hate it I hate it. #hate (Link)

@jontheniceguy: @GizziErskine what’s wrong with the internet access in your house? Slow, unreliable or randomly rubbish? Happy to try to troubleshoot! (Link)

@GizziErskine: @jontheniceguy only works one side of my very tiny flat but does work the other side when it fancies it. (Link)

The solution? Well, first thing is first, you need to work out what exactly you’re fixing. You can install a WiFi analyser (this handy free android app is what I use at home) to see whether you’re getting a signal from your access point (sometimes it’s part of the router or modem you got from your ISP) at the place where you’re having issues.

Signal meter for android wifi analyser
The signal meter display in the wifi analyser application. It’s simple display makes it very simple to determine where you’re having wifi issues, and why.

A quick win might just be to move the router around a bit until you get a stronger signal, and it’s usually best to put the antennas on your access point (if you have them) to about a 45 degree angle to the floor, and as high up as possible. You might want to review this article for some pro tips on where to put it!

Aside from that, your next best bet is to get a wifi extender installed into your house. There are a few different models from a wide variety of vendors, but this one seems like the easiest I’ve found.

Image of the TP-Link TL-WPA4220KIT AV500 Powerline 300M Wi-Fi Booster/Extender with Two LAN Ports Kit
TP-Link TL-WPA4220KIT AV500 Powerline 300M Wi-Fi Booster/Extender with Two LAN Ports Kit as found at Amazon

Essentially, with this produce, there are two boxes, one that you plug a network cable into your router and into the wall, and the other you plug somewhere in the area where you’ve got little or no coverage. The network signals travel over your mains cabling, which you’ve got strung around your house anyway, and so can give you more coverage. Of course, it’s still a signal being broadcast (albeit just down a wire), so in a large block of flats you might get some interference from other devices on the same bit of wire, and some older models (with no encryption) reported that people a couple of doors away from where they were got to see some of that traffic, if they were looking… so turn on encryption!

I think I’ve covered everything here, but if you’ve got any questions, feel free to ask in the comments below, or send me an e-mail to jon@sprig.gs and I’ll see what I can do to help!

Broadcom BCM43228 and recent Linux support

I have an Acer V5-171 laptop, with a BCM43228 802.11a/b/g/n wireless network adaptor. In Ubuntu 12.04 and 12.10, I had absolutely no issues with my wireless connectivity. I upgraded to Ubuntu 13.04, and the wifi device dropped out.

I fixed the wifi by performing the following command (found via this forum post):

sudo apt-get install --reinstall bcmwl-kernel-source

I’d had a few issues with my Ubuntu install – mostly due to tinkering, so I thought I’d give a few other distributions a shot. Unfortunately, the state of the support of this driver was even worse on the others I installed.

Sabayon 13.04 (note, this is from memory!): You need to edit /etc/modprobe.d/blacklist.conf and uncomment the blacklisting of the b43 module. You need to comment the blacklisting of the 5 or so modules above it (mostly to enable the “wl” module). While this brought the NIC up, it didn’t survive an upgrade of packages, and by this point I’d spent about 2 days on it, so I was getting ratty, and wanted to try something else.

Fedora 18: Also didn’t work – I checked this distro because of my issues with Sabayon, but I figured that as it wasn’t working, perhaps there was something fundamental going on – probably either installing a package, or blacklisting a module would have solved this – I won’t know now!

OpenSUSE 12.3: I finally settled on installing OpenSUSE after I’d realised my issues were just with the module and not the distribution. I’d considered running OpenSUSE for some time and thought I’d give it a shot. I found a post (which I’ve subsequently lost) which showed that the package wasn’t installed by default to support this adaptor, so I found this page which listed both the relevant kernel module (in my case the x86_64 12.3 package) and the matching software package. As I was doing the install semi-offline (I can’t tether my phone to the laptop right now, and had no wired access) I transferred the relevant RPMs over, and installed them using rpm (the RedHat/Fedora/OpenSUSE/etc package manager). Wireless came up, but I’m missing certain APs – probably a configuration item that I’ve not yet fixed. It’s not disastrous, but is annoying :)

A quick note on autoloaders for PHP

Over the past few days, as you may have noticed, I’ve been experimenting with PHPUnit, and writing up notes on what I’ve learned. Here’s a biggie, but it’s such a small actual change, I didn’t want to miss it.

So, when you have your autoloader written, you’ll have a function like this (probably):

<?php
function __autoload($classname)
{
    if (file_exists(dirname(__FILE__) . '/classes/' . $classname . '.php')) {
        require_once dirname(__FILE__) . '/classes/' . $classname . '.php';
    }
}

Load this from your test, or in a bootstrap file (more to come on that particular subject, I think!), like this:

<?php
require_once dirname(__FILE__) . '/../autoloader.php';
class SomeClassTest extends ........

And you’ll probably notice the autoloader doesn’t do anything… but why is this? Because PHPUnit has it’s own autoloader, and you need to chain our autoloader to the end. So, in your autoloader file, add this line to the end:

<?php
function __autoload($classname)
{
    if (file_exists(dirname(__FILE__) . '/classes/' . $classname . '.php')) {
        require_once dirname(__FILE__) . '/classes/' . $classname . '.php';
    }
}

spl_autoload_register('__autoload');

And it all should just work, which is nice :)

Getting started with Unit Testing for PHP

Unit testing seems like a bit of a dark art when you’re first introduced to it. “Create this new file. Tell it what is supposed to be the result when you run a test, and it’ll tell you if you’re right nor not.”

Let’s start with a pseudocode example:

test->assertTrue(1+1 = 2); // Test returns true, huzzah!
test->assertFalse(1+1 = 3); // Test returns false. Those integers must not have been large enough

I want to use PHPUnit, and for me the easiest way to get this and the rest of the tools I’ll be referring to in this collection of posts is to install “The PHP Quality Assurance Toolchain“. On my Ubuntu install, this was done as follows:

sudo pear upgrade PEAR
sudo pear config-set auto_discover 1
sudo pear install --all-deps pear.phpqatools.org/phpqatools

Now we’ve got the tools in place, let’s set up the directory structure.

/
+ -- Classes
|    + -- Config.php
+ -- Tests
     + -- ConfigTest.php

In here, you see we’ve created two files, one contains the class we want to use, and the other contains the tests we will be running.

So, let’s slap on the veneer of coating that these two files need to be valid to test.

/Classes/Config.php

<?php
class Config
{
}

/Tests/Config.php

<?php

include dirname(__FILE__) . '/../Classes/Config.php';

class ConfigTest extends PHPUnit_Framework_TestCase
{
}

So, just to summarise, here we have two, essentially empty classes.

Let’s put some code into the test file.

<?php

include dirname(__FILE__) . '/../Classes/Config.php';

class ConfigTest extends PHPUnit_Framework_TestCase
{
  public function testCreateObject()
  {
    $config = new Config();
    $this->assertTrue(is_object($config));
  }
}

We can now run this test from the command line as follows:

phpunit Tests/ConfigTest.php

phpunit Tests/01_ConfigTest.php
PHPUnit 3.6.10 by Sebastian Bergmann.

.

Time: 1 second, Memory: 3.00Mb

OK (1 test, 1 assertion)

That was nice and straightforward!

Let’s add some more code!

In ConfigTest, let’s tell it to load some configuration, using a config file.

<?php

include dirname(__FILE__) . '/../Classes/Config.php';

class ConfigTest extends PHPUnit_Framework_TestCase
{
  public function testCreateObject()
  {
    $config = new Config();
    $this->assertTrue(is_object($config));
  }

  public function testLoadConfig()
  {
    $config = new Config();
    $config->load();
  }
}

And now when we run it?

PHP Fatal error:  Call to undefined method Config::load() in /var/www/PhpBetterPractices/Tests/ConfigTest.php on line 16

Ah, perhaps we need to write some code into /Classes/Config.php

<?php
class Config
{
  public function load()
  {
    include dirname(__FILE__) . '/../Config/default_config.php';
  }
}

But, running this, again, we get an error message!

PHPUnit 3.6.10 by Sebastian Bergmann.

.E

Time: 0 seconds, Memory: 3.00Mb

There was 1 error:

1) ConfigTest::testLoadConfig
include(/var/www/PhpBetterPractices/Config/default_config.php): failed to open stream: No such file or directory

/var/www/PhpBetterPractices/Classes/Config.php:7
/var/www/PhpBetterPractices/Classes/Config.php:7
/var/www/PhpBetterPractices/Tests/ConfigTest.php:16

FAILURES!
Tests: 2, Assertions: 1, Errors: 1.

So, we actually need to check that the file exists first, perhaps we should throw an error if it doesn’t? We could also pass the name of the config file to pass to the script, which would let us test more and different configuration options, should we need them.

class Config
{
    public function load($file = null)
    {
        if ($file == null) {
            $file = 'default.config.php';
        }

        $filename = dirname(__FILE__) . '/../Config/' . $file;

        if (file_exists($filename)) {
            include $filename;
        } else {
            throw new InvalidArgumentException("File not found");
        }
    }
}

So, here’s the new UnitTest code:

class ConfigTest extends PHPUnit_Framework_TestCase
{
    public function testCreateObject()
    {
        $config = new Config();
        $this->assertTrue(is_object($config));
    }

    public function testLoadConfig()
    {
        $config = new Config();
        $config->load();
    }

    /**
     * @expectedException InvalidArgumentException
     */
    public function testFailLoadingConfig()
    {
        $config = new Config();
        @$config->load('A file which does not exist');
    }
}

This assumes the file /Config/default.config.php exists, albeit as an empty file.

So, let’s run those tests and see what happens?

PHPUnit 3.6.10 by Sebastian Bergmann.

...

Time: 0 seconds, Memory: 3.25Mb

OK (3 tests, 2 assertions)

Huzzah! That’s looking good. Notice that to handle a test of something which should throw an exception, you can either wrapper the function in a try/catch loop and, in the try side of the loop, have $this->assertTrue(false) to prevent false positives and in the catch side, do your $this->assertBlah() on the exception. Alternatively, (and much more simplely), use a documentation notation of @expectedException NameOfException and then prefix the function you are testing with the @ symbol. This is how I did it with the test “testFailLoadingConfig()”.

This obviously doesn’t handle setting and getting configuration values, so let’s add those.

Here’s the additions to the Config.php file:

    public function set($key = null, $value = null)
    {
        if ($key == null) {
            throw new BadFunctionCallException("Key not set");
        }
        if ($value == null) {
            unset ($this->arrValues[$key]);
            return true;
        } else {
            $this->arrValues[$key] = $value;
            return true;
        }
    }

    public function get($key = null)
    {
        if ($key == null) {
            throw new BadFunctionCallException("Key not set");
        }
        if (isset($this->arrValues[$key])) {
            return $this->arrValues[$key];
        } else {
            return null;
        }
    }

And the default.config.php file:

<?php
$this->set('demo', true);

And lastly, the changes to the ConfigTest.php file:

    public function testLoadConfig()
    {
        $config = new Config();
        $this->assertTrue(is_object($config));
        $config->load('default.config.php');
        $this->assertTrue($config->get('demo'));
    }

    /**
     * @expectedException BadFunctionCallException
     */
    public function testFailSettingValue()
    {
        $config = new Config();
        @$config->set();
    }

    /**
     * @expectedException BadFunctionCallException
     */
    public function testFailGettingValue()
    {
        $config = new Config();
        @$config->get();
    }

We’ve not actually finished testing this yet. Not sure how I can tell?

phpunit --coverage-text Tests/ConfigTest.php
PHPUnit 3.6.10 by Sebastian Bergmann.

....

Time: 0 seconds, Memory: 3.75Mb

OK (4 tests, 5 assertions)

Generating textual code coverage report, this may take a moment.

Code Coverage Report
  2012-05-08 18:54:16

 Summary:
  Classes: 0.00% (0/1)
  Methods: 0.00% (0/3)
  Lines:   76.19% (16/21)

@Config::Config
  Methods: 100.00% ( 3/ 3)   Lines:  76.19% ( 16/ 21)

Notice that there are 5 lines outstanding – probably around the unsetting values and using default values. If you use an IDE (like NetBeans) you can actually get the editor to show you, using coloured lines, exactly which lines you’ve not yet tested! Nice.

So, the last thing to talk about is Containers and Dependency Injection. We’ve already started with the Dependency Injection here – that $config->load(‘filename’); function handles loading config files, or you could just bypass that with $config->set(‘key’, ‘value); but once you get past a file or two, you might just end up with a lot of redundant re-loading of config files, or worse, lots of database connections open.

So, this is where Containers come in (something I horrifically failed to understand before).

Here’s a container:

class ConfigContainer
{
  protected static $config = null;

  public static function Load()
  {
    if (self::$config == null) {
      self::$config = new Config();
      self::$config->load();
    }
    return self::$Config;
  }
}

It’s purpose (in this case) is to load the config class, including any dependencies that you may need for that class, and then return that class to you. You could conceivably create a Database container, or a Request container or a User container with very little extra work, and with a few short calls, have a single function for each of your regular and routine sources of processing data, but without preventing you from being able to easily and repeatably test that data – by not going through the container.

Of course, there’s nothing to stop you just having these created in a registry class, or store them in a global from the get-go, but, I am calling these “Better Practices” after all, and these are considered to be not-so-good-practices.

Just as a note, code from this section can be seen at GitHub, if you want to use them at all.

Update 2012-05-11: Added detail to the try/catch exception catching as per frimkron’s comment. Thanks!

php-PDO-MySQL versus php-mysql

PHP and MySQL was the duo that took dynamic web development from /cgi/formmail.cgi to… well, where we are today for me. My first proper experience of working with a web application, as opposed to a static web site with a few perl scripts bolted on was PHPNuke, and I’ve not looked back since.

Almost the first lines of PHP you learn to code are

<?php
mysql_connect("localhost", "root", "");
mysql_use_db("mysite");

Followed closely by mysql_query(), then mysql_fetch_array(), mysl_num_rows(), and if the rest of us are lucky, mysql_real_escape_string().

So you see, for me, it’s really weird to find that even though this is what we’re teaching everyone to use, this weird incantation where we have to wrap every query up in a cotton wool ball of SQL injection protection… there’s an easier way where you don’t have to wrapper everything, which, even better, is more efficient at your SQL server and, if you’ve made the transition to using Classes in your code, will give you an object per-result rather than an array.

Let’s look at some examples of code, and see how we would transition things.

Here’s some rather unoriginal code to get the contents of a user table.

<?php
$db = mysql_connect("localhost", "root", "");
mysql_use_db("mysite");
$sql = "select userid, realname, lastloggedin from users "
     . "where username = '" . mysql_real_escape_string($_POST['username'])
     . "' and password = '" . mysql_real_escape_string($_POST['password']) . "'";
$query = mysql_query($sql);
$data = false;
if (mysql_num_rows($query) == 1) {
  $data = mysql_fetch_array($query);
  echo "Hello {$data['realname']}, your userid is {$data['userid']} and "
     . "you last logged in at " . date("H:i:s", strtotime($data['lastloggedin']))
     . " on " . date("d-M-Y" , strtotime($data['lastloggedin']));
}

There, now there’s not much wrong with that (ignoring the security hole of having your passwords in cleartext) is there? All pretty readable, right? Well… what happens if your database is down, how do you handle that in some kind of sensible way? You need to wrap some code around the mysql_connect to show it actually connected. What about if someone dropped the database by mistake, or you connected to the wrong database server and that table wasn’t there, now you’re wrappering the use_db line too. Did you make a typo in the SQL? Are you sure you didn’t miss a mysql_real_escape_string in there somewhere…

Here’s the same thing in PDO, protected against all (again, except the cleartext password) of those things.

<?php
try {
  $db = new PDO("mysql:host=localhost;dbname=mysite", "root", "");
  $db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
  $sql = "select userid, realname, lastloggedin from users where username = ? and password = ?";
  $query = $db->prepare($sql);
  $query->execute(array($_POST['username'], $_POST['password']));
  $data = $query->fetch();
  if ($data != false) {
    echo "Hello {$data['realname']}, your userid is {$data['userid']} and "
       . "you last logged in at " . date("H:i:s", strtotime($data['lastloggedin']))
       . " on " . date("d-M-Y" , strtotime($data['lastloggedin']));
  }
} catch (PDOException $e) {
  error_log("User unable to login: " . $e->getMessage());
}

So, let’s look at what transforms this into something much more interesting: $query->fetchObject();

<?php
try {
  $db = new PDO("mysql:host=localhost;dbname=mysite", "root", "");
  $db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
  $sql = "select userid, realname, lastloggedin, username, password " 
       . "from users where username = ? and password = ?";
  $query = $db->prepare($sql);
  $query->execute(array($_POST['username'], $_POST['password']));
  $data = $query->fetchObject('user');
  if ($data != false) {
    echo $data;
  }
} catch (PDOException $e) {
  error_log("User unable to login: " . $e->getMessage());
}

class user
{
  // Columns from the database
  protected $userid = null;
  protected $realname = null;
  protected $lastloggedin = null;
  protected $username = null;
  protected $password = null;
  // Processed Data
  protected $transformed_lastloggedin = null

  public function __construct()
  {
    if ($this->lastloggedin != null) {
      $this->transformed_lastloggedin = date("H:i:s", strtotime($this->lastloggedin)) 
                             . " on " . date("d-M-Y" , strtotime($this->lastloggedin);
    }
  }

  public function toString()
  {
    echo "Hello {$this->realname}, your userid is {$this->userid} and " 
       . "you last logged in at {$this->transformed_lastloggedin}";
  }
}

I’m starting to stretch the analogy here a little, but the important things here are:

  1. The __construct() function of the class is run after the data is inserted into it, meaning you can post-process any data you receive from the database without needing any new functions. Arguably, you could make the construct function accept the same values you’d receive from the database, and process those in the same way.
  2. The toString() function means you can mangle the content you want to output, without having to apparently run another function on the class you’ve just created.
  3. By using classes you get to do wonderful things like Unit Testing (this is a topic to follow), have automatic code documentation (err… to follow), reduce code duplication by extending classes and check on that, using “Copy and Paste Detection”.

Logitech Media Server vs Ubuntu 12.04

A while back I upgraded my home server to Ubuntu 12.04 (while it was still in beta) and immediately the first thing I noticed was that the Logitech Media Server (previously known as Squeezebox Server) had stopped working.

Checking through the logs, I saw a lot of messages about perl dependencies being missing or not working [1]. As I was a bit busy at the time (the decision to upgrade had been due to something else entirely), I put it to one side (much to my wife’s annoyance!) to pick up later.

As I’ve been wallowing at home the past couple of days with a stomach bug, and not really been fit to do much other than moan, lie there and feel sorry about myself, I thought about what I could do to get my squeezebox server back up and running.

A few Google searches later, and I turn up this page: http://forums.slimdevices.com/archive/index.php/t-89057.html which suggests that this message below is due to Ampache… which now that I look at the log entry, it kinda makes sense. Queue digging into the depths of the server.

Under Ubuntu, all the serious configuration for the server is stored in /var/lib/squeezeboxserver, which includes the plugins.

So, firstly, I deleted the downloaded Zip file from /var/lib/squeezeboxserver/cache/DownloadedPlugins/Ampache.zip

rm /var/lib/squeezeboxserver/cache/DownloadedPlugins/Ampache.zip

Next, I removed the unpacked plugin from /var/lib/squeezeboxserver/cache/InstalledPlugins/Ampache

rm -Rf /var/lib/squeezeboxserver/cache/InstalledPlugins/Ampache

I made a mistake here on my box, and restarted the server. Woohoo it came back up, but the first thing it did was to re-download the plugin again! D’oh. So, now I need to find what’s telling it to re-install the plugin. Queue a quick grep. Ahhh, there’s a file called extensions.prefs, which says:

prefs/plugin/extensions.prefs:  Ampache: 1

And another file called state.prefs which says:

prefs/plugin/state.prefs:Ampache: needs-install

Note, these are both the output from grep – so the filename includes the path from the point /var/lib/squeezeboxserver. A quick nano away (or whatever editor you prefer) and I’d removed the line from the extensions.prefs which showed Ampache: 1, but the state.prefs was marginally more tricky. In here it lists them in three states, enabled, disabled and needs-install. So, I changed it to show disabled and then restarted the service. Tada. I’ve got a working Logitech Media Server again! Huzzah!

[1] Log file looks like this:

Slim::bootstrap::tryModuleLoad (285) Warning: Module [Plugins::GrabPlaylist::Plugin] failed to load:
Can't locate Math/VecStat.pm in @INC (@INC contains: /usr/sbin/Plugins/Gallery /var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/Gallery /var/lib/squeezeboxserver/cache/Installed
Plugins/Plugins/CustomScan/lib /var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/Ampache/lib CODE(0xb911c40) /var/lib/squeezeboxserver/cache/InstalledPlugins /usr/share/squeezebo
xserver/CPAN/arch/5.14/i386-linux-thread-multi-64int /usr/share/squeezeboxserver/CPAN/arch/5.14/i386-linux-thread-multi-64int/auto /usr/share/squeezeboxserver/CPAN/arch/5.14.2/i686-linu
x-gnu-thread-multi-64int /usr/share/squeezeboxserver/CPAN/arch/5.14.2/i686-linux-gnu-thread-multi-64int/auto /usr/share/squeezeboxserver/CPAN/arch/5.14/i686-linux-gnu-thread-multi-64int
 /usr/share/squeezeboxserver/CPAN/arch/5.14/i686-linux-gnu-thread-multi-64int/auto /usr/share/squeezeboxserver/CPAN/arch/i686-linux-gnu-thread-multi-64int /usr/share/squeezeboxserver/li
b /usr/share/squeezeboxserver/CPAN /usr/share/squeezeboxserver /usr/sbin /etc/perl /usr/local/lib/perl/5.14.2 /usr/local/share/perl/5.14.2 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/
5.14 /usr/share/perl/5.14 /usr/local/lib/site_perl . CODE(0xb911e20)) at Slim/Player/Player.pm line 18.
BEGIN failed--compilation aborted at Slim/Player/Player.pm line 18.
Compilation failed in require at /var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/GrabPlaylist/Plugin.pm line 22.
BEGIN failed--compilation aborted at /var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/GrabPlaylist/Plugin.pm line 22.
Compilation failed in require at (eval 924) line 2.
BEGIN failed--compilation aborted at (eval 924) line 2.

Nice, right?