The Apathy of the Lone Coder

I think I might be having a bit of a mid-life crisis. It’ll be my 35th birthday this year, and I’ve started to realise that I don’t really want to do much more of the Open Source’y stuff that I’ve been a part of for the past 10 or so years.

Don’t get me wrong. This isn’t me saying I want to hang up my linux user hat, put away the android phone, wipe the PHP manuals from my kindle or return an HTTP 410 code for everything I’ve ever published… but it’s getting close.

The rot has been setting in for some time.

November 2011 was the “first birthday” of CCHits.net – I’d planned to have my site-wide re-write of the whole code base ready for the birthday, but frankly, I’d massively underestimated the amount of work involved, so it wasn’t ready for November. As it was, a critical failure on my web host prompted me to “make” the rewrite work in April – nearly half a year after it was supposed to be in by. I’m not at all happy with the site layout, the way the tracks are build, the lack of adoption of the service by any other podcasters than the three who currently submit to the site (no criticisms there for anyone else, just a frustration really) and, well, the fact it never really achieved the vision I had for it.

In April, I helped to organise UCubed – a one day unconference about Linux and Open Source [1], held at MadLab, Manchester. We put less effort into organising it than we had the last few times, I pretty much wimped out on the day, taking my son to his swimming lesson (which meant leaving two hours after the event started, and returning an hour before it finished), and after the event, I felt like all I’d done was go to get the refreshments.

In July and August, I pulled a lot of 2 and 3AM finishes to get CampFireManager ready for OggCamp. I had some solid support from a guy called Jack who committed a load of great code to the project, plus loads of encouragement from the organisation team for OggCamp, the big day came, and, well, let’s just say there were issues. Quite a lot of issues really. I missed all of both mornings of talks because I was fire fighting those issues, and on the second day, I was held up as an example of “why not to code something instead of just doing it”. I had a top notch PHP engineer [2] sitting next to me while I was looking through issues, and even though I’ve gone through the theory of how the site works with her before, she couldn’t get her head around it. OK, I was skimming through the code pretty fast and I know most of it like the back of my hand so I knew roughly where code had gone and was going to next but still… code is code, right? Not if it’s crap code with unusual structure, insufficient testing, incomprehensible logic and, well, it’s just crap…

Before OggCamp, I inadvertently became the project lead for something I still don’t fully understand (although I’m a lot closer on it, to be fair): MOTP-AS. An implementation of the Mobile One Time PIN algorithm, written in PHP, tied up to a FreeRadius Server with a pretty web UI to give something a bit like RSA SecurID Authentication Manager server. Essentially, I made some suggestions on how to improve the code, and was told “Well, actually, we were pretty much going to kill off the project after the next release – do you want to take it over?” and I, in hindsight, stupidly said “Oh, OK”. I said that from October, I’d have “loads” of time, and was going to re-write the code base using Object Oriented principals, was going to roll in Unit Testing, PHPDocumentor and, theoretically, move to using a sensible framework to render the whole thing.

The hindsight thing I mentioned there? On the 28th August, my father passed away. I’ve not really talked about it much on Social Media. It’s a pretty hard thing to do, as it may mean airing an awful lot of dirty laundry as a result, but I guess the outcome of that was that I’ve been spending a lot more time away from my home, staying instead at my fathers home where I have been clearing it to sell it, and when I’ve not been away from home, I’ve wanted to spend more time with Jules and Daniel.

The first couple of trips down to my Dad’s house were on the train. I tried to break open a text editor and start turning out reusable PHP which I could form into something in MOTP-AS, but let’s be serious about this, it was like trying to read a book in the same circumstances – you just keep reading the same page over and over again, but nothing “right” comes out the other side. I’ve not had the enthusiasm to even start to look at that project since then.

Everyone I was working with – CCHits, CampFireManager, MOTP-AS – all knew I was offline, and would be “for some time”, but the funk that set in on that train hasn’t shifted yet, and I still can’t work out if it’s something to do with my Dad, or just the fact that I’m not really feeling the code right now.

At a recent PHPNW session, Lorna said (although I am paraphrasing) that most of my bad practices come from a lack of exposure to other PHP developers, and that working as part of a team towards something would help. My day job has nothing to do with coding (and there’s no scope to bring it into my role, and the few times I’ve tried to bring it in, it’s caused me more issues with my work than if I hadn’t) and 5% to do with open source software (the 5% is due to the OS that many of the devices we support are RedHat, BSD or Solaris based). I don’t want to, and can’t afford to make a career change now (aside from anything else, I still love my job, especially what I’m doing at the moment) to get that experience, and I’m getting closer and closer to burning out on the projects I’m involved in – just because there’s no one else who understands it like I do… which is sad.

When I do start to code in the evenings, what I tend to do is think of something I’d like to write (yep, starting a new project will fix *everything* Jon!), open my IDE, try and work out what I want to learn to use this time, and start reading the documentation for it… and not actually start working on the project. And then 2 hours have passed, I’ve done nothing, and frankly I could do with going to bed.

So, how do I beat this apathy folks. Is there anyone out there who can help?

I think if I’ve not sorted something out by June, I’ll close down CCHits.net. It’s been a great blast, but I’m so nervous of something going wrong with the system and it collapsing like a pack of cards… which is a real shame as HPR [3] have just said they’ll be running the daily shows in their Icecast server when “real” feeds aren’t being streamed, that and I love discovering, or re-discovering the music which is played through the system.

Likewise, I think I’ll probably try and find someone to hand CFM over to during OggCamp this year, and if I can’t find someone to hand it over to, I’ll shut it down. Again, it’s been fun, but I don’t need 2 months of sleepless nights and 2 days of sheer panic for something which ultimately could be replaced by a sheet of paper and some post-it notes.

Of all of the projects I’ve mentioned, the MOTP-AS part is most likely to be something of use to me in my day job (which was, in fact, how I came across it… for our lab network), so I might make more of an effort with that, but again, I really can’t see me being happy with it at the end of it all.

[1] It used to be about more than that, but frankly, it’s what it turned into.
[2] Plug for that top notch PHP engineer who, fortunately for me, was happy (or if not actually happy, appeared to be happy enough) to be an observer, a person to bounce ideas off, a muse and cheerleader (sort-of) for those two days of hell – http://LornaJane.net
[3] HackerPublicRadio.net – a podcast network made up from individual posts by the community.

Building a WPA2 Protected Wireless Access Extender for Jogglers using Ubuntu 12.04

Shesh! What a lot of keywords in the title!

For those who don’t know what some of those key words were, I’ll break down the title

  • Ubuntu is a Linux distribution, and 12.04 is the version number of the latest Long Term Stable version.
  • Joggler is the name of a device sold by O2 a couple of years ago. It is a re-branded OpenPeak tablet.
  • A Wireless Access Extender is a device like a WiFi enabled router, but it uses the same DHCP pool and should use the same SSID name and WPA2 passphrase.
  • WPA2 is the latest incarnation of the WiFi security protocol. It is currently (at this time, as far as I know) uncracked, unlike WPA1 or WEP.

So, now that we know what I’m talking about, let’s look at what components we will be using today.

  • An O2 Joggler. EBay lists them from between £30 and £100. They originally sold for around £100, but got popular when O2 dropped the price to £50. They are no longer available for sale from O2, hence EBay.
  • A wired network connection. I’m using a pair of Ethernet over Power (or “HomePlug”) devices to let me position this device in a useful place in my house. I’ve had a lot of success with the 200M devices sold by 7DayShop.com, but if I were buying new today, I’d probably stretch up to the 500M devices, as they will be Half Duplex (like a narrow street permitting traffic only one way at a time), and will loose some data due to interference and “collisions” – where two devices on the Ethernet over Power “network” are talking at the same time. Ultimately, you won’t get the equivalent to 100M Full Duplex with the 200M devices, but should do with the 500M devices.
  • A USB stick. This needs to be 4Gb or greater, but not all devices are suitable. I bought some 4Gb sticks from 7DayShop.com and found they only actually held around 3.5Gb… making them unsuitable. I bought three 8Gb sticks from 7DayShop.com, but only used one for this task!
  • A Ubuntu 12.04 install. Actually, I used the Xubuntu 12.04 image, because I didn’t need everything that Ubuntu 12.04 gave me. This is a special non-official build of Xubuntu, customised for Joggler hardware and it’s touchscreen, and is what I’ll be moving all my Jogglers in the house to, eventually, however, the principals in making all of this stuff work will apply just as much to Ubuntu as it would Xubuntu – special build or not!
  • Once installed, you’ll use a combination of VNC and SSH to manage your device, these will be through the X11VNC project and OpenSSH-Server. You should have an SSH client (for Linux/Mac, ssh should be fine, for Windows, use PuTTY) and a VNC client (for Ubuntu, I use Remmina, for Windows, I use TightVNC).

So, you’ve got all your goodies, and you’re ready to go. Let’s do this!

  1. Transfer the Xubuntu image to the USB stick. This is a simple task, and is clearly documented on the site where I got the Xubuntu image from, and involves you copying the image directly to the USB stick, not to one of it’s partitions. It sounds complicated, it really isn’t.
  2. Stick the Xubuntu stick into the side of the Joggler. Get used to that shape, as it’s going to be in the side of that from now on. This is because the Linux distribution needs more than the 1Gb that the Joggler holds internally.
  3. Plug in the HomePlug device – make it as close to the wall as you can make it! I’ve had experience of it being three 4way plug strips away from the wall and it worked fine, but I’ve also had the same HomePlug only one 4way away, and it’s completely failed to work, and had to juggle all my sockets to get it plugged directly into the wall. I think it may be down to the number of “noisy” plugs in the same 4way, but I can’t be sure. Just experiment!
  4. Plug your Ethernet cable between the HomePlug and the Joggler.
  5. Power on the Joggler. It will start up with an O2 logo (or possibly an “OpenPeak” logo – depends on when the device was manufactured)  – sometimes either of these may corrupt or show with a big white block as it’s booting. Don’t worry too much about this, we’ll stay away from the boot screen as much as possible! :)
  6. Once you get to a blue screen with icons on it – this is Xubuntu (well, actually XFCE4, but the semantics are moot really). Click on the blue spot in the top left corner of the screen – it may be a little fiddly – and select Ubuntu Software Centre.
  7. Open the “Florence” keyboard – found by pressing the small grid icon near the clock in the top right corner of the screen. If you struggle with this keyboard (I did), you may find it easier to use the “OnBoard” keyboard, found through the applications menu (again, via the blue button in the top corner).
  8. Select the Search box in the Software Centre and search for OpenSSH-Server. Click on the only entry which comes back (you need to search for the exact term) and then click install. While that’s installing, click on the two arrows icon in the top right corner, and select Connection Information. Make a note of the IP address you have received. Once it’s finished installing you can move away to something a little more comfortable to work on your Joggler!
  9. SSH to your Joggler’s IP address – the username for the device is “joggler” and the password is also “joggler”. All of the following you’ll need to be root for. I always use the following line to become root:
    sudo su -
  10. The wireless driver that is installed by default on the Jogglers don’t support “Master” mode – the mode you need to be a wifi access point or extender, so you’ll need to change the wireless driver. Thanks to this post, we know that you edit the file /etc/modprobe.d/joggler.conf and move the comment symbol (#) from before the line blacklist rt2870sta to the line blacklist rt2800usb. It should look like this after you’re done:
    # blacklist rt2800usb
    blacklist rt2870sta
  11. We need to bridge the wlan0 and eth0 interfaces.
    1. Install bridge-utils using apt-get install.
    2. Now we’ll start to configure the bridge. Edit /etc/network/interfaces to create your bridge interfaces.
      auto lo
      iface lo inet loopback
      
      auto eth0
      iface eth0 inet manual
      
      auto wlan0
      iface wlan0 inet manual
          pre-up service hostapd start
          post-up brctl addif br0 wlan0
      
      auto br0
      iface br0 inet dhcp
          bridge_ports eth0 wlan0
          pre-up iptables-restore -c < /etc/iptables.rules
          post-down iptables-save -c > /etc/iptables.rules

      If you want to use a static IP address instead of a DHCP one, then change the last block (auto br0; iface br0 inet dhcp) to the following (this assumes your network is a 192.168.0/24 with .1 as your router to the outside world):

      auto br0
      iface br0 inet static
          bridge_ports eth0 wlan0
          address 192.168.0.2
          broadcast 192.168.0.255
          netmask 255.255.255.0
          gateway 192.168.0.1
    3. Setup /etc/sysctl.conf to permit forwarding of packets. Find, and remove the comment symbol (#) from the line which looks like this:
      # net.ipv4.ip_forward = 1
    4. Create your initial /etc/iptables.rules (this is based on details from this page) and then “restore” them using iptables.
      *filter
      :INPUT ACCEPT [0:0]
      :FORWARD ACCEPT [0:0]
      :OUTPUT ACCEPT [1:81]
      -A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
      -A FORWARD -m state --state INVALID -j DROP
      -A FORWARD -i wlan0 -o eth0 -j ACCEPT
      -A FORWARD -i eth0 -o wlan0 -j ACCEPT
      COMMIT
    5. Check the iptables have restored properly by running iptables -L -v which should return the following data:
      # iptables -L -v
      Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
       pkts bytes target     prot opt in     out     source               destination         
      
      Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
       pkts bytes target     prot opt in     out     source               destination
          0     0 ACCEPT     all  --  any    any     anywhere             anywhere             state RELATED,ESTABLISHED
          0     0 DROP       all  --  any    any     anywhere             anywhere             state INVALID
          0     0 ACCEPT     all  --  wlan0  eth0    anywhere             anywhere
          0     0 ACCEPT     all  --  eth0   wlan0   anywhere             anywhere            
      
      Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
       pkts bytes target     prot opt in     out     source               destination
  12. Now you’ve got a bridged interface, and your wifi adaptor is ready to go, let’s get the DHCP relay in and working right.
    1. apt-get install dhcp3-relay
    2. It’ll ask you where to forward the DHCP requests to – that is your current gateway – if you have your network as 192.168.0.0/24 with the gateway as .1, then it should be 192.168.0.1.
    3. Next, it’ll ask which interfaces to listen on – this is br0.
    4. The last screen asks for some options to configure – this is “-m forward” (without the quote marks).
  13. Last thing to do, we need to configure something to listen on the wifi interface to provide the Access Point facility to your device. This is “hostapd”.
    1. apt-get install hostapd
    2. zcat /usr/share/doc/hostapd/examples/hostapd.conf.gz > /etc/hostapd/hostapd.conf
    3. Edit /etc/hostapd/hostapd.conf replacing the following config items:
      FROM: # driver = hostapd
      TO:   driver = nl80211
      FROM: #country_code = US
      TO:   country_code = GB
      FROM: hw_mode = a
      TO:   hw_mode = g
      FROM: channel = 60
      TO:   channel = 12
      FROM: #ieee80211n = 1
      TO:   ieee80211n = 1
      FROM: #wpa = 1
      TO:   wpa = 2
      FROM: #wpa_passphrase=secret passphrase
      TO:   wpa_passphrase=MySecretPassword
      FROM: #wpa_pairwise = TKIP CCMP
      TO:   wpa_pairwise = TKIP CCMP
    4. Edit /etc/default/hostapd amending the DAEMON_CONF line to show /etc/hostapd/hostapd.conf

Reboot, and your access point should come to life! Huzzah!! Initially it’ll have the SSID of “test” (it’s in /etc/hostapd/hostapd.conf as the config line “ssid = test”) but you should probably change it to the same SSID as your main router. If you do that, ensure your WPA passphrase is the same as your main router too, otherwise your network will get very confused!

So, now you’ve got an Access extender, running Ubuntu… what else could you do with it? Well, I run one of two things on all of mine – sqeezeplay or vlc monitoring a webcam. All very useful stuff, and stuff I was doing with it before it was an access extender!

Getting started with Unit Testing for PHP

Unit testing seems like a bit of a dark art when you’re first introduced to it. “Create this new file. Tell it what is supposed to be the result when you run a test, and it’ll tell you if you’re right nor not.”

Let’s start with a pseudocode example:

test->assertTrue(1+1 = 2); // Test returns true, huzzah!
test->assertFalse(1+1 = 3); // Test returns false. Those integers must not have been large enough

I want to use PHPUnit, and for me the easiest way to get this and the rest of the tools I’ll be referring to in this collection of posts is to install “The PHP Quality Assurance Toolchain“. On my Ubuntu install, this was done as follows:

sudo pear upgrade PEAR
sudo pear config-set auto_discover 1
sudo pear install --all-deps pear.phpqatools.org/phpqatools

Now we’ve got the tools in place, let’s set up the directory structure.

/
+ -- Classes
|    + -- Config.php
+ -- Tests
     + -- ConfigTest.php

In here, you see we’ve created two files, one contains the class we want to use, and the other contains the tests we will be running.

So, let’s slap on the veneer of coating that these two files need to be valid to test.

/Classes/Config.php

<?php
class Config
{
}

/Tests/Config.php

<?php

include dirname(__FILE__) . '/../Classes/Config.php';

class ConfigTest extends PHPUnit_Framework_TestCase
{
}

So, just to summarise, here we have two, essentially empty classes.

Let’s put some code into the test file.

<?php

include dirname(__FILE__) . '/../Classes/Config.php';

class ConfigTest extends PHPUnit_Framework_TestCase
{
  public function testCreateObject()
  {
    $config = new Config();
    $this->assertTrue(is_object($config));
  }
}

We can now run this test from the command line as follows:

phpunit Tests/ConfigTest.php

phpunit Tests/01_ConfigTest.php
PHPUnit 3.6.10 by Sebastian Bergmann.

.

Time: 1 second, Memory: 3.00Mb

OK (1 test, 1 assertion)

That was nice and straightforward!

Let’s add some more code!

In ConfigTest, let’s tell it to load some configuration, using a config file.

<?php

include dirname(__FILE__) . '/../Classes/Config.php';

class ConfigTest extends PHPUnit_Framework_TestCase
{
  public function testCreateObject()
  {
    $config = new Config();
    $this->assertTrue(is_object($config));
  }

  public function testLoadConfig()
  {
    $config = new Config();
    $config->load();
  }
}

And now when we run it?

PHP Fatal error:  Call to undefined method Config::load() in /var/www/PhpBetterPractices/Tests/ConfigTest.php on line 16

Ah, perhaps we need to write some code into /Classes/Config.php

<?php
class Config
{
  public function load()
  {
    include dirname(__FILE__) . '/../Config/default_config.php';
  }
}

But, running this, again, we get an error message!

PHPUnit 3.6.10 by Sebastian Bergmann.

.E

Time: 0 seconds, Memory: 3.00Mb

There was 1 error:

1) ConfigTest::testLoadConfig
include(/var/www/PhpBetterPractices/Config/default_config.php): failed to open stream: No such file or directory

/var/www/PhpBetterPractices/Classes/Config.php:7
/var/www/PhpBetterPractices/Classes/Config.php:7
/var/www/PhpBetterPractices/Tests/ConfigTest.php:16

FAILURES!
Tests: 2, Assertions: 1, Errors: 1.

So, we actually need to check that the file exists first, perhaps we should throw an error if it doesn’t? We could also pass the name of the config file to pass to the script, which would let us test more and different configuration options, should we need them.

class Config
{
    public function load($file = null)
    {
        if ($file == null) {
            $file = 'default.config.php';
        }

        $filename = dirname(__FILE__) . '/../Config/' . $file;

        if (file_exists($filename)) {
            include $filename;
        } else {
            throw new InvalidArgumentException("File not found");
        }
    }
}

So, here’s the new UnitTest code:

class ConfigTest extends PHPUnit_Framework_TestCase
{
    public function testCreateObject()
    {
        $config = new Config();
        $this->assertTrue(is_object($config));
    }

    public function testLoadConfig()
    {
        $config = new Config();
        $config->load();
    }

    /**
     * @expectedException InvalidArgumentException
     */
    public function testFailLoadingConfig()
    {
        $config = new Config();
        @$config->load('A file which does not exist');
    }
}

This assumes the file /Config/default.config.php exists, albeit as an empty file.

So, let’s run those tests and see what happens?

PHPUnit 3.6.10 by Sebastian Bergmann.

...

Time: 0 seconds, Memory: 3.25Mb

OK (3 tests, 2 assertions)

Huzzah! That’s looking good. Notice that to handle a test of something which should throw an exception, you can either wrapper the function in a try/catch loop and, in the try side of the loop, have $this->assertTrue(false) to prevent false positives and in the catch side, do your $this->assertBlah() on the exception. Alternatively, (and much more simplely), use a documentation notation of @expectedException NameOfException and then prefix the function you are testing with the @ symbol. This is how I did it with the test “testFailLoadingConfig()”.

This obviously doesn’t handle setting and getting configuration values, so let’s add those.

Here’s the additions to the Config.php file:

    public function set($key = null, $value = null)
    {
        if ($key == null) {
            throw new BadFunctionCallException("Key not set");
        }
        if ($value == null) {
            unset ($this->arrValues[$key]);
            return true;
        } else {
            $this->arrValues[$key] = $value;
            return true;
        }
    }

    public function get($key = null)
    {
        if ($key == null) {
            throw new BadFunctionCallException("Key not set");
        }
        if (isset($this->arrValues[$key])) {
            return $this->arrValues[$key];
        } else {
            return null;
        }
    }

And the default.config.php file:

<?php
$this->set('demo', true);

And lastly, the changes to the ConfigTest.php file:

    public function testLoadConfig()
    {
        $config = new Config();
        $this->assertTrue(is_object($config));
        $config->load('default.config.php');
        $this->assertTrue($config->get('demo'));
    }

    /**
     * @expectedException BadFunctionCallException
     */
    public function testFailSettingValue()
    {
        $config = new Config();
        @$config->set();
    }

    /**
     * @expectedException BadFunctionCallException
     */
    public function testFailGettingValue()
    {
        $config = new Config();
        @$config->get();
    }

We’ve not actually finished testing this yet. Not sure how I can tell?

phpunit --coverage-text Tests/ConfigTest.php
PHPUnit 3.6.10 by Sebastian Bergmann.

....

Time: 0 seconds, Memory: 3.75Mb

OK (4 tests, 5 assertions)

Generating textual code coverage report, this may take a moment.

Code Coverage Report
  2012-05-08 18:54:16

 Summary:
  Classes: 0.00% (0/1)
  Methods: 0.00% (0/3)
  Lines:   76.19% (16/21)

@Config::Config
  Methods: 100.00% ( 3/ 3)   Lines:  76.19% ( 16/ 21)

Notice that there are 5 lines outstanding – probably around the unsetting values and using default values. If you use an IDE (like NetBeans) you can actually get the editor to show you, using coloured lines, exactly which lines you’ve not yet tested! Nice.

So, the last thing to talk about is Containers and Dependency Injection. We’ve already started with the Dependency Injection here – that $config->load(‘filename’); function handles loading config files, or you could just bypass that with $config->set(‘key’, ‘value); but once you get past a file or two, you might just end up with a lot of redundant re-loading of config files, or worse, lots of database connections open.

So, this is where Containers come in (something I horrifically failed to understand before).

Here’s a container:

class ConfigContainer
{
  protected static $config = null;

  public static function Load()
  {
    if (self::$config == null) {
      self::$config = new Config();
      self::$config->load();
    }
    return self::$Config;
  }
}

It’s purpose (in this case) is to load the config class, including any dependencies that you may need for that class, and then return that class to you. You could conceivably create a Database container, or a Request container or a User container with very little extra work, and with a few short calls, have a single function for each of your regular and routine sources of processing data, but without preventing you from being able to easily and repeatably test that data – by not going through the container.

Of course, there’s nothing to stop you just having these created in a registry class, or store them in a global from the get-go, but, I am calling these “Better Practices” after all, and these are considered to be not-so-good-practices.

Just as a note, code from this section can be seen at GitHub, if you want to use them at all.

Update 2012-05-11: Added detail to the try/catch exception catching as per frimkron’s comment. Thanks!

Trials and Tribulations of StatusNet with Meteor

I have recently moved this domain to a VPS, and the main reason was so I could speed up my StatusNet site, but a nice side effect to that was that I could add the auto-content-update plugins to my StatusNet site.

I plumped for Meteor, as it was more-or-less the defacto choice (or so it seems at the moment), and went away to follow the instructions at http://meteorserver.org. Having added my meteor server, and knowing that there’s only me that is likely to be using the auto-update plugin, I set up Apache to proxy the meteor connections.

Here’s what I’ve got:

I used the default /etc/meteord.conf, but added at the top of the file the following two lines:

SubscriberIP 127.0.0.1
ControllerIP 127.0.0.1

I started meteor and checked that meteor was running:

# netstat -an | grep 467
tcp        0      0 127.0.0.1:4670          0.0.0.0:*               LISTEN
tcp        0      0 127.0.0.1:4671          0.0.0.0:*               LISTEN

Next, I added a new file to /etc/apache2/sites-available called metor-proxy

<VirtualHost *:80>
        ServerAdmin webmaster@localhost
        ServerName meteor.jon.sprig.gs

        ErrorLog ${APACHE_LOG_DIR}/meteor-error.log

        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn

        CustomLog ${APACHE_LOG_DIR}/meteor-access.log combined

        ProxyPass / http://127.0.0.1:4670/
        ProxyPassReverse / http://127.0.0.1:4670/
</VirtualHost>

Then, I used the a2ensite script:

# a2ensite meteord-proxy
Enabling site meteord-proxy.
Run '/etc/init.d/apache2 reload' to activate new configuration!

I ensured my DNS had an entry for this hostname, it does.

Lastly, I added some lines to my StatusNet config.php file:

addPlugin('Meteor', array(
    'webserver' => 'meteor.jon.sprig.gs',
    'webport' => 80,
    'channelbase' => 'statusnet',
    'controlserver' => '127.0.0.1',
    'controlport' => '4671'
));
addPlugin('Realtime');

All looks good so far, right?

I fire up my StatusNet site, and check in firebug – the meteor.js file is being loaded OK, but straight away, it’s loading /poll.html, and not /stream.html, so I’m guessing there’s an issue here…

Head over to the console tab, and what do we see here?

Permission denied to access property 'Meteor'
parent.Meteor.register(this);

AAARRRGGGGHHH!

I’ve fallen foul of the XSS that we were trying to prevent.

But, hang on a second. Perhaps this is because we’ve configured StatusNet to use HTTPS always. ARGH.

Let’s put the proxy lines into the SSL config for apache (inside the VirtualHost *:443 section):

        ProxyPass /poll.html http://127.0.0.1:4670/poll.html
        ProxyPassReverse /poll.html http://127.0.0.1:4670/poll.html
        ProxyPass /stream.html http://127.0.0.1:4670/stream.html
        ProxyPassReverse /stream.html http://127.0.0.1:4670/stream.html
        ProxyPass /meteor.js http://127.0.0.1:4670/meteor.js
        ProxyPassReverse /meteor.js http://127.0.0.1:4670/meteor.js
        ProxyPass /push/ http://127.0.0.1:4670/push/
        ProxyPassReverse /push/ http://127.0.0.1:4670/push/

Edit the meteor.js file (from /usr/local/meteor/public_html/meteor.js) and changing all http:// to https:// and port==80 to port=443, then amending the StatusNet’s config.php to show:

addPlugin('Meteor', array(
    'webserver' => 'jon.sprig.gs',
    'webport' => 443,
    'channelbase' => 'statusnet',
    'controlserver' => '127.0.0.1',
    'controlport' => '4671',
    'protocol' => 'https'
));

OK, that’s looking a lot healthier. Oh, no it isn’t – now, my poll.html requests are going to http://jon.sprig.gs:443/poll.html ARGH.

I’m very confused now.

I’ve commented out the Meteor and realtime plugins while I try to figure it all out :(

Installing MOTP-AS under Ubuntu 11.10

Please note, I am having issues with localhost authentication. See below

MOTP-AS is a simple installable two-factor authentication system using the mOTP algorythm for generating one-time passwords. MOTP-AS integrates with FreeRadius to provide the same authentication to log in to managed servers in a consistent manner.

I’ve recently installed this on my Ubuntu 11.10 laptop and on my Ubuntu 12.04 Beta server, and the installation instructions worked on both, so I thought I’d share them with you.

Installing appropriate packages

sudo apt-get install libpam-radius-auth freeradius mysql-server phpmyadmin

Alternatively, use tasksel to install the LAMP server task, then

sudo apt-get install libpam-radius-auth freeradius

Download the latest version of motp-as from http://motp-as.network-cube.de/index.php/download/current-version

Unpack it.

tar xfz ~/Downloads/motp-as*

Setting up the database

Go into the Setup/MySQL directory of the MOTP-AS directory. Edit motp_schema.sql at the line “CREATE USER”. Change the password from motp to something more secure.

mysql -u root -p < motp_schema.sql

Now update Setup/config.php with the new password you just created.

Setting up the web site

Copy the HTML directory to /var/www/motp (or somewhere else in your web root). You may need to do this either as root, or as a user with permissions to write to /var/www

cp -Rf ~/MOTP-AS_*/HTML /var/www/motp

Note this must be done after you’ve made your changes to Setup/config.php

Setting up FreeRadius

Stop the FreeRadius service

sudo /etc/init.d/freeradius stop

Users

Backup the users file

sudo mv /etc/freeradius/users /etc/freeradius/users.dist

Edit the users file you’re about to copy in

nano ~/MOTP-AS_*/Setup/Freeradius/users

Find the part where it says “/var/www/htdocs/radius-auth.php” and change that to “/var/www/motp/radius-auth.php

Copy in the new users file

sudo cp ~/MOTP-AS_*/Setup/Freeradius/users /etc/freeradius/users

Dynamic Clients

Backup the dynamic-clients file

sudo mv /etc/freeradius/sites-available/dynamic-clients /etc/freeradius/sites-available/dynamic-clients.dist

Edit the new dynamic-clients file

nano ~/MOTP-AS_*/Setup/Freeradius/dynamic-clients

Find the three lines saying “/var/www/htdocs” and replace that string with “/var/www/motp” (I use Ctrl+W, Ctrl+R in nano to do a replace-all.)

Copy in the new dynamic-clients file

sudo cp ~/MOTP-AS_*/Setup/Freeradius/dynamic-clients /etc/freeradius/sites-available/dynamic-clients

Then make that function available

sudo ln -s /etc/freeradius/sites-available/dynamic-clients /etc/freeradius/sites-enabled/dynamic-clients

Accounting

Amend the default script to enable accounting

sudo cp /etc/freeradius/sites-available/default /etc/freeradius/sites-available/default.dist

Then edit it to use the MOTP accounting functions

sudo nano /etc/freeradius/sites-available/default

Search for the line “accounting {” then comment that whole block out with the hash/pound sign “#“. Fortunately in the distribution supplied default file, this only means commenting out a few lines, which are “detail“, “unix“, “radutmp“, “exec“, “attr_filter.accounting_response“, and then the closing “}” for that block.

If you’re using nano, press the insert key (or Ctrl+R if you can’t find that easily) and enter /home/MyUserName/MOTP-AS_v0.7.2/Setup/Freeradius/accounting (amend the path as appropriate). Replace the section “/var/www/htdocs” with “/var/www/motp“.

Save and exit

Finishing off FreeRadius

sudo /etc/init.d/freeradius start

Install your client

Personally, I have an Android device, and I chose to install the Mobile-OTP app from the Android Marketplace. I also, through work, have a Nokia 6303i Classic, on which I installed the MOTP application from the MOTP site.

I’ve heard good things about iOTP for iPhone, although I personally don’t have one.

Configuring MOTP

Go to http://localhost/motp (or https://yourdomain.com/motp)

Login with the username admin and password of motp.

Securing the admin account

Click on the red text in “First time configuration

Click on “Change password of User ‘admin’

Enter a new password. Do not set the time or uses section of this page. Click “Set“. Ignore the warning.

Click on “Home

Setting up your first user

Click on “Quick Add” (under “Wizards”)

Enter a username. It should be the username for your Ubuntu 11.10 device.

On the client, create a profile for the device. Most of them create a profile by asking for a seed, rather than a secret, so those will likely be more than 16 characters long – maybe even 20 (Mobile-OTP for Android) or 25 (MOTP Java app).

Once you’ve got your secret (on Mobile-OTP, by pushing-and-holding on the profile name and selecting “Show Secret“, on MOTP Java app, once you’ve put 0000 as the PIN for the first time to initialize it, you get a string “Init-Secret:“), put that into the “Secret” field, and then ask the user to set their pin here – I suggest 1234 initially, as the user can change it to something they want after.

Click OK, then click “Logout” and test authentication. If it all goes OK, they should be presented with “Welcome to the Mobile OTP Authentication Server“.

Under “Settings” they can change their own PIN.

Testing radius authentication works OK

Run the radius testing program, like this, as a user:

radtest username passcode localhost 0 testing123

(This assumes the default localhost password hasn’t changed)

If you get anything like “rad_recv: Access-Reject packet from host“, then you’ve failed to configure something properly, or you’ve entered the PIN or code wrong.

Restart FreeRadius in debugging mode by doing the following:

/etc/init.d/freeradius stop
/usr/sbin/freeradius -X

This will produce a large quantity of logs on-screen, so I’d suggest running the test itself from a separate window. Run the radtest command (listed above) again. Look for your error messages. In my case, I forgot to update the line in users, so I saw this error message: Could not open input file: /var/www/htdocs/radius-auth.php

To find where this fault was, I did (as root, in /etc/freeradius)

find -R 'htdocs' /etc/freeradius

And got back: users: Exec-Program-Wait = “/usr/bin/php /var/www/htdocs/radius-auth.php %{User-Name} %{User-Password} %{Client-Shortname}”

That told me the fault was in the users file.

Fix the issue, check it again, and when you get this message “rad_recv: Access-Accept packet from host” press Ctrl+C to cancel the test mode of FreeRadius, and then run:

sudo /etc/init.d/freeradius start

Configuring pam_radius_auth.conf

Edit /etc/pam_radius_auth.conf

sudo nano /etc/pam_radius_auth.conf

Find the line which says “127.0.0.1” and replace the shared secret with something you want your server to use. You will also need to amend /etc/freeradius/clients.conf and replace the “secret” in the localhost client there (by default, it’s “testing123” in freeradius).

If you want to use your OTP for all authentication credentials, edit /etc/pam.d/common-auth, or if you just want to use it with specific access protocols, edit the relevant file in /etc/pam.d for the authentication systems you want to use OTP for.

You need to add the following line – either on the line before “@include common-auth” (for non common-auth files) or after the primary comment block for common-auth.

auth sufficient pam_radius_auth.so

Open a separate terminal session to your box (especially! if you’re remote) and ensure you can still login with your regular credentials.

Then try a connection with your radius credentials. It should just work! If not, stop the freeradius server and re-run it using /usr/sbin/freeradius -X and see whether you’re getting a different error message.

** UPDATE **

I have noticed that I’m getting locked out when using my non-radius credentials. This is probably due to the placement of the line in the /etc/pam.d/common-auth – it should probably come after the pam_unix.so line, but I’ve not tested that yet. I’m also going to try to suggest that there be an optional time-out period on locked accounts to the developers of MOTP-AS.

The second issue I’m struggling with is that I’m getting errors when using the LightDM. I’m getting the following error message in /var/log/auth.log:

pam_succeed_if(lightdm:auth): requirement "user ingroup nopasswdlogin" not met by user "spriggsj"

I don’t know if this is because I’m using ecryptfs as well, or because there’s something wonky going on with the common-auth structure I’m using.

Using Amazon an EC2 instance as an off-site CronJob

I run the CCHits.net website, and part of the day-to-day tasks that running the site entailed was the daily show creation which involved creating some text-to-speech audio for insertion into the podcasts. As I run the website on shared hosting, to which I didn’t have full access to the OS, I couldn’t just install Festival on the platform, and for whatever reason (I never did figure out what went wrong), I couldn’t build Festival to run on the shared host either.

Until “The Big Rewrite” (the capitals are totally worth it), I’d been doing the text-to-speech on my home server, but frankly, I’m on DSL, which meant I needed to set up Dynamic DNS, I had to be sure the server was always up (and it wasn’t!), etc, etc, etc. While I was looking into why I couldn’t get Festival to build, someone said “Well, why not just use EC2 to do it”.

After nearly a year of faffing about trying to make the …….. stupid thing work (as is testified by the draft in this very blog called “How I built my Audio Stack for CCHits”), I finally decided to spin up an EC2 instance for just this one task.

Now, I’m not the greenest guy on the block – hell, I drive 45 minutes into work each day, but I figured, why keep an EC2 instance running all the time, when I only need it for less than 20 minutes each day, so I did some reading, and found a post on making EC2 do the hard work for you, using the scalable computing APIs, but frankly, all I actually needed was the code to make it spin up, run the task and shut down again, especially as with using his methods, I’d have needed to either create an AMI image, or download the festival voice files each time… at around 100Mb. Not good. I ended up using the stuff I did know (bash scripting, cron tasks) and ditching the stuff I didn’t (AMI files, scalable computing API). I may revisit this later to do it the way he said instead. We’ll see :)

So, here’s the crack.

Create your EC2 image. It doesn’t need to do anything fancy yet – just boot up and keep running. You’ll do some tweaks later. Make a note of the instance number – it’ll probably start i- and then 8 or so hexedecimal digits, like this: i-12345678.

On your shared web host, download the EC2 API tools. According to this page, the API tools are available from here. The first link of those two is unlikely to change for a while, the second, maybe more so. You’ll need to make sure you have Java installed on that web host.

Once you’ve got the tools, you’ll need to create an X.509 certificate and key pair. See this page to find instructions. It was pretty straightforward.

So, you now have in, let’s say for sake of argument, your home directory:

  • /home/user/ec2-api-tools-x.x.x.x # The directory containing all the EC2 API tools
  • /home/user/ec2-keys/pk-{32[A-Za-z1-0]}.pem
  • /home/user/ec2-keys/cert-{32[A-Za-z1-0]}.pem

Also, you have java in /usr/bin/java.

Create the following script also in /home/user/ec2-api-tools-x.x.x.x – I called it ec2-wrapper.

#!/bin/bash
export EC2_HOME=/home/user/ec2-api-tools-x.x.x.x
export JAVA_HOME=/usr
export EC2_KEY=/home/user/ec2-keys/pk-{32[A-Za-z1-0]}.pem
export EC2_CERT=/home/user/ec2-keys/cert-{32[A-Za-z1-0]}.pem
${EC2_HOME}/bin/$* -K ${EC2_KEY} -C ${EC2_CERT

Obviously, you should change your paths to match what you have. What this script does is to add the X.509 certs to every EC2 request, plus adds the appropriate java and EC2_HOME paths to the script before running it.

I set up a CRON job (using crontab -e) to schedule the regular startup of the instance. Here’s the entry from my crontab:

#M   H  DoM Mth DoW  Command (Regular Crontab columns)
30   1   *   *   *   /home/user/ec2-api-tools-x.x.x.x/ec2-wrapper ec2-start-instances i-12345678
30   2   *   *   *   /home/user/ec2-api-tools-x.x.x.x/ec2-wrapper ec2-stop-instances i-12345678

So, this runs the start task at 30 minutes past 1am, local server time, and the stop task at 30 minutes past 2am. The second one there is just to be on the safe side, as we’ll try to shut down the box once it’s finished processing anyway. This way, the maximum time you’ll be billed for is 1 hour of time each day.

I then logged into my EC2 machine, and created, then tweaked the script from the earlier blog post (the scalable computing one).

#!/bin/bash -x
exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1
#
# This code is licensed under an Apache License - for the original 
# code and license, please see the footer of this script.
#
# !!!IMPORTANT!!!
# Edit this file and change this next line to your own email address:
#

EMAIL=user@example.com

# Get some information about the running instance
instance_id=$(wget -qO- instance-data/latest/meta-data/instance-id)
public_hostname=$(wget -qO- instance-data/latest/meta-data/public-hostname)

# Wait 5 minutes in case we want to get in to stop this from doing stuff
Sleep 300

if [ -f /home/ubuntu/donotrun ]
then
  exit 0
fi

# Send status email
/usr/sbin/sendmail -oi -t -f $EMAIL <<EOM
From: $EMAIL
To: $EMAIL
Subject: Running EC2 Scripts

== Making sure everything is up to date ==

`cd /home/ubuntu/website-rewrite && su -c "git pull" ubuntu 2>&1`

== Running the scheduled task ==

`php /home/ubuntu/website-rewrite/scheduled_task.php`

== Notes ==

This email was generated on the EC2 instance: $instance_id

If the instance is still running, you can monitor the output of this
job using a command like:

  ssh ubuntu@$public_hostname tail -1000f /var/log/user-data.log

EOM

# Give the email some time to be queued and delivered
sleep 300 # 5 minutes

if [ -f /home/ubuntu/shutdownwhendone ]
then
  shutdown -h now
fi

exit 0

########################################################################
# For more information about the original version of this code see:
#   http://alestic.com/2011/11/ec2-schedule-instance
# The original code and its license are available on github:
#   https://github.com/alestic/demo-ec2-schedule-instance
########################################################################

So, with that, I get a script which runs on schedule, on an EC2 platform, I get a confirmation e-mail it’s running. It shuts itself down, and hopefully, keeps on trucking :)

Getting my head around coding Object Orientated PHP

I’ve been writing two open source projects over the last couple of years. My code has never really been particularly great, but I’ve been trying to learn how to improve my code and over the last few months, I’ve been really trying to polish up my coding skills.

A few months back, I attended a series of fantastic sessions at PHPNW about using Unit Testing, PHP CodeSniffer and phpDocumentor, and how these can be incorporated into Object Orientated code (or in fact, requiring Object Orientated code to implement them).

So, I went back into my main projects and started to look at how I could fix the code to start adopting these tools.

So, the first thing I needed to do was to start thinking about the structure. CCHits.net (which is the “big project” I’ve been working on recently) has several chunks of data, and these are:

User
Track
Artist
Show
ShowTrack
Vote
Chart

Each of these have been broken down into three “things” – The Object itself, a “Broker” which finds all the relevant objects, and a class to create new items, so let’s start with a user class. We’ll define a few properties in the initial class creation.

class UserObject
{
    protected $intUserID = 0;
    protected $strOpenID = "";
    protected $strCookieID = "";
    protected $sha1Pass = "";
    protected $isAuthorized = 0;
    protected $isUploader = 0;
    protected $isAdmin = 0;
    protected $datLastSeen = "";
}

By setting these as protected, it stops me from directly setting or accessing these variables from outside of the class – instead I want to do it from a function, so let’s add those in (I’ll just do one – assume these will be copied on to the rest of the values).

class UserObject
{
    protected $strOpenID = "";

    function set_strOpenID($strOpenID = "") {
        if ($strOpenID != $this->strOpenID) {
            $this->strOpenID = $strOpenID;
        }
    }

    function get_strOpenID() {
        return $this->strOpenID;
    }
}

In the set_ functions, we already do a little bit of error checking here (is the value already set to this?) but we could add other things like, on the integer items, is it actually an integer, with the boolean values ($is[SOMETHING]), make sure it’s set to 1 or 0 (or true/false).

Now, let’s add some documentation to this:

/**
 * CCHits.net is a website designed to promote Creative Commons Music,
 * the artists who produce it and anyone or anywhere that plays it.
 * These files are used to generate the site.
 *
 * PHP version 5
 *
 * @category Default
 * @package  CCHitsClass
 * @author   Jon Spriggs 
 * @license  http://www.gnu.org/licenses/agpl.html AGPLv3
 * @link     http://cchits.net Actual web service
 * @link     http://code.cchits.net Developers Web Site
 * @link     http://gitorious.net/cchits-net Version Control Service
 */
/**
 * This class deals with user objects
 *
 * @category Default
 * @package  Objects
 * @author   Jon Spriggs 
 * @license  http://www.gnu.org/licenses/agpl.html AGPLv3
 * @link     http://cchits.net Actual web service
 * @link     http://code.cchits.net Developers Web Site
 * @link     http://gitorious.net/cchits-net Version Control Service
 */
class UserObject
{
    /**
     * This is the Setter function for the strOpenID value
     *
     * @param string $strOpenID The value to set
     *
     * @return void There is no response from this function
     */
    function set_strOpenID($strOpenID = "") {
        // Do stuff
    }
}

So, let’s make it do stuff with the database. Firstly, we need to set up a connection to the database. I’ll be using PDO (PHP Database Object, I think) to set up the connection (I’ll show why in a minute).

So, here’s another class – Database, this time it’s using a singleton factory (which is to say, while it may exist many times in the code, it’ll only ever have one connection open at once) – note it’s got hard-coded authentication details here – this isn’t how it actually is in my code, but this way it’s a bit more understandable!

class Database
{
    protected static $handler = null;
    protected $db = null;

    /**
     * This function creates or returns an instance of this class.
     *
     * @return object $handler The Handler object
     */
    private static function getHandler()
    {
        if (self::$handler == null) {
            self::$handler = new self();
        }
        return self::$handler;
    }

    /**
     * This creates or returns the database object - depending on RO/RW requirements.
     *
     * @return object A PDO instance for the query.
     */
    public function getConnection()
    {
        $self = self::getHandler();
        try {
            $self->rw_db = new PDO('mysql:host=localhost;dbname=cchits', 'cchits', 'cchits', array(PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES utf8"));
            return $self->rw_db;
        } catch (Exception $e) {
            echo "Error connecting: " . $e->getMessage();
            die();
        }
    }
}

So, now let’s create yet another class, called GenericObject. This will be re-used in all of the “Object” classes to perform our main database calls. Try to move as much code up to your highest level objects – so you could have a “validateBoolean($value)” function that is used any time you set or get a boolean value here… we even mentioned something like this above!

class GenericObject
{
    protected $arrDBItems = array();
    protected $strDBTable = "";
    protected $strDBKeyCol = "";
    protected $arrChanges = array();

    /**
     * Commit any changes to the database
     *
     * @return boolean Status of the write action
     */
    function write()
    {
        if (count($this->arrChanges) > 0) {
            $sql = '';
            $strDBKeyCol = $this->strDBKeyCol;
            $values[$strDBKeyCol] = $this->$strDBKeyCol;
            $values = array();
            foreach ($this->arrChanges as $change) {
                if ($sql != '') {
                    $sql .= ", ";
                }
                if (isset($this->arrDBItems[$change])) {
                    $sql .= "$change = :$change";
                    $values[$change] = $this->$change;
                }
            }
            $full_sql = "UPDATE {$this->strDBTable} SET $sql WHERE {$this->strDBKeyCol} = :{$this->strDBKeyCol}";
            try {
                $db = Database::getConnection();
                $query = $db->prepare($full_sql);
                $query->execute($values);
                return true;
            } catch(Exception $e) {
                return false;
            }
        }
    }

    /**
     * Create the object
     *
     * @return boolean status of the create operation
     */
    protected function create()
    {
        $keys = '';
        $key_place = '';
        foreach ($this->arrDBItems as $field_name=>$dummy) {
            if ($keys != '') {
                $keys .= ', ';
                $key_place .= ', ';
            }
            $keys .= $field_name;
            $key_place .= ":$field_name";
            $values[$field_name] = $this->$field_name;
        }
        $full_sql = "INSERT INTO {$this->strDBTable} ($keys) VALUES ($key_place)";
        try {
            $db = Database::getConnection();
            $query = $db->prepare($full_sql);
            $query->execute($values);
            if ($this->strDBKeyCol != '') {
                $key = $this->strDBKeyCol;
                $this->$key = $query->lastInsertId();
            }
            return true;
        } catch(Exception $e) {
            return false;
        }
    }

    /**
     * Return an array of the collected or created data.
     *
     * @return array A mixed array of these items
     */
    function getSelf()
    {
        if ($this->strDBKeyCol != '') {
            $key = $this->strDBKeyCol;
            $return[$key] = $this->$key;
        }
        foreach ($this->arrDBItems as $key=>$dummy) {
            $return[$key] = $this->$key;
        }
        return $return;
    }
}

Any protected functions or variables are only accessible to the class itself or classes which have been extended from it (referred to as a child class). We need to make some changes to our UserObject to make use of these new functions:

class UserObject extends GenericObject
{
    // Inherited Properties
    protected $arrDBItems = array(
        'strOpenID'=>true,
        'strCookieID'=>true,
        'sha1Pass'=>true,
        'isAuthorized'=>true,
        'isUploader'=>true,
        'isAdmin'=>true,
        'datLastSeen'=>true
    );
    protected $strDBTable = "users";
    protected $strDBKeyCol = "intUserID";
    // Local Properties
    protected $intUserID = 0;
    protected $strOpenID = "";
    protected $strCookieID = "";
    protected $sha1Pass = "";
    protected $isAuthorized = 0;
    protected $isUploader = 0;
    protected $isAdmin = 0;
    protected $datLastSeen = "";

    function set_strOpenID($strOpenID = "") {
        if ($this->strOpenID != $strOpenID) {
            $this->strOpenID = $strOpenID;
            $this->arrChanges[] = 'strOpenID';
        }
    }
}

Notice I’ve stripped the “phpDoc” style comments from this re-iteration to save some space! In the real code, they still exist.

Now we have an object we can work with, let’s extend it further. It’s probably not best practice, but I find it much more convenient to create new objects by extending the UserObject into a new class called NewUserObject. Notice once we’ve set our database items, we run the create(); function, which was previously defined in the GenericObject class.

class NewUserObject extends UserObject
{
    public function __construct($data = "")
    {
        if (strpos($data, "http://") !== false or strpos($data, "https://") !== false) {
            $this->set_strOpenID($data);
        } elseif ($data != "") {
            $this->set_sha1Pass($data);
        } else {
            if (isset($_SERVER['HTTP_X_FORWARDED_FOR'])) {
                $cookie_string = $_SERVER['HTTP_X_FORWARDED_FOR'];
            } else {
                $cookie_string = $_SERVER['REMOTE_ADDR'];
            }
            $cookie_string .= $_SERVER['HTTP_USER_AGENT'];
            $cookie_string .= $_SERVER['HTTP_ACCEPT_LANGUAGE'];
            $cookie_string .= $_SERVER['HTTP_ACCEPT_ENCODING'];
            $cookie_string .= $_SERVER['HTTP_ACCEPT_CHARSET'];
            $this->set_strCookieID(sha1sum($cookie_string));
        }
        $this->datLastSeen = date("Y-m-d H:i:s");
        $_SESSION['cookie'] = sha1($cookie_string);
        return $this->create();
    }
}

Using the code $user = new NewUserObject($login_string); we can create a new user, but how about retrieving it.

This is where the reason I’m loving PDO comes into play. See, before PDO, when you did a database request, you might have had something like this:

$db = mysql_connect("localhost", "root", "");
if ($db == false) {
    die ("Failed to connect to the Database Server");
}
if (! mysql_select_db("database")) {
    die ("Failed to select the database");
}
$intUserID = mysql_real_escape_string($_GET['intUserID']);
$sql = "SELECT * FROM users WHERE intUserID = '$intUserID' LIMIT 1";
$qry = mysql_query($sql);
if (mysql_errno() > 0) {
    echo "Failed to make Database Call: " . mysql_error();
} else {
    if (mysql_num_rows($qry) == 0) {
        echo "Failed to retrieve record 1";
    } else {
        $row = mysql_fetch_array($qry);
    }
}

Now, let’s assume you’re looking for a few users? You need to do more of those mysql_real_escape_strings and mysql_num_rows() and mysql_fetch_array()s to get your data out – it’s far from clean and clear code.

How about this instead?

try {
    $db = Database::getConnection();
    $sql = "SELECT * FROM users WHERE intUserID = ? LIMIT 1";
    $query = $db->prepare($sql);
    $query->execute(array($_GET['intUserID']));
    $row = $query->fetch();
} catch(Exception $e) {
    echo $e;
    $row = false;
}

The most complicated bit here is that you’re having to prepare your SQL query and then tell it what to get. The reason we do that is that PDO will automatically ensure that anything being passed to it using the ? is sanitized before passing it into the query. If you look back at the GenericObject class we created earlier, it uses something like this there too, except there (if you work out what it’s doing) it prepares something like INSERT INTO users (intUserID) VALUES (:intUserID); and then executes it with the values like this: array(‘:intUserID’=>1) and with this you can have some very complex statements. Other code you might spot while mooching through CCHits (if you decide to) looks like this:

$sql = "UPDATE IGNORE votes SET intTrackID = ? WHERE intTrackID = ?; DELETE FROM votes WHERE intTrackID = ?";
$query = $db->prepare($sql);
$query->execute(array($intNewTrackID, $intOldTrackID, $intOldTrackID));

By wrapping it all up in a try/catch block, you can get your error dumped in one place, including a stack trace (showing where the issue turned up). You don’t need to check for mysql_num_rows – if the row doesn’t exist, it’ll just return a false. Sweet.

Where it gets REALLY nice, is that if you swap $row = $query->fetch() with $object = $query->fetchObject(‘UserObject’); and it’ll create the object for you!

So, now, with the function getUser (visible at https://gitorious.org/cchits-net/website-rewrite/blobs/master/CLASSES/class_UserBroker.php) it’ll try to return a UserObject based on whether they’re using OpenID, Username/Password or just browsing with a cookie… and if it can’t, it’ll create you a new user (based on the above criteria) and then return you that UserObject.

The last thing to add, is that I wrapped up all the nice phpDocumentor, PHP CodeSniffer functions, plus wrote a script to check for missing or incorrectly labelled functions across a suite of classes. These sit in https://gitorious.org/cchits-net/website-rewrite/trees/master/TESTS if you want to take a look around :)

EDIT 2011-08-25: Correcting some errors in the code, and to adjust formatting slightly.
EDIT 2012-05-05: Changed category, removed the duplicated title in the top line, removed some whitespace.

Experimenting with Tiny Core Linux on QEMU

In response to a post on the Ubuntu UK Loco mailing list today, I thought the perfect way to produce a cross-platform, stable web server… would be to create a QEMU bootable image of Tiny Core.

So, the first thing I did was to download a Tiny Core image. This I obtained from the Tiny Core Download Page. I then created a 512MB disk image to store my packages on.

qemu-img create tinycore-tce.img 512M

After a bit of experimenting, I ended up with this command to boot TinyCore. At the moment, it’s relatively cross-platform, but will need some tweaking to get to the point where I can do anything with it…

qemu -hda tinycore-tce.img -m 512 -cdrom tinycore-current.iso -boot d -net nic -net user,hostfwd=tcp:127.0.0.1:8008-:80 -vnc 127.0.0.1:0 -daemonize

So, let’s explain some of those options.

-hda tinycore-tce.img

This means, use the image we created before, and install it in /dev/hda on the visualised machine.

-cdrom tinycore-current.iso -boot d

Create a virtual CD using the ISO file we downloaded. Boot from the CD rather than any other media.

-m 512

Allocate the virtual machine 512Mb RAM.

-net nic -net user,hostfwd=tcp:127.0.0.1:8008-:80

Create a virtual network interface in “UserMode”, and port forward from port 80 on the dynamically allocated IP address on the virtual machine to port 127.0.0.1:8008 (which means it’s only accessible from the host machine, not from any other machine on the network)

-vnc 127.0.0.1:0 -daemonize

This makes the service “headless” – basically meaning it won’t show itself, or need a terminal window open to keep it running. If you want to interact with the system, you need to VNC to localhost. If you’ve already got a VNC service running on the machine (for example, if you’re using Vino under Ubuntu), increment the :0 to something else – I used :2, but you could use anything.

At the moment, because I’ve not had much opportunity to tweak TinyCore’s boot process, it won’t start running automatically (you have to tell it what to start when it boots), nor will it start any of the services I want from it, I’ve had to use VNC to connect to it. I’ll be trying out more things with this over the next few days, and I’ll update this as I go.

Also, I’ve not tried using the Windows qemu packages to make sure the same options all work with that system, and I’ll probably be looking into using the smb switch for the -net user option, so that as much of the data is clearly accessible without needing to drop in to the qemu session just to upload a few photos into the system. I guess we’ll see :)

Watching for file changes on a shared linux web server

$NEWPROJECT has a script which runs daily to produce a file which will be available for download, but aside from that one expected daily task, there shouldn’t be any unexpected changes to the content on the website.

As I’m hosting this on a shared webhost, I can’t install Tripwire or anything like that, and to be honest, for what I’m using it for, I probably don’t need it. So, instead, I wrote my own really simple file change monitor which runs as a CronJob.

Here’s the code:

#! /bin/bash
# This file is called scan.sh
function sha512sum_files() {
find $HOME/$DIR/* -type f -exec sha512sum '{}' \; >> $SCAN_ROOT/current_status
}
SCAN_ROOT=$HOME/scan
mv $SCAN_ROOT/current_status $SCAN_ROOT/old_status
for DIR in site_root media/[A-Za-z]*
do
sha512sum_files
done
diff -U 0 $SCAN_ROOT/old_status $SCAN_ROOT/current_status

And here’s my crontab:


MAILTO="my.email@add.ress"
# Minute Hour Day of Month Month Day of Week Command
# (0-59) (0-23) (1-31) (1-12 or Jan-Dec) (0-6 or Sun-Sat)
0,15,30,45 * * * * /home/siteuser/scan/scan.sh

And lastly, a sample of the output

--- /home/siteuser/scan/old_status 2010-10-25 14:30:03.000000000 -0700
+++ /home/siteuser/scan/current_status 2010-10-25 14:45:06.000000000 -0700
@@ -4 +4 @@
-baeb2692403619398b44a510e8ca0d49db717d1ff7e08bf1e210c260e04630606e9be2a3aa80f7db3d451e754e189d4578ec7b87db65e6729697c735713ee5ed /home/siteuser/site_root/LIBRARIES/library.php
+c4d739b3e0a778009e0d53315085d75cf8380ac431667c31b23e4b24d4db273dfc98ffad6842a1e5f59d6ea84c33ecc73bed1437e6105475fefd3f3a966de118 /home/siteuser/site_root/LIBRARIES/library.php
@@ -71 +71 @@
-88ddd746d70073183c291fa7da747d7318caa697ace37911db55afce707cd1634f213f340bb4870f1194c48292f846adaf006ad61b4ff1cb245972c26962b42d /home/siteuser/site_root/api.php
+d79e8a6e6c3db39e07c22e7b7485050007fd265ad7e9bdda728866f65638a8aa534f8cb51121a68e9287f384e8694a968b48d840d37bcd805c117ff871e7c618 /home/siteuser/site_root/api.php

While this isn’t the most technically sound way (I’m sure) of checking for file changes, at least it gives me some idea (to within 15 minutes or so) of what files have been changed, so gives me a time to start hunting.