Getting started with Unit Testing for PHP

Unit testing seems like a bit of a dark art when you’re first introduced to it. “Create this new file. Tell it what is supposed to be the result when you run a test, and it’ll tell you if you’re right nor not.”

Let’s start with a pseudocode example:

test->assertTrue(1+1 = 2); // Test returns true, huzzah!
test->assertFalse(1+1 = 3); // Test returns false. Those integers must not have been large enough

I want to use PHPUnit, and for me the easiest way to get this and the rest of the tools I’ll be referring to in this collection of posts is to install “The PHP Quality Assurance Toolchain“. On my Ubuntu install, this was done as follows:

sudo pear upgrade PEAR
sudo pear config-set auto_discover 1
sudo pear install --all-deps pear.phpqatools.org/phpqatools

Now we’ve got the tools in place, let’s set up the directory structure.

/
+ -- Classes
|    + -- Config.php
+ -- Tests
     + -- ConfigTest.php

In here, you see we’ve created two files, one contains the class we want to use, and the other contains the tests we will be running.

So, let’s slap on the veneer of coating that these two files need to be valid to test.

/Classes/Config.php

<?php
class Config
{
}

/Tests/Config.php

<?php

include dirname(__FILE__) . '/../Classes/Config.php';

class ConfigTest extends PHPUnit_Framework_TestCase
{
}

So, just to summarise, here we have two, essentially empty classes.

Let’s put some code into the test file.

<?php

include dirname(__FILE__) . '/../Classes/Config.php';

class ConfigTest extends PHPUnit_Framework_TestCase
{
  public function testCreateObject()
  {
    $config = new Config();
    $this->assertTrue(is_object($config));
  }
}

We can now run this test from the command line as follows:

phpunit Tests/ConfigTest.php

phpunit Tests/01_ConfigTest.php
PHPUnit 3.6.10 by Sebastian Bergmann.

.

Time: 1 second, Memory: 3.00Mb

OK (1 test, 1 assertion)

That was nice and straightforward!

Let’s add some more code!

In ConfigTest, let’s tell it to load some configuration, using a config file.

<?php

include dirname(__FILE__) . '/../Classes/Config.php';

class ConfigTest extends PHPUnit_Framework_TestCase
{
  public function testCreateObject()
  {
    $config = new Config();
    $this->assertTrue(is_object($config));
  }

  public function testLoadConfig()
  {
    $config = new Config();
    $config->load();
  }
}

And now when we run it?

PHP Fatal error:  Call to undefined method Config::load() in /var/www/PhpBetterPractices/Tests/ConfigTest.php on line 16

Ah, perhaps we need to write some code into /Classes/Config.php

<?php
class Config
{
  public function load()
  {
    include dirname(__FILE__) . '/../Config/default_config.php';
  }
}

But, running this, again, we get an error message!

PHPUnit 3.6.10 by Sebastian Bergmann.

.E

Time: 0 seconds, Memory: 3.00Mb

There was 1 error:

1) ConfigTest::testLoadConfig
include(/var/www/PhpBetterPractices/Config/default_config.php): failed to open stream: No such file or directory

/var/www/PhpBetterPractices/Classes/Config.php:7
/var/www/PhpBetterPractices/Classes/Config.php:7
/var/www/PhpBetterPractices/Tests/ConfigTest.php:16

FAILURES!
Tests: 2, Assertions: 1, Errors: 1.

So, we actually need to check that the file exists first, perhaps we should throw an error if it doesn’t? We could also pass the name of the config file to pass to the script, which would let us test more and different configuration options, should we need them.

class Config
{
    public function load($file = null)
    {
        if ($file == null) {
            $file = 'default.config.php';
        }

        $filename = dirname(__FILE__) . '/../Config/' . $file;

        if (file_exists($filename)) {
            include $filename;
        } else {
            throw new InvalidArgumentException("File not found");
        }
    }
}

So, here’s the new UnitTest code:

class ConfigTest extends PHPUnit_Framework_TestCase
{
    public function testCreateObject()
    {
        $config = new Config();
        $this->assertTrue(is_object($config));
    }

    public function testLoadConfig()
    {
        $config = new Config();
        $config->load();
    }

    /**
     * @expectedException InvalidArgumentException
     */
    public function testFailLoadingConfig()
    {
        $config = new Config();
        @$config->load('A file which does not exist');
    }
}

This assumes the file /Config/default.config.php exists, albeit as an empty file.

So, let’s run those tests and see what happens?

PHPUnit 3.6.10 by Sebastian Bergmann.

...

Time: 0 seconds, Memory: 3.25Mb

OK (3 tests, 2 assertions)

Huzzah! That’s looking good. Notice that to handle a test of something which should throw an exception, you can either wrapper the function in a try/catch loop and, in the try side of the loop, have $this->assertTrue(false) to prevent false positives and in the catch side, do your $this->assertBlah() on the exception. Alternatively, (and much more simplely), use a documentation notation of @expectedException NameOfException and then prefix the function you are testing with the @ symbol. This is how I did it with the test “testFailLoadingConfig()”.

This obviously doesn’t handle setting and getting configuration values, so let’s add those.

Here’s the additions to the Config.php file:

    public function set($key = null, $value = null)
    {
        if ($key == null) {
            throw new BadFunctionCallException("Key not set");
        }
        if ($value == null) {
            unset ($this->arrValues[$key]);
            return true;
        } else {
            $this->arrValues[$key] = $value;
            return true;
        }
    }

    public function get($key = null)
    {
        if ($key == null) {
            throw new BadFunctionCallException("Key not set");
        }
        if (isset($this->arrValues[$key])) {
            return $this->arrValues[$key];
        } else {
            return null;
        }
    }

And the default.config.php file:

<?php
$this->set('demo', true);

And lastly, the changes to the ConfigTest.php file:

    public function testLoadConfig()
    {
        $config = new Config();
        $this->assertTrue(is_object($config));
        $config->load('default.config.php');
        $this->assertTrue($config->get('demo'));
    }

    /**
     * @expectedException BadFunctionCallException
     */
    public function testFailSettingValue()
    {
        $config = new Config();
        @$config->set();
    }

    /**
     * @expectedException BadFunctionCallException
     */
    public function testFailGettingValue()
    {
        $config = new Config();
        @$config->get();
    }

We’ve not actually finished testing this yet. Not sure how I can tell?

phpunit --coverage-text Tests/ConfigTest.php
PHPUnit 3.6.10 by Sebastian Bergmann.

....

Time: 0 seconds, Memory: 3.75Mb

OK (4 tests, 5 assertions)

Generating textual code coverage report, this may take a moment.

Code Coverage Report
  2012-05-08 18:54:16

 Summary:
  Classes: 0.00% (0/1)
  Methods: 0.00% (0/3)
  Lines:   76.19% (16/21)

@Config::Config
  Methods: 100.00% ( 3/ 3)   Lines:  76.19% ( 16/ 21)

Notice that there are 5 lines outstanding – probably around the unsetting values and using default values. If you use an IDE (like NetBeans) you can actually get the editor to show you, using coloured lines, exactly which lines you’ve not yet tested! Nice.

So, the last thing to talk about is Containers and Dependency Injection. We’ve already started with the Dependency Injection here – that $config->load(‘filename’); function handles loading config files, or you could just bypass that with $config->set(‘key’, ‘value); but once you get past a file or two, you might just end up with a lot of redundant re-loading of config files, or worse, lots of database connections open.

So, this is where Containers come in (something I horrifically failed to understand before).

Here’s a container:

class ConfigContainer
{
  protected static $config = null;

  public static function Load()
  {
    if (self::$config == null) {
      self::$config = new Config();
      self::$config->load();
    }
    return self::$Config;
  }
}

It’s purpose (in this case) is to load the config class, including any dependencies that you may need for that class, and then return that class to you. You could conceivably create a Database container, or a Request container or a User container with very little extra work, and with a few short calls, have a single function for each of your regular and routine sources of processing data, but without preventing you from being able to easily and repeatably test that data – by not going through the container.

Of course, there’s nothing to stop you just having these created in a registry class, or store them in a global from the get-go, but, I am calling these “Better Practices” after all, and these are considered to be not-so-good-practices.

Just as a note, code from this section can be seen at GitHub, if you want to use them at all.

Update 2012-05-11: Added detail to the try/catch exception catching as per frimkron’s comment. Thanks!

php-PDO-MySQL versus php-mysql

PHP and MySQL was the duo that took dynamic web development from /cgi/formmail.cgi to… well, where we are today for me. My first proper experience of working with a web application, as opposed to a static web site with a few perl scripts bolted on was PHPNuke, and I’ve not looked back since.

Almost the first lines of PHP you learn to code are

<?php
mysql_connect("localhost", "root", "");
mysql_use_db("mysite");

Followed closely by mysql_query(), then mysql_fetch_array(), mysl_num_rows(), and if the rest of us are lucky, mysql_real_escape_string().

So you see, for me, it’s really weird to find that even though this is what we’re teaching everyone to use, this weird incantation where we have to wrap every query up in a cotton wool ball of SQL injection protection… there’s an easier way where you don’t have to wrapper everything, which, even better, is more efficient at your SQL server and, if you’ve made the transition to using Classes in your code, will give you an object per-result rather than an array.

Let’s look at some examples of code, and see how we would transition things.

Here’s some rather unoriginal code to get the contents of a user table.

<?php
$db = mysql_connect("localhost", "root", "");
mysql_use_db("mysite");
$sql = "select userid, realname, lastloggedin from users "
     . "where username = '" . mysql_real_escape_string($_POST['username'])
     . "' and password = '" . mysql_real_escape_string($_POST['password']) . "'";
$query = mysql_query($sql);
$data = false;
if (mysql_num_rows($query) == 1) {
  $data = mysql_fetch_array($query);
  echo "Hello {$data['realname']}, your userid is {$data['userid']} and "
     . "you last logged in at " . date("H:i:s", strtotime($data['lastloggedin']))
     . " on " . date("d-M-Y" , strtotime($data['lastloggedin']));
}

There, now there’s not much wrong with that (ignoring the security hole of having your passwords in cleartext) is there? All pretty readable, right? Well… what happens if your database is down, how do you handle that in some kind of sensible way? You need to wrap some code around the mysql_connect to show it actually connected. What about if someone dropped the database by mistake, or you connected to the wrong database server and that table wasn’t there, now you’re wrappering the use_db line too. Did you make a typo in the SQL? Are you sure you didn’t miss a mysql_real_escape_string in there somewhere…

Here’s the same thing in PDO, protected against all (again, except the cleartext password) of those things.

<?php
try {
  $db = new PDO("mysql:host=localhost;dbname=mysite", "root", "");
  $db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
  $sql = "select userid, realname, lastloggedin from users where username = ? and password = ?";
  $query = $db->prepare($sql);
  $query->execute(array($_POST['username'], $_POST['password']));
  $data = $query->fetch();
  if ($data != false) {
    echo "Hello {$data['realname']}, your userid is {$data['userid']} and "
       . "you last logged in at " . date("H:i:s", strtotime($data['lastloggedin']))
       . " on " . date("d-M-Y" , strtotime($data['lastloggedin']));
  }
} catch (PDOException $e) {
  error_log("User unable to login: " . $e->getMessage());
}

So, let’s look at what transforms this into something much more interesting: $query->fetchObject();

<?php
try {
  $db = new PDO("mysql:host=localhost;dbname=mysite", "root", "");
  $db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
  $sql = "select userid, realname, lastloggedin, username, password " 
       . "from users where username = ? and password = ?";
  $query = $db->prepare($sql);
  $query->execute(array($_POST['username'], $_POST['password']));
  $data = $query->fetchObject('user');
  if ($data != false) {
    echo $data;
  }
} catch (PDOException $e) {
  error_log("User unable to login: " . $e->getMessage());
}

class user
{
  // Columns from the database
  protected $userid = null;
  protected $realname = null;
  protected $lastloggedin = null;
  protected $username = null;
  protected $password = null;
  // Processed Data
  protected $transformed_lastloggedin = null

  public function __construct()
  {
    if ($this->lastloggedin != null) {
      $this->transformed_lastloggedin = date("H:i:s", strtotime($this->lastloggedin)) 
                             . " on " . date("d-M-Y" , strtotime($this->lastloggedin);
    }
  }

  public function toString()
  {
    echo "Hello {$this->realname}, your userid is {$this->userid} and " 
       . "you last logged in at {$this->transformed_lastloggedin}";
  }
}

I’m starting to stretch the analogy here a little, but the important things here are:

  1. The __construct() function of the class is run after the data is inserted into it, meaning you can post-process any data you receive from the database without needing any new functions. Arguably, you could make the construct function accept the same values you’d receive from the database, and process those in the same way.
  2. The toString() function means you can mangle the content you want to output, without having to apparently run another function on the class you’ve just created.
  3. By using classes you get to do wonderful things like Unit Testing (this is a topic to follow), have automatic code documentation (err… to follow), reduce code duplication by extending classes and check on that, using “Copy and Paste Detection”.

Logitech Media Server vs Ubuntu 12.04

A while back I upgraded my home server to Ubuntu 12.04 (while it was still in beta) and immediately the first thing I noticed was that the Logitech Media Server (previously known as Squeezebox Server) had stopped working.

Checking through the logs, I saw a lot of messages about perl dependencies being missing or not working [1]. As I was a bit busy at the time (the decision to upgrade had been due to something else entirely), I put it to one side (much to my wife’s annoyance!) to pick up later.

As I’ve been wallowing at home the past couple of days with a stomach bug, and not really been fit to do much other than moan, lie there and feel sorry about myself, I thought about what I could do to get my squeezebox server back up and running.

A few Google searches later, and I turn up this page: http://forums.slimdevices.com/archive/index.php/t-89057.html which suggests that this message below is due to Ampache… which now that I look at the log entry, it kinda makes sense. Queue digging into the depths of the server.

Under Ubuntu, all the serious configuration for the server is stored in /var/lib/squeezeboxserver, which includes the plugins.

So, firstly, I deleted the downloaded Zip file from /var/lib/squeezeboxserver/cache/DownloadedPlugins/Ampache.zip

rm /var/lib/squeezeboxserver/cache/DownloadedPlugins/Ampache.zip

Next, I removed the unpacked plugin from /var/lib/squeezeboxserver/cache/InstalledPlugins/Ampache

rm -Rf /var/lib/squeezeboxserver/cache/InstalledPlugins/Ampache

I made a mistake here on my box, and restarted the server. Woohoo it came back up, but the first thing it did was to re-download the plugin again! D’oh. So, now I need to find what’s telling it to re-install the plugin. Queue a quick grep. Ahhh, there’s a file called extensions.prefs, which says:

prefs/plugin/extensions.prefs:  Ampache: 1

And another file called state.prefs which says:

prefs/plugin/state.prefs:Ampache: needs-install

Note, these are both the output from grep – so the filename includes the path from the point /var/lib/squeezeboxserver. A quick nano away (or whatever editor you prefer) and I’d removed the line from the extensions.prefs which showed Ampache: 1, but the state.prefs was marginally more tricky. In here it lists them in three states, enabled, disabled and needs-install. So, I changed it to show disabled and then restarted the service. Tada. I’ve got a working Logitech Media Server again! Huzzah!

[1] Log file looks like this:

Slim::bootstrap::tryModuleLoad (285) Warning: Module [Plugins::GrabPlaylist::Plugin] failed to load:
Can't locate Math/VecStat.pm in @INC (@INC contains: /usr/sbin/Plugins/Gallery /var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/Gallery /var/lib/squeezeboxserver/cache/Installed
Plugins/Plugins/CustomScan/lib /var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/Ampache/lib CODE(0xb911c40) /var/lib/squeezeboxserver/cache/InstalledPlugins /usr/share/squeezebo
xserver/CPAN/arch/5.14/i386-linux-thread-multi-64int /usr/share/squeezeboxserver/CPAN/arch/5.14/i386-linux-thread-multi-64int/auto /usr/share/squeezeboxserver/CPAN/arch/5.14.2/i686-linu
x-gnu-thread-multi-64int /usr/share/squeezeboxserver/CPAN/arch/5.14.2/i686-linux-gnu-thread-multi-64int/auto /usr/share/squeezeboxserver/CPAN/arch/5.14/i686-linux-gnu-thread-multi-64int
 /usr/share/squeezeboxserver/CPAN/arch/5.14/i686-linux-gnu-thread-multi-64int/auto /usr/share/squeezeboxserver/CPAN/arch/i686-linux-gnu-thread-multi-64int /usr/share/squeezeboxserver/li
b /usr/share/squeezeboxserver/CPAN /usr/share/squeezeboxserver /usr/sbin /etc/perl /usr/local/lib/perl/5.14.2 /usr/local/share/perl/5.14.2 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/
5.14 /usr/share/perl/5.14 /usr/local/lib/site_perl . CODE(0xb911e20)) at Slim/Player/Player.pm line 18.
BEGIN failed--compilation aborted at Slim/Player/Player.pm line 18.
Compilation failed in require at /var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/GrabPlaylist/Plugin.pm line 22.
BEGIN failed--compilation aborted at /var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/GrabPlaylist/Plugin.pm line 22.
Compilation failed in require at (eval 924) line 2.
BEGIN failed--compilation aborted at (eval 924) line 2.

Nice, right?

Installing MOTP-AS under Ubuntu 11.10

Please note, I am having issues with localhost authentication. See below

MOTP-AS is a simple installable two-factor authentication system using the mOTP algorythm for generating one-time passwords. MOTP-AS integrates with FreeRadius to provide the same authentication to log in to managed servers in a consistent manner.

I’ve recently installed this on my Ubuntu 11.10 laptop and on my Ubuntu 12.04 Beta server, and the installation instructions worked on both, so I thought I’d share them with you.

Installing appropriate packages

sudo apt-get install libpam-radius-auth freeradius mysql-server phpmyadmin

Alternatively, use tasksel to install the LAMP server task, then

sudo apt-get install libpam-radius-auth freeradius

Download the latest version of motp-as from http://motp-as.network-cube.de/index.php/download/current-version

Unpack it.

tar xfz ~/Downloads/motp-as*

Setting up the database

Go into the Setup/MySQL directory of the MOTP-AS directory. Edit motp_schema.sql at the line “CREATE USER”. Change the password from motp to something more secure.

mysql -u root -p < motp_schema.sql

Now update Setup/config.php with the new password you just created.

Setting up the web site

Copy the HTML directory to /var/www/motp (or somewhere else in your web root). You may need to do this either as root, or as a user with permissions to write to /var/www

cp -Rf ~/MOTP-AS_*/HTML /var/www/motp

Note this must be done after you’ve made your changes to Setup/config.php

Setting up FreeRadius

Stop the FreeRadius service

sudo /etc/init.d/freeradius stop

Users

Backup the users file

sudo mv /etc/freeradius/users /etc/freeradius/users.dist

Edit the users file you’re about to copy in

nano ~/MOTP-AS_*/Setup/Freeradius/users

Find the part where it says “/var/www/htdocs/radius-auth.php” and change that to “/var/www/motp/radius-auth.php

Copy in the new users file

sudo cp ~/MOTP-AS_*/Setup/Freeradius/users /etc/freeradius/users

Dynamic Clients

Backup the dynamic-clients file

sudo mv /etc/freeradius/sites-available/dynamic-clients /etc/freeradius/sites-available/dynamic-clients.dist

Edit the new dynamic-clients file

nano ~/MOTP-AS_*/Setup/Freeradius/dynamic-clients

Find the three lines saying “/var/www/htdocs” and replace that string with “/var/www/motp” (I use Ctrl+W, Ctrl+R in nano to do a replace-all.)

Copy in the new dynamic-clients file

sudo cp ~/MOTP-AS_*/Setup/Freeradius/dynamic-clients /etc/freeradius/sites-available/dynamic-clients

Then make that function available

sudo ln -s /etc/freeradius/sites-available/dynamic-clients /etc/freeradius/sites-enabled/dynamic-clients

Accounting

Amend the default script to enable accounting

sudo cp /etc/freeradius/sites-available/default /etc/freeradius/sites-available/default.dist

Then edit it to use the MOTP accounting functions

sudo nano /etc/freeradius/sites-available/default

Search for the line “accounting {” then comment that whole block out with the hash/pound sign “#“. Fortunately in the distribution supplied default file, this only means commenting out a few lines, which are “detail“, “unix“, “radutmp“, “exec“, “attr_filter.accounting_response“, and then the closing “}” for that block.

If you’re using nano, press the insert key (or Ctrl+R if you can’t find that easily) and enter /home/MyUserName/MOTP-AS_v0.7.2/Setup/Freeradius/accounting (amend the path as appropriate). Replace the section “/var/www/htdocs” with “/var/www/motp“.

Save and exit

Finishing off FreeRadius

sudo /etc/init.d/freeradius start

Install your client

Personally, I have an Android device, and I chose to install the Mobile-OTP app from the Android Marketplace. I also, through work, have a Nokia 6303i Classic, on which I installed the MOTP application from the MOTP site.

I’ve heard good things about iOTP for iPhone, although I personally don’t have one.

Configuring MOTP

Go to http://localhost/motp (or https://yourdomain.com/motp)

Login with the username admin and password of motp.

Securing the admin account

Click on the red text in “First time configuration

Click on “Change password of User ‘admin’

Enter a new password. Do not set the time or uses section of this page. Click “Set“. Ignore the warning.

Click on “Home

Setting up your first user

Click on “Quick Add” (under “Wizards”)

Enter a username. It should be the username for your Ubuntu 11.10 device.

On the client, create a profile for the device. Most of them create a profile by asking for a seed, rather than a secret, so those will likely be more than 16 characters long – maybe even 20 (Mobile-OTP for Android) or 25 (MOTP Java app).

Once you’ve got your secret (on Mobile-OTP, by pushing-and-holding on the profile name and selecting “Show Secret“, on MOTP Java app, once you’ve put 0000 as the PIN for the first time to initialize it, you get a string “Init-Secret:“), put that into the “Secret” field, and then ask the user to set their pin here – I suggest 1234 initially, as the user can change it to something they want after.

Click OK, then click “Logout” and test authentication. If it all goes OK, they should be presented with “Welcome to the Mobile OTP Authentication Server“.

Under “Settings” they can change their own PIN.

Testing radius authentication works OK

Run the radius testing program, like this, as a user:

radtest username passcode localhost 0 testing123

(This assumes the default localhost password hasn’t changed)

If you get anything like “rad_recv: Access-Reject packet from host“, then you’ve failed to configure something properly, or you’ve entered the PIN or code wrong.

Restart FreeRadius in debugging mode by doing the following:

/etc/init.d/freeradius stop
/usr/sbin/freeradius -X

This will produce a large quantity of logs on-screen, so I’d suggest running the test itself from a separate window. Run the radtest command (listed above) again. Look for your error messages. In my case, I forgot to update the line in users, so I saw this error message: Could not open input file: /var/www/htdocs/radius-auth.php

To find where this fault was, I did (as root, in /etc/freeradius)

find -R 'htdocs' /etc/freeradius

And got back: users: Exec-Program-Wait = “/usr/bin/php /var/www/htdocs/radius-auth.php %{User-Name} %{User-Password} %{Client-Shortname}”

That told me the fault was in the users file.

Fix the issue, check it again, and when you get this message “rad_recv: Access-Accept packet from host” press Ctrl+C to cancel the test mode of FreeRadius, and then run:

sudo /etc/init.d/freeradius start

Configuring pam_radius_auth.conf

Edit /etc/pam_radius_auth.conf

sudo nano /etc/pam_radius_auth.conf

Find the line which says “127.0.0.1” and replace the shared secret with something you want your server to use. You will also need to amend /etc/freeradius/clients.conf and replace the “secret” in the localhost client there (by default, it’s “testing123” in freeradius).

If you want to use your OTP for all authentication credentials, edit /etc/pam.d/common-auth, or if you just want to use it with specific access protocols, edit the relevant file in /etc/pam.d for the authentication systems you want to use OTP for.

You need to add the following line – either on the line before “@include common-auth” (for non common-auth files) or after the primary comment block for common-auth.

auth sufficient pam_radius_auth.so

Open a separate terminal session to your box (especially! if you’re remote) and ensure you can still login with your regular credentials.

Then try a connection with your radius credentials. It should just work! If not, stop the freeradius server and re-run it using /usr/sbin/freeradius -X and see whether you’re getting a different error message.

** UPDATE **

I have noticed that I’m getting locked out when using my non-radius credentials. This is probably due to the placement of the line in the /etc/pam.d/common-auth – it should probably come after the pam_unix.so line, but I’ve not tested that yet. I’m also going to try to suggest that there be an optional time-out period on locked accounts to the developers of MOTP-AS.

The second issue I’m struggling with is that I’m getting errors when using the LightDM. I’m getting the following error message in /var/log/auth.log:

pam_succeed_if(lightdm:auth): requirement "user ingroup nopasswdlogin" not met by user "spriggsj"

I don’t know if this is because I’m using ecryptfs as well, or because there’s something wonky going on with the common-auth structure I’m using.

Transfer my files using SFTP and SCP only?

A colleague today asked for some guidance around setting up an SFTP and SCP only account on a RedHat based Linux machine.

I sent him a collection of links, including one to the CopSSH project, and he implemented the code on that link, but then struggled when it didn’t work.

Aside from the fact the shell wasn’t copied into /etc/shells (which wasn’t disastrous, but did mean we couldn’t reuse it again later), it was still returning an error on each load.

Doing some digging into it, and running some debugging, I noticed that pscp (the PuTTY SCP) tool uses the SFTP subsystem rather than the SCP command to upload files, so we need to also check that the SFTP server hasn’t been called, instead of the SCP command, and also the SCP command needs to be corrected.

Here follows a script, complete with comments. Personally, I’d save this in /bin/sftponly, created and owned by root, and set to permissions 755 (rwxr-xr-x). Then, set the shell to this for each user which needs to do SFTP or SCP only.

#!/bin/bash
# Based on code from http://www.itefix.no/i2/node/12366
# Amended by Jon Spriggs (jon@sprig.gs)
# Last update at 2011-09-16

# Push the whole received command into a variable
tests=`echo $*`

# Set up a state handler as false
isvalid=0

# Test for the SFTP handler.
# The 0:36 values are the start character and length of the handler string.
if [ "${tests:0:36}" == "-c /usr/libexec/openssh/sftp-server" ]; then
  # Set the state handler to true
  isvalid=1
  # Configure the handling service
  use=/usr/libexec/openssh/sftp-server
fi

# Test for the SCP handler.
if [ "${tests:0:6}" == "-c scp" ]; then
  # Set the state handler to true
  isvalid=1
  # Configure the handling service
  use=/usr/bin/scp
fi

# If the state handler is set to false (0), exit with an error message.
if [ "$isvalid" == "0" ]; then
  echo "SCP only!"
  exit 1
fi

# Run the handler
exec $use $*

Quick Fix for Apache CVE-2011-3192 – Ubuntu/Debian

Please note – Apache have released a fix to this issue, and as such the below guidance has now been superseded by their fix.

I have been aware of the Apache web server issue for the last few days, where an overly wide range is requested from the server, leading to a crash in the server. As a patch hasn’t yet been released by Apache, people are coding their own solutions, and one such solution was found at edwiget.name.

That fix was for CentOS based Linux distributions, so this re-write covers how to do the same fix under Debian based distributions.
Read More

VNC in Ubuntu 11.04 with Unity

I recently bought myself a new laptop. Sometimes though, I want to check something on it on a rare occasion when I’ve not taken it with me. In comes VNC. Under Ubuntu 11.04, turning on VNC support is pretty straight forward.

To turn on VNC, go to the power icon in the top right corner (I think they call it the “Session Menu”, but it looks like a power button to me) and select “System Settings”. Under the “Internet and Network” heading, is an option called “Remote Desktop”. Click on that. Tick the top two boxes “Allow other users to view your desktop” and “Allow other users to control your desktop”. Tick the box “Require the user to enter this password” (and enter a password) and “Configure network automatically to accept connections”. Untick “You must confirm each access to this machine” and select “Only display an icon when there is someone connected”. Close it.

Now, try connecting to your device, and see what happens. I had some issues with Compiz elements not rendering correctly, and found a few hints to fix it. The first says to turn on the “disable_xdamage” option. It says to use gconf-editor, but I’m SSHing in, so I need to use gconftool-2 as follows:

gconftool-2 --set "/desktop/gnome/remote_access/disable_xdamage" --type boolean "true"

Personally, I only want to ever connect over OpenVPN to this, so I added the following:

gconftool-2 --set "/desktop/gnome/remote_access/network_interface" --type string "tun0"

You may wish to only ever access it over SSH, in which case replace “tun0” with “lo”

Now, I next made a big mistake. I followed some duff guidance, and ended up killing my vino server (I’m still not sure if I was supposed to do this or not), but to get it back, I followed this instruction to restart it. I had to tweak it a little:

sudo x11vnc -rfbport 5901 -auth guess

Once you’ve started this, tunnel an extra port (5901) to your machine, start VNC to the tunnelled port, and then go back through the options above. Exit your VNC session to the new tunnelled port, and then hit Control+C on the SSH session to close that x11vnc service.

A tip for users who SSH to a system running ecryptfs and byobu

I’ve been an Ubuntu User for a while (on and off), and a few versions back, Ubuntu added two great installed-by-default options (both of which are turned off by default), called Byobu (a Pimp-My-GnuScreen app) and ECryptFS (an “Encrypt my home directory” extension).

Until just recently, if you wanted to enable both, and then SSH to the box using public/private keys, it would use the fact you’d connected and authenticated with keys to unlock the ECryptFS module and then start Byobu. A few months back, I noticed that if I rebooted, it wouldn’t automatically unlock the ECryptFS module, so I’d be stuck without either having started. A few login attempts later, and it was all sorted, but just recently, this has got worse, and now every SSH session leaves me at a box with an unmounted ECryptFS module and no Byobu.

So, how does one fix such a pain? With a .profile file of course :)

SSH in, and before you unlock your ECryptFS module run this:

sudo nano .profile

You need to run the above using sudo, as the directory you access before you start ECryptFS is owned by root, and you have no permissions to write to it.

In that editor, paste this text.

#! /bin/bash
`which ecryptfs-mount-private`
cd
`which byobu-launcher`

Then use Ctrl+X to exit the editor and save the file.

The next time you log in, it’ll ask you for your passphrase to unlock the ECryptFS module. Once that’s in, it’ll start Byobu. Job’s a good’n.

Watching for file changes on a shared linux web server

$NEWPROJECT has a script which runs daily to produce a file which will be available for download, but aside from that one expected daily task, there shouldn’t be any unexpected changes to the content on the website.

As I’m hosting this on a shared webhost, I can’t install Tripwire or anything like that, and to be honest, for what I’m using it for, I probably don’t need it. So, instead, I wrote my own really simple file change monitor which runs as a CronJob.

Here’s the code:

#! /bin/bash
# This file is called scan.sh
function sha512sum_files() {
find $HOME/$DIR/* -type f -exec sha512sum '{}' \; >> $SCAN_ROOT/current_status
}
SCAN_ROOT=$HOME/scan
mv $SCAN_ROOT/current_status $SCAN_ROOT/old_status
for DIR in site_root media/[A-Za-z]*
do
sha512sum_files
done
diff -U 0 $SCAN_ROOT/old_status $SCAN_ROOT/current_status

And here’s my crontab:


MAILTO="my.email@add.ress"
# Minute Hour Day of Month Month Day of Week Command
# (0-59) (0-23) (1-31) (1-12 or Jan-Dec) (0-6 or Sun-Sat)
0,15,30,45 * * * * /home/siteuser/scan/scan.sh

And lastly, a sample of the output

--- /home/siteuser/scan/old_status 2010-10-25 14:30:03.000000000 -0700
+++ /home/siteuser/scan/current_status 2010-10-25 14:45:06.000000000 -0700
@@ -4 +4 @@
-baeb2692403619398b44a510e8ca0d49db717d1ff7e08bf1e210c260e04630606e9be2a3aa80f7db3d451e754e189d4578ec7b87db65e6729697c735713ee5ed /home/siteuser/site_root/LIBRARIES/library.php
+c4d739b3e0a778009e0d53315085d75cf8380ac431667c31b23e4b24d4db273dfc98ffad6842a1e5f59d6ea84c33ecc73bed1437e6105475fefd3f3a966de118 /home/siteuser/site_root/LIBRARIES/library.php
@@ -71 +71 @@
-88ddd746d70073183c291fa7da747d7318caa697ace37911db55afce707cd1634f213f340bb4870f1194c48292f846adaf006ad61b4ff1cb245972c26962b42d /home/siteuser/site_root/api.php
+d79e8a6e6c3db39e07c22e7b7485050007fd265ad7e9bdda728866f65638a8aa534f8cb51121a68e9287f384e8694a968b48d840d37bcd805c117ff871e7c618 /home/siteuser/site_root/api.php

While this isn’t the most technically sound way (I’m sure) of checking for file changes, at least it gives me some idea (to within 15 minutes or so) of what files have been changed, so gives me a time to start hunting.

Weirdness with Bash functions and Curl

I’m writing a script (for $NEW_PROJECT) which, due to my inability to figure out how to compile a certain key library on Dreamhost, runs SSH to a remote box (with public/private keys and a limitation on what that key can *actually* achieve) to perform an off-box process of some data.

After it’s all done, I am using curl to call the API of the project like this:

curl --fail -F "file=@`pwd`/file" -F "other=form" -F "options=are_set" http://user:password@server/api/function

Because I’m making a few calls against the API, I wrote a function like this:

function callAPI() {
API=$1
if [ "$2" != "" ]
then
API=$API/$2
fi
if [ "$3" != "" ]
then
API=$API/$3
fi
if [ "${OPTION}" != "" ]
then
FORM="${OPTION}"
else
FORM=""
fi
if [ $DEBUG == "1" ]
then
echo "curl --fail ${FORM} http://${USER}:**********@${SITE}/api/${API}"
fi
eval `curl --fail ${FORM} http://${USER}:${PASS}@${SITE}/api/${API} 2>/dev/null`
}

and then call it like this:

OPTION="-F \"file=@filename\" -F \"value=one\" -F \"value=two\""
callAPI function

For all the rest of my API calls (those which ask for data, rather than supply it, these calls work *fine*, but as soon as I tell it to post a form containing a file, it throws this error:

curl: (26) failed creating form post data

I did some digging around, and found that this means that the script can’t read from the file. The debug line, when run outside of the script processed the command perfectly, so what’s going on?

To be honest, in the end, I just copied the command into the body of the code, and I’m praying that I can figure out why I can’t compile this library on Dreamhost, before I need to work out why running that curl line doesn’t work from inside a function.