A summary of my ongoing Open Source projects

I’m a pretty frequent contributor to various Open Source projects, either when I’m starting them myself, or getting involved in someone else’s project. I thought, as I’m probably stretching myself a bit thin with these projects right now, I’d list off what I’m doing, so I can find out whether anyone’s interested in getting involved in any of them. Read More

Need to quickly integrate some IRC into your app? Running Linux? Try ii

I know, it looks like a typo, but the script ii makes IRC all better for small applications which don’t need their own re-implementation of an IRC client.

I know it’s available under Ubuntu and Debian (apt-get install ii), but I don’t know what other platforms it’s available for.

It’s not much use as a user-focused IRC client (although it would vaguely work like that with a little scripting!), but for scripts it works like a charm.

Read More

Some notes on OpenSSH

At the hackspace recently, I was asked for a brief rundown of what SSH can do, and how to do it.

Just as an aside, for one-off connections to hosts, you probably don’t need to use a public/private key pair, but for regular access, it’s probably best to have a key pair, if not per-host, then per-group of hosts (for example, home servers, work servers, friends machines, web servers, code repositories). We’ll see how to keep these straight later in this entry. For some reasons, you may want to have multiple keys for one host even!

If you want to create a public/private key pair, you run a very simple command. There are some tweaks you can make, but here’s the basic command


Generating public/private key pair
Enter the file in which to save the key (/home/bloggsf/.ssh/id_rsa): /home/bloggsf/.ssh/hostname
Enter passphrase (empty for no passphrase): A Very Complex Passphrase
Enter same passphrase again: A Very Complex Passphrase
Your identification has been saved in /home/bloggsf/.ssh/hostname.
Your public key has been saved in /home/bloggsf/.ssh/hostname.pub.
The key fingerprint is:
00:11:22:33:44:55:66:77:88:99:aa:bb:cc:dd:ee:ff bloggsf@ur-main-machine

See that wasn’t too hard was it? Transfer the PUBLIC portion (the .pub file) to your destination box, as securely as possible, whether that’s by SFTP, putting them on a pen drive and posting it to your remote server, or something else… but those .pub files should be appended to the end of /home/USERNAME/.ssh/authorized_keys

You achieve that by typing:

cat /path/to/file.pub >> /home/username/.ssh/authorized_keys

Note that, if you don’t spell it the American way (authoriZed), it’ll completely fail to work, and you’ll stress out!

So, now that key is on your remote host, how do we do stuff with it?

1) SSH to a console (this won’t try to use the public/private key pair, unless you left the default filename when you made your key)

ssh user@host

2) SSH to a host on an unusual port

ssh user@host -p 12345

3) SSH using a private key (see towards the end of the document about public and private keys)

ssh user@host -i /path/to/private_key

4) SSH on a new port and with a private key

ssh user@host -p 54321 -i /home/user/.ssh/private_key

5) Pulling a port (e.g. VNC service) back to your local machine

ssh user@host -L 5900:

The format of the portion starting -L is local-port:destination-host:destination-port.

Note, I would then connect to localhost on port 5900. If you are already running a VNC service on port 5900, you would make the first port number something not already in use – I’ll show an example of this next.

6) Pulling multiple ports from different remote hosts to your local machine.
This one I do for my aunt! It forwards the VNC service to a port I’m not using at home, and also gives me access to her router from her laptop.

ssh user@host -L 1443: -L 5901:localhost:5900

Here I’ve used two formats for selecting what host to forward the ports from – I’ve asked the SSH server to transfer connections I make to my port 1443 to the host on port 443. I’ve also asked it to transfer connections I make on port 5901 to the machine it resolves the name “localhost” as (probably – a virtual IP address signifying my local machine) and to it’s port 5901.

7) Reverse Port Forwarding… offering services from the client end to the server end.

ssh user@host -R 1080:localhost:80

I’ve identified here the most common reason you’ll do a reverse port forward – if you’re not permitted to run sftp (in case you transfer files out of the system), but you need to transfer a file to the target host. In that case, you’d run a web server on your local machine (port 80) and access the web server over port 1080 from your destination host.

8) Running a command instead of a shell on the remote host

ssh user@host run-my-very-complex-script –with-options

9) If you only want your user to be able to use a specific command when they SSH to your host, edit their authorized_keys file, and add at the beginning:

command=”/the/only/command/that/key/can/run $SSH_ORIGINAL_COMMAND” ssh-rsa ……

This command will be run instead of any commands they try to run, with the command they tried to run as options passed to it.

10) Make a file to make it easier for you to connect to lots of different machines without needing to remember all this lot!

The file I’m talking about is called config and is stored in /home/bloggsf/.ssh/config

If it’s not already there, create it and then start putting lines into it. Here’s what mine looks like (hosts and files changed to protect the innocent!)

Host home external.home.server.name
Hostname external.home.server.name
User jon
Port 12345
LocalForward 1080 localhost:1080
LocalForward 9080 router:80
LocalForward 9443 router:443
Host github github.com
Hostname github.com
User git
IdentityFile /home/jon/.ssh/github_key
Host main.projectsite.com
User auser
RemoteForward 1080:localhost:80
Host *.projectsite.com
User projectowner
IdentityFile /home/jon/.ssh/supersecretproject
Host *
IdentityFile /home/jon/.ssh/default_ssh_key
Compression yes

The config file parser steps through it from top to bottom, and will ignore any subsequent lines which it matches already (with the exception of LocalForward and RemoteForward), so if I try to SSH to a box, and my SSH key isn’t already specified, it’ll use the default_ssh_key. Likewise, it’ll always try and use compression when connecting to the remote server.

A warning about the evils of Facebook

Facebook is one of the current breed of “Social networking” websites – which means that they let you exchange information, pictures and videos with each other… sounds good so far, right?

Here’s where the problem is. Facebook is a company which is trying to make money. Your profile (the collection of all your information) on their website belongs to them. They can market that information to anyone and do whatever they want with it. If you put any pictures on there, then they own those photos too. On top of that, every “application” (or service that isn’t written by Facebook) knows everything about you and the people you are friends with… which means that if you’ve decided not to install an application that collects e-mail addresses, but your friend does – then that application knows your e-mail address. Wonderful!

Facebook have a real problem with their “privacy policy” and the pages which let you share details with the rest of the world – every few months they write a new version of both to help them get even more of a chance to sell off your information, to use your photos and videos in new and interesting ways… so much so, that about a year ago, their CEO (Chief Executive Officer – the person who makes all the day-to-day decisions about where the company goes next) had all his details shared publicly because he forgot that they started using the new privacy settings page on that day and he’d not set his details to the most private settings. This happens all the time – to the extent another website was created called http://youropenbook.org that shows what people are making publicly available!

A few months back, Facebook changed their privacy policy again to let you log into other websites using your Facebook details, which sounds like a great idea, but it means that the website then (again) knows your e-mail address, all your friends, your birthday and (if you enter it) your phone number… not good!

Realistically, it is possible to use Facebook in a vaguely safe way if you take a lot of precautions about what you are sharing and doing on their website, but I really wouldn’t recommend using it, and in fact, I’d recommend who ever suggested you use it be forwarded a link to this page, warning them not to use it! Sadly, there’s nothing else available right now that does the same thing in a way that still maintains your privacy. I’m watching a few projects, and once something safe and easy to use comes out, I’ll let you know!

(Just as a disclaimer, I do use Facebook, but I don’t like it and I want to move away from it, PRONTO!)

CampFireManager Workshop

NOTE: Images have been removed from this post 2017-05-02

What is CampFireManager?

CampFireManager is a tool to schedule talks at a Conference or Unconference (such as a barcamp). It is written in PHP and uses a MySQL backend to store the data about the event. It optionally can use SMS messaging and Microblogging services (like identi.ca or Twitter) to perform routine tasks on the system.

A modified version (CampFireManager-Lite) of CampFireManager was recently used at OggCamp ’10, in combination with tools created by Xibo. Here’s a look at what you would have seen if you were there:


This is the main timetable. Note, it only shows two talks before “now”, the talk on “now” and the next 6 slot’s worth of talks. This caused us problems on the Saturday afternoon when we tried to display the timetable for the next day. The only way we could do that was to change the system time on the server.

This is the now and next screen for the main stage, and a screen showing similar information for each of the stages. By default, each screen shows, in turn, the timetable, the “now and next” screen for all stages and then the “now and next” screen for each individual screen. This is entered by the database, but should *really* have been configured at the screen.

Administration was performed by organisers. Data was collected on sheets, and then entered into the system. Using an in-built version of the timetable, these were placed into the appropriate slots in the timetable, which then updated all the other screens.

This, however, isn’t how I originally envisaged CampFireManager. The full version of CampFireManager is designed to operate with minimal administrative overhead from on-site staff.


Users (as well as support staff and administrators) would log into the website using OpenID. Several common providers are pre-populated and available using icons to select them.

On logging in, all users initially see a timetable, with no definitions of rooms, but with the complete timetable for this day. At the top, under the “Slot” times, is a hyperlink showing “New Talk”. Clicking on this brings you to a page where you can enter your talk proposal. Clicking on “Go” inserts that talk into your timetable.

If you wanted to enter some contact details for your talk (so that your peers at the event can reach you), this is done by clicking on “Amend Contact Details”, and entering your details. This will retrospectively correct your details on all the talks you’re giving.

By clicking on “Add other access method” – you can add an “authorization code” (which is a 5 character string of case sensitive, non-similarly rendered letters and numbers) to allow you to update your talks, your contact details or the talks you’ll be attending from other communication methods – SMS or XMPP being the two primary examples.

However, I realised that I could use this access method logic to allow administrators the ability to grant administrative access to other organisers, or to promote staff to “Support” staff. In the same way you’d enter a 5 character string to give the same rights to your phone as your web access, you could enter a 9 character string (again, non-similarly rendered, case sensitive letters and numbers) to become an administrator or support staff member. Here you see an Administrator’s view of the timetable, which adds two extra options at the top box – Provide support to attendees and modify config values.

So, let’s support an attendee. We enter some details, either their Auth Code (if they know it), or some details about them they entered previously – a blog address, e-mail, some other contact method, or in extreme cases, their name. If we’ve had to use any form of wildcard in the search (that is, I typed “Jon” into the name box), then we get prompted to confirm it’s them, and can give them their unique Auth Code. If they’ve not registered on the site, then click on Create New Auth Code to create a new user. In either case, we get a page showing “This is with an AuthCode: ". From here we can amend their contact details, propose a new talk on their behalf, or if there's a talk in the future, show their attendance (which, to be fair, here I couldn't do, as I'd not created a talk for them to attend).

Here’s the administrative screen, where you see options that have configured the screens and access methods.

There’s a mobile interface to the site, which is quite similar to the support staff’s view of the site, and again it uses OpenID to authenticate. It gives the user access to a slimmed down portal – showing all talks which are yet to occur (allowing the user to show or remove their attendance), just this user’s talks yet to occur, and allowing the user to add a new talk and amend their contact details.

That is all the interactive screens shown, so here’s the two most regularly seen, non-interactive screens, both with a scrolling box between the event title and the content saying (in this instance):

Phones: 07 88 24 64 91 8 on the Three Network (with 18% signal)
Website: https://home.north.sprig.gs/CampFireManager/
Mobile site: https://home.north.sprig.gs/CampFireManager/m/
Event Hashtag: #oc10
Identify with this service by sending
I <your name> [email:your@email.address] [http://your.web.site]
(there are more options for identification by going to the website)
Propose a talk by sending P <Time Slot> <Slots Used> <Talk Title>
Cancel a talk by sending C <Talk Number> <Time Slot> [Reason]
Rename a talk by sending E <Talk Number> <Time Slot> <New Talk Title>
Attend a talk by sending A <Talk Number>
Decline the attendance of a talk by sending R <Talk Number>
Note: You can combine multiple A and R commands in one message.
Statements surrounded with <> are mandatory options, those statements surrounded with [] are optional.
These commands should be sent to your preferred mobile service listed above.

The Timetable:

And the “Direction” screens (like the now and next screens on the CampFireManager-Lite):

The above URLs do work, and the system is available for experimentation. The code and ticket tracker (if you want to run your own, local, instance) is at http://code.google.com/p/campfiremanager. Please contact me if you want to get access to my instance of CampFireManager from an administrative or support perspective. Please note, the SMS engine is not running.

Using the recursive_import.php script for importing photos to the #Horde module Ansel with subdirectories

I have a problem with the excellent Horde module “Ansel” – their photo
display and manipulation application – which I’m

If you have a lot of photos and you want to import the lot in one go,
there’s a script called recursive_import.php – you’ll find this under
/path/to/your/horde/install/ansel/scripts/recursive_import.php and it
takes the following arguments: -d /path/to/directory -u USERNAME -p

I’d been using it thinking it would handle directory navigation a bit
better than it did, by running it as follows:

php recursive_import.php -d import_dir -u fred -p bloggs

Infact, I needed to do it like this:

php recursive_import.php -d `pwd`/import_dir -u fred -p bloggs

This is because the script navigates up and down the directory
structure as it works out the contents of each directory, instead of
handling the referencing properly. I plan to look at this properly
tomorrow when I’ve got a day off, but if I don’t, or if the patch
doesn’t get accepted, at least you know how to fix it now! :)

Posted via email from Jon’s posterous

Use GMail’s SMTP gateway using the command line from Ubuntu without lots of config tips

I’m writing a few little scripts at the moment, and one of them needed to be able to send an e-mail. I’d not got around to sorting out what my SMTP gateway was from my ISP – but I do tend to use GMail’s SMTP gateway for non-essential stuff.

I thought I could easily setup sendmail, but no, that’s SCARY stuff, and then I thought of Postfix, but that needs an awful lot of configuration for an TLS based SMTP connection, so I did a bit of digging.

Thanks to this post over at the Ubuntu Forums, I worked out how to get a local port 10025 to run, but PHP kept complaining, so I next looked for a “sendmail replacement”, in comes nullmailer.

So, thankfully this is all rather easy.

  • sudo apt-get install openssl xinetd nullmailer
  • sudo tee /usr/bin/gmail-smtp <<EOF >/dev/null
    # Thanks to http://ubuntuforums.org/showthread.php?t=918335 for this install guide
    /usr/bin/openssl s_client -connect smtp.gmail.com:465 -quiet 2>/dev/null
    sudo chmod +x /usr/bin/gmail-smtp
  • sudo tee /etc/xinetd.d/gmail-smtp <<EOF >/dev/null
    # default: on
    # description: Gmail SMTP wrapper for clients without SSL support
    # Thanks to http://ubuntuforums.org/showthread.php?t=918335 for this install guide
    service gmail-smtp
        disable         = no
        bind            = localhost
        port            = 10025
        socket_type     = stream
        protocol        = tcp
        wait            = no
        user            = root
        server          = /usr/bin/gmail-smtp
        type            = unlisted
    sudo /etc/init.d/xinetd reload
  • sudo tee /etc/nullmailer/remotes <<EOF >/dev/null smtp --port=10025 --user=your@user.tld --pass=Y0urC0mp3xGM@ilP@ssw0rd
    sudo /etc/init.d/nullmailer reload

Setting all this lot up was pretty easy with these guides. There’s no reason why it wouldn’t work on any other version of Linux (provided you can install all these packages).

Good luck with your project!

Posted via web from Jon’s posterous

Locally Monitoring Interfaces on Nokia Firewalls (and – by a link – McAfee Sidewinders) for Failover

I recently wrote a document on http://jon.spriggs.org.uk/blog explaining how to monitor the interface of a McAfee sidewinder to see when it failed over. I don’t know why I didn’t write it on Posterous, but if you’re following me on Posterous, and you think that you might want to know how to use Perl to repeatedly loop over the same command, and show the results with a date stamp underneath it (a bit like the watch command) then you’ll find this page really useful. In the mean time, I’ve also written the same script for the CSH shell, which is used, amongst other places, on Nokia Firewalls.


One of our requirements with one of our customers is to perform regular and routine failover tests. As the interface is not responsive to providing information about when service has failed from Primary to Secondary and back again, I re-wrote the script I adjusted for McAfee Sidewinders to run on the SECONDARY NODE to show the interface address of one NIC every 5 seconds. I’ll also show how to slightly modify the script with different time delays and interface names. Please note, there may be much better ways of doing this. I needed something in a hurry, and this gave me what I needed. If you’ve got any better ideas, please drop me a note at jon@spriggs.org.uk or note below how to do it :)

Steps to perform

  1. SSH to the Secondary node.
  2. Check you’re not already primary with the command ifconfig eth-s1p1c0 | grep inet this should return one line showing something like inet mtu 1500 broadcast
  3. Type this
    while (-e /bin/csh)
    ifconfig eth-s1p1c0 | grep inet
    sleep 5
  4. Perform your action to provoke fail-over, which may be to unplug an interface attached to the primary firewall, reboot the firewall or unplug a switch directly attached to the firewall. In response (and after approx 1 minute, based on your HA configuration) you should now see in the script’s output, it now shows two lines (or maybe three) – as follows:

    inet mtu 1500
    inet broadcast
    inet broadcast
    vrrpmac 0:0:aa:bb:cc:dd
  5. Perform your failback and after 1 minute or so, it should revert to just the single line – or equivelent for your network.

In the bold section above, replace the interface name identified (here it’s eth-s1p1c0) with an interface you know will fail over, you can also make bigger or smaller the sleep command – here it’s 5 seconds, but there’s probably no reason why it couldn’t be 1 or 10.

Posted via web from Jon’s posterous

Supporting multiple machines in GNOME using VNC

I was recently asked how to configure VNC for user support across a series of machines running GNOME. I’m in the process of trying out a few different platforms at the moment, and didn’t have my GNOME machine to hand and working right, so I decided to work it out from what I’ve done in the past. Here’s the bulk of the e-mail I sent him to try and help him out. Maybe this will help you at some point.

If you find any errors (especially around the option names in the actual dialogue boxes) please post a note so I can correct this!


On most GNOME based systems (which includes Fedora), you can active “Remote Desktop Sharing” for users.

Go to System -> Preferences -> Remote Desktop Sharing (or something similar). I’m afraid I’ve just recently moved my systems to KDE, so I don’t know the exact options, but I believe it’ll say something like “Enable remote connections” (tick that), and “User is prompted to permit connection” (this will be down to policy) and “Remote user needs to enter a password” (this will need some text to be entered).

Once you have these for one system, you can automatically set this for all the other computers.

From the command line, type
  gconftool-2 -R /desktop/gnome/remote_access

This will return all the settings you have made. Here’s mine:

 view_only = false                                         
 alternative_port = 5900                                   
 prompt_enabled = false                                    
 icon_visibility = client                                  
 lock_screen_on_disconnect = false                         
 disable_xdamage = false                                   
 mailto =                                                  
 use_alternative_port = false                              
 enabled = true                                            
 disable_background = false                                
 network_interface =                                       
 require_encryption = false                                
 authentication_methods = [vnc]                            
 vnc_password = &&&&&&&&&&&&                               
 use_upnp = false

(I’ve removed the password for my box)

You can use this gconftool to set the same variables on your computers you’ve already deployed, either per-user, as a default policy for each machine, or as a mandatory policy for each machine.

This article from Sun’s GNOME configuration guide explains how to set variables: http://docs.sun.com/app/docs/doc/806-6878/6jfpqt2t5?a=view while this is an overview of all of the GNOME configuration tool (including that article): http://docs.sun.com/app/docs/doc/806-6878/6jfpqt2sv?a=view and lastly, this is how “Vino” the VNC client for GNOME works: http://www.gnome.org/~markmc/remote-desktop.html

I hope this helps you!

Posted via web from Jon’s posterous

Watching an interface on McAfee’s Sidewinder with Perl


One of our requirements with one of our customers is to perform regular and routine failover tests. As the interface is not responsive to providing information about when service has failed from Primary to Secondary and back again, I have found [1] and modified this perl script to run on the SECONDARY NODE to show the interface address of one NIC every 5 seconds. I’ll also show how to slightly modify the script with different time delays and interface names. Please note, there may be much better ways of doing this. I needed something in a hurry, and this gave me what I needed. If you’ve got any better ideas, please drop me a note at jon@spriggs.org.uk or update this script yourself :)

Steps to perform

  1. SSH to the Secondary node.
  2. Check you’re not already primary with the command ifconfig em0 | grep inet this should return one line showing something like inet netmask 0xffffff00 broadcast
  3. Please note the exact syntax of this command – perl is a tricky beast at the best of times, and if you don’t have your apostrophies and backticks in the right place, it won’t run right!

    Type this command: perl -e ‘for (;1;) {print `ifconfig em0 | grep inet`; print `date`; sleep 5}’ and press return. This loops until you press Ctrl+C, showing the line, like I showed you before starting inet and then the next line shows a date and time – this is so you don’t go crazy and think the process has stopped…

  4. Perform your action to provoke fail-over [2], which may be to unplug an interface attached to the primary firewall, reboot the firewall or unplug a switch directly attached to the firewall. In response (and after approx 1 minute, based on your HA configuration) you should now see in the script’s output, it now shows two lines – as follows:

    inet netmask 0xffffff00 broadcast
    net netmask 0xffffff00 broadcast

  5. Perform your failback and after 1 minute or so, it should revert to just the single line – or equivelent for your network.

Breaking down the script

Here, I’ve broken down the command before with short blocks of information about each part of the command you’re running.

perl (which is a scripting language) -e (which means “evaluate the following string in quotes”) (that’s an apostrophy – the symbol on my keyboard at least, below the @ character) for (;1;) (note, those are normal brackets – shift+9 and shift+0, which, in combination with the for and ;1; bits, means loop around the following commands forever, or until the break command is sent) { (that’s a curled brace symbol – shift and the character next to the letter P) print (sends the following string) ` (backtick – the character to the left of the 1 key – which means execute the command between two of these symbols and return the result) ifconfig em0 (get the interface configuration for the interface em0) | (the pipe or bar character – next to the letter Z) grep inet (grep is a unix command to search for strings – in this case, the string inet which identifies the IP address) `; (backtick again and then a semicolon to say stop running that command and start doing the next thing) print `date`; (print the response from the command “date” – which returns a date string – and then do the next command) sleep 5 (the sleep command waits integer X number of seconds – here 5, but you could easily say 2 or 10 here – depends on how impatient you and the project manager are!) } (curly brace – meaning to close the for loop from near the beginning) (apostropy – which instructs it to close the string that the perl interpreter is evaluating)


[1] from http://stackoverflow.com/questions/555116/repeat-a-unix-command-every-x-seconds-forever – sadly, we can’t use the watch command on a Sidewinder, as it doesn’t have it installed.
[2] Assuming your firewall is in a Primary/Standby configuration (not Standby/Standby), has enabled “Monitor link status” and has addresses to monitor with – presumably the non-HSRP addresses of your attached routers attached to that interface.