Memoirs of a Roadie

The Green Manalishi (With the Two-Prong Crown)

Posted on 9th May 2009

For quite sometime now, I like others I've seen on various forums, have got really irritated with one aspect of the colour scheme used on Linux when doing a directory listing. If you have the colour scheme enabled and haven't changed any settings, then likely as not, if you have any directory that is world writeable, then you probably won't be able to read the text.

When designing websites, it is common practice to use high contrast colours, as you want the site to be readable, and not cause headaches among your readers as they try to understand what you've written. Recently I was investigating some high contrasting colours, and found some good websites that explain what I mean.

With this in mind, why on earth would anyone think that blue text on a dark green background would be easily readable. Unless you get close to the screen and squint!

On Linux systems there is a application called 'dircolors' and if you're lucky enough on RedHat based systems, you even have the '/etc/DIR_COLORS' configuration file, so it is possible to try and figure out what needs to be changed. However, until now I've never manaed to find the offending attribute to change. After a lot of searching this morning, I finally found a site that explains all the short form and long form keys, and additional colours that can be used in the colour scheme, beyond the 8 foreground and 8 background colours. Even better it even explains the effects available. In most places only bold is mentioned.

So in the event anyone else has had the same problem with their directories disappearing before their eyes, take a look at Configuring LS_COLORS by David Newcomb. It turns out the offending key is OTHER_WRITABLE (or ow in LS_COLORS format). I've now set mine to something much more sensible (bold white on a blue background), and it's much much easier on the eyes :)

File Under: design / linux / usability NO COMMENTS

April Skies

Posted on 1st May 2009

For those that might not be aware, I got made redundant on 31st March (the day after the QA Hackathon had finished). Thankfully, I start a new job next week, so I've managed to land on my feet. However, this has meant that I've ended up having the whole of April off to do stuff. My plan was to work on some of the Open Source projects that I'm involved with to move them further along to where I wanted them to be. As it turned out two specific projects got my attention over the last 4 weeks, and I thought it worth giving a summary of what has been going on.

YAPC Conference Surveys

Since 2006, I've been running the conference surveys for YAPC::Europe. The results have been quite interesting and hopefully have help organisers improve the conferences each year. For 2009 I had already planned to run the survey for YAPC::Europe in Lisbon, but this year will also see YAPC::NA in Pittsburgh having a survey of their own.

The survey site for Copenhagen in 2008 added the ability to give feedback to Master Classes and talks. The Master Classes feedback was a little more involved, as I was able to get the attendee list, but the talks feedback was quite brief. As such, I wanted to try and expand on this aspect and generally improve the process of running the surveys. Part of this involved contacting Eric and BooK to see if ACT had an API I could use to automate some of the information. I was delighted to get an email back from Eric, who very quickly incorporated an API that I could use, to retrieve the necessary data to keep the survey site for a particular conference up to date, even during the conference.

With the API and updates done, it was time to focus on expanding the surveys and skinning the websites to match that of the now live conference sites. The latter was relatively easy, and only required a few minor edits to the CSS to get them to work with the survey site. The survey site now has 3 types of survey available, though only 2 are visible to anyone not taking a Master Class. Those that have taken one of the YAPC::Europe surveys will be aware I don't use logins, but a key code to access the survey. This has been extended so that it can now be used to access your portion of the survey website. This can now be automatically emailed to attendees before the conference, and during if they pay on the door, and will allow everyone to feedback on talks during the conference. On the last day of the conference the main survey will be put live, so you can then answer questions relating to your conference experience.

I'm hoping the slight change won't be too confusing, and that we'll see some ever greater returns for the main survey. Once it does go live, I'd be delighted to receive feedback on the survey site, so I can improve it for the future.

CPAN Testers Reports

Since taking over the CPAN Testers Reports site in June 2008, I have spent a great deal of time improving it's usability for users. However, it's come at a price. By using more and more Javascript to dynamically change the contents of the core pages, it's meant that I have received a number of complaints that the site doesn't work for those with Javascript disabled or who use a browser that doesn't implement Javascript. For this reason I had decided that I should create a dynamic site and static site. The problem with this is that the current system to create all the files takes several hours for each set of updates (currently about 16 hours per day). I needed a way to drive the site without worrying about how long everything was taking, but also add some form of prioritisation so that the more frequently requested pages would get updated more quickly than those rarely seen.

During April, JJ and I went along to the Milton Keynes Perl Mongers technical meeting. One of the talks was about memcached and it got me thinking as to whether I could use it for the Reports site. Discussing this with JJ on the way home, we threw a few ideas around and settled on a queuing system to decide what needed updating, and to better managed the current databases to add indexes to speed up some of the complex lookups. I was still planning to use caching, but as it turned out memcached wasn't really the right way forward.

The problem with caching is that when there is too much stuff in the cache, the older stuff gets dumped. But what if the oldest item to get dumped is extremely costly on the database, and although it might not get hit very often, it's frequent enough to be worth keeping in the cache permanently. It's possible this could be engineered with memcached if this was for a handful of pages, but for the Reports site it's true for quite a few pages. So I hit on a slightly different concept of caching. As the backend builder process is creating all these static files, part of the process involves grabbing the necessary data to display the basic page, with the reports then being read in via the now static Javascript file for that page. Before dropping all the information and going on to the next in the list, the backend can simply write the data to the database. The dynamic site can then simply grab that data and display the page pretty quickly, saving ALOT of database lookups. Add to the fact that the database tables have been made more accessible to each other, the connection overhead has also been reduced considerably.

The queuing system I've implemented is extremely simple. On grabbing the data from the cache, the dynamic site checks quickly to see if there is a more recent report in existence. If there is, then a entry is added to the queue, with a high weighting to indicate that a website user is actually interested in that data. Behind the scenes the regular update system simply adds an entry in the queue to indicate that a new entry is available, but at a low weighting. The backend builder process then looks to build the entries with the most and highest weightings and builds all the static files, both for the dynamic site and the static site, including all the RSS, YAML and JSON files. It seems to work well on the test system, but the live site will be where it really gets put through its paces.

So you could be forgiven in thinking that's it, the new site is ready to go. Well not quite. Another part of the plan had always been to redesign the website. Leon had designed the site based on the YUI layouts, and while it works for the most part, there are some pages which don't fit well in that style. It also has been pretty much the same kind of style since it was first launched, and I had been feeling for a while that it needed a lick of paint. Following Adam's blog post recently about the state of Perl websites, I decided that following the functional changes, the site would get a redesign. It's not perhaps as revolutionary as some would want, judging from some of the ideas for skins I've seen, but then the site just needs to look professional, not state of the art. I think I've managed that.

The work to fit all the pieces together and ensure all the templates are correct is still ongoing, but I'm hopeful that at some point during May, I'll be able to launch the new look websites on the world.

So that's what I've been up to. I had hoped to work on Maisha, my other CPAN distributions, the YAPC Conference Survey data, the videos from the QA Hackathon among several other things, but alas I've not been able to stop time. These two projects perhaps have the highest importance to the Perl community, so I'm glad I've been able to get on with them and get done what I have. It's unlikely I'll have this kind of time again to concentrate solely on Open Source/Perl for several years, which in some respects is a shame, as it would be so nice to be paid to do this as a day job :) So for now, sit tight, it's coming soon...

File Under: community / conference / opensource / perl / website NO COMMENTS

The Battle of Evermore

Posted on 30th April 2009

Now that The Lord Of The Rings trilogy has finally made it to the big screen in Hollywood style, production is in progress to bring The Hobbit to the big screen for 2011. But that's a long way away for many fans.

It was far too long to wait for one fan, who decided it would be a great idea to do a fan-film, based on the Tolkien's Middle Earth. The writer/director Chris Bouchard enlisted Tolkien fans on the internet, to help with the casting and the pre- and post-production. The film has cost around £3,000 to make, and is a testiment to what can be achieved on a low budget. The film is only 40mins long, but looks to be every bit as enjoyable as the Hollywood counterparts. The film, The Hunt For Gollum, was inspired from Tolkien's writings, adapting the appendices from The Lord Of The Rings books, and fits inbetween The Hobbit and The Lord Of The Rings stories.

The film is to have its first official screening at the Sci-Fi-London film festival on Sunday 3rd May 2009, but thankfully everyone will be able to see it. As The Hunt For Gollum was made as a non-profit venture, part of the goals of the project was to release the film for free. After the official screening on Sunday, the film will be available online for download from the official website, for everyone to watch at their leisure.

In some ways it's a pity that they aren't selling this as a DVD/Blu-Ray release to help fund possible future fan-films, but I suspect that this may cause problems with licensing from the Tolkien Estate, etc. As the sites disclaimer states "[The Hunt For Gollum] is in no way affiliated with, or sponsored or approved, by Tolkien Enterprises, the heirs or estate of J.R.R. Tolkien, Peter Jackson, New Line Cinema, HarperCollins Publishers Ltd. or any of their repective affiliates or licensees." As such getting appropriate approval for a DVD/Blu-Ray release would likely be traumatic to say the least.

Having watched the trailers I'm looking forward to seeing the final film, as it looks to be every bit as magnificent as the big budget films. I also hope that Chris Bouchard gets the chance to work of a big budget feature telling another tale from Tolkien's Middle Earth, as there are plenty more stories that I'm sure other fans would love to see come to life :)

File Under: films NO COMMENTS

From the Time You Say Goodbye

Posted on 31st March 2009

Today was my last day at MessageLabs/Symantec. After 6 years it was sad to say goodbye to a lot of people, as it has been an absolute pleasure working with such a great group of people. There are some people I'm going to really miss, but thankfully several of them I will be able to catch up with at various Perl events throughout the year.

I've worked on some intresting projects and learnt a lot about SMTP, email and networking during my time with MessageLabs. In fact it's been so interesting it has been worth the morning and evening drive along the M5 for every working day in the last 6 years! But now it's time to move on.

My future challenges currently involve working on all the Open Source and Perl (obviously) projects that I haven't had time for over the past year, taking some time out, relishing NOT having to travel along the M5 every morning and evening, and planning my international speaking tour (okay there are currently only 4 dates throughout the year, but it sounds good ;)) 

I'm not planning to rush into anything, and instead want to take the time to find the right job for me, and to find the right company that can make good use of me. There are a few possibilities that have already been mentioned to me, so I'm hopeful that I won't be looking for too long.

In the meantime, expect several updates to CPAN, CPAN Testers, Maisha, among many other projects I haven't time to work on properly for the past year or so. I shall not be idle :)

File Under: life 2 COMMENTS

New Sensation

Posted on 5th March 2009

In the UK, Walkers Crips are perhaps the strongest brand of crisps on the market. They're also the most inivitive too. Every so often they try a different flavour out, and while most rarely see the light of day again, some do stick around. A more recent one is the Marmite crisps, which having proved so successful are now made by Marmite themselves. Last year Walkers Crisps ran a competition to find some new flavours. Out of the hundreds of entries they picked 6 and have now mass produced them to let the public vote on their favourites.

Walkers have been quite clever about this as they packaged them into two 6-bag bundles, with three flavours in each. It means rather than trying to find the flavours individually in your local corner shop, you can grab a couple of extra 6-bag bundles and try them all out twice.

The six flavours are:

• Fish & Chips
• Crispy Duck & Hoisin
• Cajun Squirrel
• Builder's Breakfast
• Onion Bhaji
• Chilli & Chocolate

The mix could be seen as 2 traditional British dishes, 2 international dishes and 2 just plain weird. Now of the latter, Chilli & Chocolate isn't that unusual, as I've seen and eaten Chilli flavoured chocolate bars before, but Cajun Squirrel! Never having tried squirrel I can't comment on it's accuracy, but I can't really see it being a lasting flavour beyond it's novelty value.

I've now tried them all, and after some consideration I would have to rate them as:

1. Onion Bhaji
2. Crispy Duck & Hoisin
3. Builder's Breakfast

Of the rest I wouldn't be particularly interested in trying again. Cajun Squirrel tastes like Roast Chicken, Fish & Chips tastes like Salt & Vinegar and while Chilli & Chocolate does taste exactly as it says, having first thought the chocolate bars were an interesting idea, I will most definitely not be voting for them.

It'll be interesting to see which flavour wins, and perhaps more importantly how long it lasts as a popular flavour. I think the 2 international dishes are likely to hold out longer than the others, as they both have a distinct flavour of their own, plus we Brits have a strong liking for Indian and Chinese cuisine. Still, whichever wins, I'm sure it'll be overshadowed by the next big Walkers Crisp marketing campaign :)

File Under: food / life NO COMMENTS

Open Your Eyes

Posted on 3rd March 2009

Thanks to a twitter post by Simon Phipps this morning, I read with interest a blog post by Patrick Finch, entitled Mozilla and Cybermentors. Mozilla are getting actively involved with the UK charity BeatBullying and their CyberMentors programme.

It's unlikely that anyone growing up hasn't suffered some form of bullying. It comes in all shapes and forms, and while for some of it may be minor or only last a short time, for some it can have devasating effects. A colleague recently took his own life, because the traumatic and abuse he suffered through primary and secondary school, even over ten years later, was still something that affected his life and personality, and was something he felt he could no longer cope with. I personally was first bullied by my 3rd year junior teacher. Yes you read that right a teacher, and I was just 9. It isn't just children that can be cruel and spiteful, adults can too. While I would dearly love to name the teacher in question, I have no proof beyond my word, and even now feel powerless to do anything about it, much as I did back then. Many children who suffer from bullying feel exactly the same way. Even if they told, who would listen and who would even believe them?

As Patrick points out in his post, many young people are growing up never knowing how we used to keep in contact with our friends, without using the internet or mobile phones. We play out so much of our lives online, that it shouldn't be a surprise that a recent Harvard University task force concluded that one of the biggest risks to children on the internet, isn't from sexual predators, but from bullies. The difference between the school yard bulling and cyber-bullying, is that the former is pretty much contain within a small sphere and often there are adults and peers who can deal with it and stop it. On the internet anyone can hide behind their relative anonymity and victimise just about anyone they choose. As it isn't within school grounds, teachers are often unable or ill-equiped to deal with it.

As such, the Cybermentors aims to be a way for youngsters experiencing bullying online to tell someone about it. Mozilla are offering to support 10 members of the Mozilla community to be trained as Cybermentors, who can then spend at least 2 hours a week for 4 months, helping children to cope and deal with any bullying issues. All credit to Mozilla for supporting this, and hopefully other companies will also be willing to help fund training for individuals to act as mentors.

A few years ago the GetSafeOnline campaign was initiated to help make parents and youngsters aware of the potential dangers on the internet. Identifying ways to protect themselves from viruses, phishing scams and spam, as well as unwanted websites, chatrooms and the like. While this programme is different in that it's targeting a very specific danger, it is still all about keeping the internet safe for everyone. I personally value efforts such as this, rather than the sometimes heavy-handed and misguided attempts by governments and self-appointed puritans of the internet to protect children from percieved threats.

I really hope BeatBullying and the Cybermentors programme gets a lot of internet and media exposure, as the more children are aware of it, the more chance they have of coping with it and not suffering mental anguish for the whole of their (possibly short) adult life. If you're a member of the Mozilla community, and think you can spare the time, please read Patrick's blog post and get in touch with him.

File Under: internet / life / school / security / web NO COMMENTS

Back On Line

Posted on 16th February 2009

After the last few weeks of trying to access Twitter from the command line, I set about writing something that I could expand to micro-blog to any social networking site that supports many of the Twitter API type commands. At the moment it only works with Twitter and Identi.ca, but my plan is to look at creating plugins, or more likely to allow others to create plugins, that can enable the tool to interact with other micro-blogging sites.

After trying to think of a decent name, I finally settled on Maisha. It's a Swahili word meaning "life". You can grab the code from CPAN as App-Maisha.

Currently you'll need to use the standard Perl install toolset to install the application, but ultimately I'd like to have something that you can install just about anywhere without having to go through all the headache of installing dependencies. I'll have a go at doing an .rpm and a .deb package release, and will also try using PAR. It would be nice to have this as a standalone application that just about anyone can use, but for now CPAN will have to do.

My next immediate step is to look at writing something that interfaces to Facebook without requiring a developer key or any such nonsense. It will probably have to involve a bit of screen scraping, unless there is some more official API, but as yet I haven't found it. Everything regards Facebook applications seems to centre around the developer application that can do all sorts of dubious things, but mine is purely for the user to control from their desktop, not a 3rd party website/server. Thus giving them a developer API key assigned to me is wholly inappropriate. It would be nice if they had a restricted User API, which allows you to update your status and look at your friends' statuses, but I think I'll be in the minority wanting it.

File Under: community / internet / opensource / perl / technology NO COMMENTS

You Know My Name

Posted on 9th February 2009

Having mentioned twitter in a recent post, I thought I would mention a project I decided to look at recently. The original project, twittershell, is not my own, but it appeared to be the closest to what I was after, a command line interface to Twitter, with the bonus of it being written in Perl. As a consequence of the latter, I was able to hack on the code and submit a patch to do a lot of what I wanted. I currently run the patched version, and it runs rather nicely for me. However, there is something missing.

Twitter is no longer the only micro-logging service and as such I've also signed up to identi.ca. With the APIs being pretty much the same, it should theorectically be simple to plugin an identi.ca interface to twittershell. Except it isn't. Unfortunately twittershell is written with only the Twitter API in mind. To intergrate identi.ca and other micro-blogging services, it requires a rewrite. So that's where I'm currently at. The original twittershell project hasn't been touched in over a year, so I'm hoping the orignial author won't be offended by me forking the code to a new project.

However, what do I call the new project? I would rather it not be something that identifies itself with any specific blogging service, as I would like it to have a broader appeal, that encourages others to add plugins should a new service come along. I realise this project will likely have limited appeal, as iPhone and GUI apps seem the in thing, but I want something that I can run via ssh/screen on my home box and not have to worry about watching some app running on the desktop.

One idea I had was to call it 'Mazungumzo' (Swahili for talk or conversation ... an idea stolen from Joomla!), then I thought of 'Maisha' (Swahili for life). I did look up some Welsh words, but doubt anyone would be able to pronounce them ;) I also thought of 'Rambler', but that might have too many connections to someone who goes walking across hill and dale of a weekend. So any good ideas for projects names?

File Under: internet / perl / technology NO COMMENTS

Guiding Light

Posted on 6th February 2009

In 2006 I, along with 3 others from Birmingham Perl Mongers, organised the 2006 YAPC::Europe Perl Conference. It was thankfully a great success and invigorated several with ideas of things that they could do to join or create communities. Whether that was forming a local Perl Monger user group or starting a code project that would eventually be submitted to the CPAN. However, one person was inspired to go to another YAPC the following year and then submit a talk and speak at the 2008 YAPC::Europe Perl Conference. Had the 2006 conference not been in Birmingham, UK in 2006, Edmund would likely never have gone to a YAPC, and never realised how valuable they are. Not just in terms of the presentations and speakers, but of the communities and projects that are discussed, that he might not otherwise be aware of. And perhaps most importantly, realise just how easy it is to be included into the community and how easy it is for everyone to make a difference.

At the conference dinner in 2008, Edmund was struck by the lack of younger members of the communty in attendance, and started to think about why. For some time I have been trying to understand what we as a community can do to bring new people into the community, and although my perspective has focused on YAPCs, it equally applies to projects and local user groups. However, there is one aspect that I had neglected, that was obvious to Edmund. Funding. Most of those we are trying to encourage to come along to a YAPC are likely to be unwaged or on low wages, and cannot afford the costs of travel and accommodation for 4-6 days.

Last week Edmund launched the Send-A-Newbie website, with the support of the organisers for the 2009 YAPC::Europe Perl Conference to be held in Lisbon, Portugal, together with several members of the Perl community who have voiced approval. It is a great idea, and is a great way to enable students in particular a chance to attend the biggest Perl developer conferences in Europe.

The initaive aims to send at least 6 people, although even if only 1 person is selcted to attend this year, I would consider it a success. As it happens some grant applications have already been received, so it is likely that at least 1 person will attend thanks to the programme. Hopefully more will be approved for grants providing the funding can be obtained.

So how can you help? Well if you have the ability to do so, please consider donating. Mentioned the programme to anyone who you think might be a worthy recipient of a grant, and get them to apply. Mention it at your local user group, and see whether anyone can help with a donation. In order to keep YAPCs and the Perl community healthy we need to encourage potential future stars that attending the conference is a worthwhile oppotunity. If they could benefit from a grant to cover their travel and accommodation costs, then it really is in yours and their interest to do something about it. Applications will be accepted until 1 June 2009, so there is plenty of time yet to promote and apply for grants.

File Under: community / conference / education / opensource / people / perl / yapc NO COMMENTS

Set the Fire to the Third Bar

Posted on 5th February 2009

The snow has been thick everyday of the week so far, and this morning looked particular uninviting for travelling the 45 miles to work. We had a new blanket overnight and while the kids are currently enjoying the snowball fights, I'd rather be here in the warm. If it stays like this at the weekend, I might have to take Dan up to the Waseley Hills for some sledging. I can't see Dan's football match being on, even though they managed to play in a blizzard on Sunday.

File Under: birmingham / weather NO COMMENTS