Memoirs of a Roadie

Sunshine Of Your Love

Posted on 17th July 2014

The survey results for YAPC::NA 2014 are now online.

Even we with lower numbers of attendees this year, 27% of you took the time to respond to the survey. As always, this doesn't necessarily allow us to see the whole picture, but hopefully it is enough of a cross-section of the attendees to help us improve future events. Once again we had a healthy number of respondees for whom this was their first YAPC, many having never attendeed a workshop either.

There was a bit of a mixed reaction throughout the survey. Although having read the feedback from the talk evaluations, there was a lot of positive comments, and several words of encouragement for some of the new speakers, which was great to see. Overall it seems to have been another great conference, although there are areas of communication that many felt could be improved.

I see I'll have to expand the options for the question "What other areas of the Perl Community do you contribute to?", as firstly I would include hacking on Perl core, as part of a Perl project (i.e. a group of great people doing great work to improve Perl), but also to include a new option; I donate to one of the funds managed by TPF or EPO. During the conference I saw a few Twitter posts about contributing to some of the Perl funds, which I think came about following Dan Wright's presentation. It is great that so many have donated, big and small amounts, to the various funds. They all help to improve and promote Perl, and give us good reasons to continue putting together great conferences and workshops every year.

It was great to see any good list of suggestions for topics this year, and I hope that speakers new and old, get some ideas for future talks from them.

Lastly it does seem that the location question, really does depend where the current location is. The higher numbers last year may also indicate that Austin was easier to get to for most people, whereas a more easterly location, such as Florida, may restrict the ability to attend for those on the west coast. It would be interesting to see whether a similar opposite trend would result if the conference was held in Nevada, California, Oregon, Idaho, Utah or Arizona. There must be several Perl Monger groups in those states, so if you're in one, perhaps think about balancing out the number of eatern hosting states ;)

Lullaby of London

Posted on 21st December 2013

The 2013 London Perl Workshop Conference Survey results are now online.

Although percentage wise the submissions are up, the actual number of respondents are just slightly lower than previous years. Though it has to be said I'm still pleased to get roughly a third of attendees submitting survey responses. It might not give a completely accurate picture of the event, but hopefully we still get a decent flavour of it.

Two questions, which I plan to pay closer attention to in future surveys are; 'How do you rate your Perl knowledge?' and 'How long have you been programming in Perl?' Originally the age question usually gave some indication of how long someone had been using Perl, but from experience, I now know that doesn't work. As such, these two questions hopefully give us a better idea of the level of knowledge and experience of attendees. Perhaps unsurprisingly London.pm had a lot of attendees who have been around the Perl community for many years, particularly as it was the first non-US Perl Monger group. However, we do still see a notable number of people who are relatively new to Perl. It will be interesting to see whether these numbers change over the years, as although the community doesn't appear to be growing radically, it is still attracting first-time attendees.

Looking at the list of suggested topics, I was intrigued to see "Testing" in there. Apart from my own talk and Daniel Perrett's, there wasn't anything specifically about testing. I don't know if its because the older hands are more weary of giving test talks, or whether everyone thinks everything has been said, but I do think it's a topic that worth repeating. We regularly have new attendees who have never seen these talks before, so hopefully we'll see some more submitted at future workshops and YAPCs. There was also a lot of interest in practical uses of web frameworks. Although Andrew Solomon held a Dancer tutorial, seeing how to solve specific problems with web applications would be valuable to many. Having said that, the diverse range of subjects that was on offer at the workshop, was equally as interesting. I just hope Mark and Ian are so inundated with talks next year, we have an even greater choice from the schedule.

Thank you to Mark and Ian from organising another great Perl event, and thanks to all the speakers for making it worth attending. Also to all the attendees, especially those who took the time to respond to the survey, and for all the talk evaluations. I know the speakers appreciate the evaluations, as I've had a few thank yous already :)

Enjoy the results.

The Great Gates of Kiev

Posted on 27th October 2013

I've now uploaded the survey results for YAPC::Europe 2013 and The Pittsburgh Perl Workshop 2013. Both had only a third of attendees respond, which for PPW is still 20 out of 54, and 122 out of 333 for YAPC::Europe.

YAPC::Europe

In previous years we have had higher percentages of response at YAPC::Europe, but that is possibly because I was in attendance and promoted the surveys during lightning talks, and encouraged other speakers to remind people about them. It may also be the fact that there is a newer crowd coming to YAPCs, and the fact we had 44 out of the 122 respondees saying that this was their first YAPC, who have never experienced the surveys. While definitely encouraging to see newer attendees, it would be great to see more of their feedback to help improve the conferences each year. Like YAPC::NA 2013, we have reintroduced the gender question. This time around I didn't get the negative reaction, but this may also be due to the fact I've had more feedback about approaching the subject this time around. Perhaps unsurprisingly, there were rather more male respondees, but I am also very encouraged to see that 8 respondees were female. While its difficult to know the exact numbers at the event, I'd like to think that we have been able to welcome more women to the event, and hopefully will see this number increase in the future.

Looking at the locations where attendees were travelling from to attend YAPC::Europe in Kiev, it is interesting to see a much more diverse spread. Once upon a time the UK was often the highest number, even eclipsing the host country. This year, it seems many more from across the whole of Europe took advantage of the conference. Again I think this is very encouraging. If Perl is to grow and reach newer (and younger) audiences, it needs to be of interest to a large number of people, particular from many different locations. While the UK (particularly London, thanks to Dave Cross) was perhaps the start of European Perl community, YAPC::Europe is now capable of being hosted in just about any major European city and see several hundred people attend. It will be interesting to see if Sofia next year, has a similar evenly spread of locations.

Of those that responded, it does seem that we had more people in the advanced realm. Particularly seeing as we had 56 people respond with more than 10 years experience of Perl. Back when we started the surveys, it would likely have been only a handful of people who attended who could have said that they had been programming Perl for more than 10 years. Thankfully though, it isn't just us old hands, as those only programming in Perl for a few years or less, are still making it worthwhile for speakers to come back each year and promote their projects big and small to a new audience.

One comment in the feedback however, described the Perl community as hermetic. I'm not entirely convinced that's true, but it is quite likely that some find it difficult to introduce themselves and get involved with projects. Having said that, there are plenty of attendees who have only been coming to YAPCs, or been involved with the Perl community, for a short while, who have made an impact, and are now valued contributors. So I guess it may just be down to having the right personality to just get stuck in and introduce yourself. This is one area of the Perl community that Yaakov Sloman is keen to break down barriers for, even perceived ones. We do need more Yaakov's at these events to not just break the ice, but shatter it, so we all see the benefit of getting know each other better.

And talking of getting to know others better, it was a shame I didn't get to meet the 15 CPAN Testers who responded. We have had group photos in the past, and I'd like to do more when I next attend a YAPC, but I think it would also be very worthwhile if the Catalyst, Dancer, Padre and many other projects could find the time to do some group shots while at YAPCs. At YAPC::NA it is a bit of a tradition for all those who contribute to #perl on IRC to have a large group photo, but it's never encouraged others to do the same. Perhaps this is also a way for people to get to know project contributors better, as new attendees will have a better idea of who to look out for, rather than trying to figure out who fits an IRC nick or PAUSEID.

The suggest topics for future talks were quite diverse, and "Web Development Web Frameworks Testing" is definitely an interesting suggestion, particularly as we are seeing more and more web frameworks written in Perl now, and we are after all very well known for our testing culture. One question I'm planning to include next years surveys, also looks at some of these topics and attempts to find out what primary interests people have. Again, this might help guide future speakers towards subjects that are of interest to their target audience.

Pittsburgh Perl Workshop

Workshops, by their very nature, are much smaller events, but with Pittsburgh being the home of the very first YAPC::NA, it is well established to host a workshop, and it would seem attracted some high profile speakers too. Possibly as a consequence, at least one attendee felt some of the talks were a little too advanced for them. At a smaller technical event it is much harder to try and please everyone, and with fewer tracks there often is less diversity. Having said that, I hope that the attendee didn't feel too overwhelmed, and got something out of the event in other talks.

From the feedback it would seem that more knowledgeable Perl developers were in attendance, so understandable that more talks might lean towards more advanced subjects, but as mentioned for YAPCs, speakers shouldn't feel afraid of beginner style introductions or howtos for their project, that could appeal to all levels of interest.

Overall I think the Pittsburgh Perl Workshop went down very well.

What's Next?

I now have to compile the more detailed personal feedback for these and the YAPC::NA organisers, so expect to see some further documentation updates in the near future. In addition, I want to work more on the raw data downloads. While it's interesting to see the data as currently presented, others may have other ideas to interrogate the raw data for further interesting analysis. I also still need to put the current code base on CPAN/GitHub and add the features to integrate with Act better.

The next survey will be for the London Perl Workshop at the end of November. If you are planning a workshop, YAPC or other technical event that you'd to have a survey for, please let me know and I'll set you up. It typically takes me a weekend to set up an instance, so please provide as much advanced warning as possible.

Young Parisians

Posted on 10th April 2012

Did I mention I went to Paris to take part in the 2012 QA Hackathon? Did I remember to mention all the cool stuff I got done? Well if you've been hiding for the past few weeks, have a look at the last couple of posts :)

As per usual, while there I took my camera along. However, unlike many previous visits to Paris, I didn't do any sight-seeing. And that includes failing to wander around the venue we were in and discovering the real submarine among other things, that others found while taking a breath of fresh air.

Instead I spent my time hacking away, and only occasionly coming up for air for food, drink and some camera action.

With over 40 people in attendance, it was going to be difficult to capture everyone, but I think I managed it. If I did miss you, my apologies. It was great to meet so many friends old and new, and a real pleasure to finally put faces to names that I've known for a while, but not had the opportunity to meet in person.

So many great things happened in Paris, and I'm really looking forward to see what we can achieve in London for the 2013 QA Hackathon. See you there.

• Photos from the 2012 QA Hackathon in Paris
  

Party In Paris

Posted on 31st March 2012

I'm currently at the 2012 QA Hackathon working on CPAN Testers servers, sites, databases and code. It has already been very productive, and already I have two new module releases.

CPAN::Testers::WWW::Reports::Query::AJAX

This module was originally written in response to a question by Leo Lapworth about how the summary information is produced. As a consequence he wrote CPAN::Testers::WWW::Reports::Query::JSON, which takes the data from the stored JSON file. In most cases this data is sufficient, but the module requires parsing the JSON file which may be slow for distributions with a large number of reports. On the CPAN Testers Reports site, in the side panel on the distribution page, you will see the temperature graphs measuring the percentage of PASS, FAIL, NA and UNKNOWN reports a particular release has. This is glean from an AJAX call to the server.

But what if you don't want an HTML/Javascript styled response? What if you wanted the results in plain test or XML? Enter CPAN::Testers::WWW::Reports::Query::AJAX. Now you can use this to query the live data to for a particular distribution, and optionally a specific version, all the result values and get them pack as a simple hash to do with as you please.

I anticipate this might be most useful to project website who wish to display their latest results from CPAN Testers in some way. They can now get the data, and present it however they wish.

CPAN::Testers::WWW::Reports::Query::Reports

Now we get to perhaps the bigger module, even though its smaller than the one above. This module is perhaps most useful to all those who are trying to maintain a version of the cpanstats metadata from the SQLite database. As mentioned previously the SQLite database has been giving us grief over the past year, and we haven't gotten to the bottom of it. Andreas suspects there is some unusual textual data in some reports that is causing SQLite problems when it tries to store it. I'm not quite convinced by this, but as I'm only inserting records, I'm at a lost as to what else be the cause.

The SQLite file now clocks in at over 1GB compressed and over 8GB uncompressed, and is starting to take a notable amount of disk space (though considerably smaller than the 250GB+ Metabase database ;) ). It is also a significant bandwidth consumer each day, which can slow processing and page displays, as disk access is our limiting factor now.

Enter CPAN::Testers::WWW::Reports::Query::Reports. This module uses the same principles as the AJAX module above, but now accesses an new API on the CPAN Testers Reports site to enable consumers to get either a specific record or a whole range of report metadata records. Currently the maximum number of records that can be return in a single request is 2500, but this may be increased once the system has been proven to work well. Typically we have around 30,000 reports submitted each day, so to allow consumers to make best use of this API, I will look to increasing the limit to maybe 50,000 or 100,000. I want to impose a limit as I don't want accidental requests being sent to consume the full database in one go, as again this would put a strain on disk access.

The aim of the module is to allow those that currently consume the SQLite database, to more regularly request smaller updates and store the results in any database they so choose. Even into a NoSQL style database. It will ultimately reduce the bandwidth, data stored and processing to gzip and bzip2, which then means we can reallocate effort to more useful tasks.

If you currently consume the SQLite database, please take a look at this module and see how you can use it. I plan to include some example scripts that could be drop-in replacements for your current processes, but if you get there first, please feel free to submit them to me too, and I will include them with full credit. If you spot any issues or improvements, please also let me know.

CPAN Testers Platform Metabase Facts

This morning we had a CPAN Testers presentation and discussion hosted by David Golden. As there is plenty of interest from a variety of parties about CPAN Testers, it was a good opportunity to highlight an area that needs work, but which David and myself, as well as other key developers in the CPAN Tester community, just don't have time to do. Breno de Oliveira (garu or IRC) has very kindly stepped forward to look at one particular task, which we have been wanting to write since the QA Hackathon in Birmingham, back in 2009!

Breno has written a CPAN Testers client for cpanminus. At the moment its a stand-alone application, but it may well be included within cpanminus in the future. As part of writing the application, Breno asked David and myself about how the clients for CPAN::Reporter and CPANPLUS::YACSmoke create the report. Due to the legacy system we came from (email and NNTP) we still use an email style presentation of the reports. However, it has always been our intention to produce structured data. A CPAN Testers Report currently has only two facts that are required, a Legacy Report and a Test Summary. However there are other facts that we have already scoped, except they are just not implemented.

Back last year the Birmingham Perl Mongers produced the CPAN::Testers::Fact::PlatformInfo fact, that consumes the data from Devel::Platform::Info (which we'd written the previous year). The problem with the way test reports are currently created, is that we don't always know the definite platform information for the platform the test suite was run on. Reports, particularly in the Perl Config section, can lie. Not big lies necessarily, but enough that it can disguise why a particular OS may have problems with a particular distribution.

Breno is now looking to produce a module that firstly abstracts all the metadata creation parts from CPAN::Reporter, CPANPLUS::YACsmoke, Test::Reporter as well as his own new application, and puts them into a single library that can then create all the appropriate facts before submitting the report to the metabase. Hopefully he can get this done during the Hackathon, but even if he doesn't, we're hopful that he will get enough done to make it easy to complete soon after. Once we then patch the respective clients to use the new library, we will then start to be able to do interesting things with how we present reports.

The CPAN Testers Reports site only displays the legacy style report, which for most is sufficient, but it really would be nice to have some specially styled presentations for particular sections, or even allow user preferences to show/hide sections automatically when a user reads a report.

CPAN Testers Admin site

This is a site that I have been working on, on and off, for about 4 years, before we even had a Metabase. As a consequence it has been promised at various points and I've always failed to deliver. Now I have release the modules above, and there have been several comments already about having such functionality, I think I need to put some focus on it again. I have shown Breno the site running on my laptop and he has given me some more ideas to make it even more useful. It'll still be awhile before its released, but this will likely be down to running with some beta testers first before a major launch, just so it doesn't break the eco-system too badly!

Essentially the site was written to help authors and testers to highlight dubious reports and have them deleted from the system. Although the reports won't actually be deleted, they will be marked to ignore, so that they can be removed from JSON files and summary requests, as well as on the CPAN Testers Report site. This will hopefully enable us to get more accurate data, and bogus reports about running out of memory or disk space can be disregarded.

However, following Breno suggestions, I will look to making the site more public, so that authors can more easily see the reporting patterns without having to log in. The log in aspect will still be needed to flag reports, but the alternate browsing of reports by testers will be much more accessible.

Thanks

I would like to thank a few people who have helped to get me here, and have enabled these QA projects, not just CPAN Testers, to advance further.

Firstly I would like to single out ShadowCat Systems, who have very kindly paid for my flight here. Thanks to BooK and Laurent for organising the event, and to all the sponsors and Perl community who have provided the funding for the venue, accommodation and food for the event. It has already been very much appreciated, and hopefully the significant submissions to GitHub and PAUSE are evidence of just how worthwhile this event is.

Thanks also to all those who are here, and are helping out in all shapes and forms to help Perl QA be even better than it already is.