Memoirs of a Roadie

Wondering What Everyone Knows

Posted on 26th September 2014

The YAPC::Europe 2014 survey results are now online.

YAPC::Europe Survey

Although we appear to have had an increased response this year, 42% up from 36%, there were only 74 actual responses, down from 122 last year. Sadly it was a smaller audience in total, and perhaps that's partly due to the previously core attendees not wishing to travel further. However, that said, as we are attempting to increase attendance at conferences, it was hoped that more first time attendees from South Eastern Europe would attend. Unfortunately, it would seem the opposite was the case, with only 4 responses from people who came from Bulgaria itself. I am willing to accept that many of the non-respondees are non-native English speakers, and found it difficult to complete the survey, but that would be true across a lot of Europe too. Those responding were not just the usual attendees either, there were many first and second time attendees there too.

I think part of this lack of reponse from previous years, is also down to the lack of promotion of the surveys. As I'm not able to attend in person at the moment, I have to hope that the organisers advertise the surveys, but it would be nice to have the speakers themselves mention them too. It is obvious that several speakers value the feedback they get, both publicly and privately, so it would be nice to see that encouraged after each talk. My hope is that the knock-on effect is that respondees also complete the main survey too.

Looking at both the demographics and the questions around Perl knowledge and experience, it would still seem we have an attendance that is on a upward curve for all aspects. Although attendance is perhaps getting older and wiser, that's not necessarily true of Perl programmers in general, particularly outside the traditional group of people who get involved in the community. It would be great to see how we can change that in the future. After all, Perl's future relies on those coming to the language in their teens and early twenties. If you're interested in helping, reading Daisuke Maki's post about what do you want your YAPC to be?, is very worthwhile. YAPC::Asia has helped to change attitudes, but that has only happened because the organisers set themselves goals of what they want to achieve. With YAPC::Europe, we haven't had this, partly due to the organisers changing each year, but also because there doesn't appear to be any long term goals.

From the responses of this year's attendees, we can see that we have a lot of people who are involved with the community in several ways. However, what about those who aren't involved. Is it just this conference is their first exposure to the Perl community, did those people not respond to the survey, or is it that they weren't there? YAPC::Asia's decision to be inclusive to other languages and technologies has been a benefit, not just to Perl, but to the whole OpenSource community in Japan. Couldn't we do the same in Europe or North America?

This year the conference schedule was largely handled by a small remote group of interested volunteers, who stepped up when Marian asked for help. I believed it worked, and if the same group, perhaps with input from next year's local organisers for what they would like, this might be a step to help improve the schedule. Using the same team for YAPC::Asia has worked, so why not elsewhere. It is always difficult to get to know about good speakers, particularly if the speakers was previously unknown to schedule organisers. The missing speakers are not too surprising, but it is always nice to see some unexpected suggestions, and it is useful for schedule organisers to get these hints so that they can try and reduce clashes. I would also encourage attendees to make use of the star system in Act, as this too can be used by the schedule organisers to identify popular talks, and ensure that in future they are in appropriately sized rooms.

I would also suggest that speakers and organisers take note of the topic suggestions. These are subjects people are asking to hear, and if they are returning attendees, may well be your future attendees at your next talk. The beginner track is also a great one, and would be worth looking at for the future. I know it was attempted for several years, but seems to have died off, which is a shame. Those new to Perl wanting to get more involved, can and want to learn a lot from the rest of us. It's why they come. It would be great to have them return to their bosses afterwards full of ideas, which may then enable them and/or their colleagues to return in the future.

Talk Evaluations

In an organisers mailing list, it was asked whether the talk evaluation surveys should be partly public, and exposed to the organisers. For anyone worried about that, I have said no. One reason being that I don't think that the results can or should be compared. Every attendee has a different opinion and a different scale to anyone else. We also have a wide variety of attendees to talks. Trying to compare a talk with 100 attendees to one with 10 doesn't make any sense. My second reason is that I think speakers or respondees should not be publicly compared. In private, one bad review could be taken into consideration and either disregarded, or help the speaker improve next time. Making that public is likely to have two effects; firstly a request from speakers to not be part of the evaluations, and secondly judgements made against the speaker for past work, when they have learnt and improved. They deserve our support, not rejection.

I have also been asked about providing talk evaluations for the video streams. Initially I was against this, partly because of a couple of people launching a torrent of abuse at me for not doing them already! However, a couple of people who have been more rational have asked about doing them. As such, I've started to look at what is involved. I expect these talk evaluations to be different, if only because the listener is not in the room, and the rapport and presentation will have a different feel during the stream. I may look to introduce this for YAPC::NA 2015, if I can get the rest of my plans for the surveys implemented, together with the changes to Act in time.

Future Surveys

I have several plans to write about the future of the surveys, and I'll be writing a few more blog posts about them in the months to come. If you have suggestions for improvements, please let me know. The software is now on CPAN and GitHub, so you are welcome to contribute changes, to help move things along. If you would like your event, workshop or conference, to have a survey, please get in touch and I'll see what I can set up for you.

Sunshine Of Your Love

Posted on 17th July 2014

The survey results for YAPC::NA 2014 are now online.

Even we with lower numbers of attendees this year, 27% of you took the time to respond to the survey. As always, this doesn't necessarily allow us to see the whole picture, but hopefully it is enough of a cross-section of the attendees to help us improve future events. Once again we had a healthy number of respondees for whom this was their first YAPC, many having never attendeed a workshop either.

There was a bit of a mixed reaction throughout the survey. Although having read the feedback from the talk evaluations, there was a lot of positive comments, and several words of encouragement for some of the new speakers, which was great to see. Overall it seems to have been another great conference, although there are areas of communication that many felt could be improved.

I see I'll have to expand the options for the question "What other areas of the Perl Community do you contribute to?", as firstly I would include hacking on Perl core, as part of a Perl project (i.e. a group of great people doing great work to improve Perl), but also to include a new option; I donate to one of the funds managed by TPF or EPO. During the conference I saw a few Twitter posts about contributing to some of the Perl funds, which I think came about following Dan Wright's presentation. It is great that so many have donated, big and small amounts, to the various funds. They all help to improve and promote Perl, and give us good reasons to continue putting together great conferences and workshops every year.

It was great to see any good list of suggestions for topics this year, and I hope that speakers new and old, get some ideas for future talks from them.

Lastly it does seem that the location question, really does depend where the current location is. The higher numbers last year may also indicate that Austin was easier to get to for most people, whereas a more easterly location, such as Florida, may restrict the ability to attend for those on the west coast. It would be interesting to see whether a similar opposite trend would result if the conference was held in Nevada, California, Oregon, Idaho, Utah or Arizona. There must be several Perl Monger groups in those states, so if you're in one, perhaps think about balancing out the number of eatern hosting states ;)

The Great Gates of Kiev

Posted on 27th October 2013

I've now uploaded the survey results for YAPC::Europe 2013 and The Pittsburgh Perl Workshop 2013. Both had only a third of attendees respond, which for PPW is still 20 out of 54, and 122 out of 333 for YAPC::Europe.

YAPC::Europe

In previous years we have had higher percentages of response at YAPC::Europe, but that is possibly because I was in attendance and promoted the surveys during lightning talks, and encouraged other speakers to remind people about them. It may also be the fact that there is a newer crowd coming to YAPCs, and the fact we had 44 out of the 122 respondees saying that this was their first YAPC, who have never experienced the surveys. While definitely encouraging to see newer attendees, it would be great to see more of their feedback to help improve the conferences each year. Like YAPC::NA 2013, we have reintroduced the gender question. This time around I didn't get the negative reaction, but this may also be due to the fact I've had more feedback about approaching the subject this time around. Perhaps unsurprisingly, there were rather more male respondees, but I am also very encouraged to see that 8 respondees were female. While its difficult to know the exact numbers at the event, I'd like to think that we have been able to welcome more women to the event, and hopefully will see this number increase in the future.

Looking at the locations where attendees were travelling from to attend YAPC::Europe in Kiev, it is interesting to see a much more diverse spread. Once upon a time the UK was often the highest number, even eclipsing the host country. This year, it seems many more from across the whole of Europe took advantage of the conference. Again I think this is very encouraging. If Perl is to grow and reach newer (and younger) audiences, it needs to be of interest to a large number of people, particular from many different locations. While the UK (particularly London, thanks to Dave Cross) was perhaps the start of European Perl community, YAPC::Europe is now capable of being hosted in just about any major European city and see several hundred people attend. It will be interesting to see if Sofia next year, has a similar evenly spread of locations.

Of those that responded, it does seem that we had more people in the advanced realm. Particularly seeing as we had 56 people respond with more than 10 years experience of Perl. Back when we started the surveys, it would likely have been only a handful of people who attended who could have said that they had been programming Perl for more than 10 years. Thankfully though, it isn't just us old hands, as those only programming in Perl for a few years or less, are still making it worthwhile for speakers to come back each year and promote their projects big and small to a new audience.

One comment in the feedback however, described the Perl community as hermetic. I'm not entirely convinced that's true, but it is quite likely that some find it difficult to introduce themselves and get involved with projects. Having said that, there are plenty of attendees who have only been coming to YAPCs, or been involved with the Perl community, for a short while, who have made an impact, and are now valued contributors. So I guess it may just be down to having the right personality to just get stuck in and introduce yourself. This is one area of the Perl community that Yaakov Sloman is keen to break down barriers for, even perceived ones. We do need more Yaakov's at these events to not just break the ice, but shatter it, so we all see the benefit of getting know each other better.

And talking of getting to know others better, it was a shame I didn't get to meet the 15 CPAN Testers who responded. We have had group photos in the past, and I'd like to do more when I next attend a YAPC, but I think it would also be very worthwhile if the Catalyst, Dancer, Padre and many other projects could find the time to do some group shots while at YAPCs. At YAPC::NA it is a bit of a tradition for all those who contribute to #perl on IRC to have a large group photo, but it's never encouraged others to do the same. Perhaps this is also a way for people to get to know project contributors better, as new attendees will have a better idea of who to look out for, rather than trying to figure out who fits an IRC nick or PAUSEID.

The suggest topics for future talks were quite diverse, and "Web Development Web Frameworks Testing" is definitely an interesting suggestion, particularly as we are seeing more and more web frameworks written in Perl now, and we are after all very well known for our testing culture. One question I'm planning to include next years surveys, also looks at some of these topics and attempts to find out what primary interests people have. Again, this might help guide future speakers towards subjects that are of interest to their target audience.

Pittsburgh Perl Workshop

Workshops, by their very nature, are much smaller events, but with Pittsburgh being the home of the very first YAPC::NA, it is well established to host a workshop, and it would seem attracted some high profile speakers too. Possibly as a consequence, at least one attendee felt some of the talks were a little too advanced for them. At a smaller technical event it is much harder to try and please everyone, and with fewer tracks there often is less diversity. Having said that, I hope that the attendee didn't feel too overwhelmed, and got something out of the event in other talks.

From the feedback it would seem that more knowledgeable Perl developers were in attendance, so understandable that more talks might lean towards more advanced subjects, but as mentioned for YAPCs, speakers shouldn't feel afraid of beginner style introductions or howtos for their project, that could appeal to all levels of interest.

Overall I think the Pittsburgh Perl Workshop went down very well.

What's Next?

I now have to compile the more detailed personal feedback for these and the YAPC::NA organisers, so expect to see some further documentation updates in the near future. In addition, I want to work more on the raw data downloads. While it's interesting to see the data as currently presented, others may have other ideas to interrogate the raw data for further interesting analysis. I also still need to put the current code base on CPAN/GitHub and add the features to integrate with Act better.

The next survey will be for the London Perl Workshop at the end of November. If you are planning a workshop, YAPC or other technical event that you'd to have a survey for, please let me know and I'll set you up. It typically takes me a weekend to set up an instance, so please provide as much advanced warning as possible.

Who Knows Where The Time Goes?

Posted on 24th July 2013

YAPC::NA 2013 - The Results Are Out

The YAPC::NA 2013 Conference Survey results are now online.

The number of responses was much lower than in previous years, which is a shame, but may in part be due to one comment I received, saying it was too long. Reviewing the survey, I'd have to agree, and I'll be removing some of the questions for future surveys. Some of the questions had good intentions originally, and did provide an insight to what people got out of the conference. However, there is now a degree of predictability about them, that doesn't warrant their inclusion. Such questions about holidays and speakers you missed really don't add anything any more. The latter has generated some interesting comments over the years, but typically the same names appear each year.

This year was also slightly different, as the organisers asked for a lot of additional questions. Particularly related to the Code of Conduct. I will be forwarding the results of these questions to the TPF in the next day or two. They may choose to make the results public, but for now they won't appear on the YAPC Survey site. Of the other questions they asked, most related specifically to YAPC::NA, and wouldn't be applicable to other conferences and workshops. These too will be reviewed for next year.

Interestingly, VM Brasseur has done some analysis of the survey data, particularly around the age of attendees, and the length of time people have been a Perl programmer. Although the survey includes the former, it doesn't really include the latter. We do ask what level people feel they are at, but it'll be an area I'll be reviewing for future surveys.

As both the surveys and VM's analysis shows, the Perl community (at least those answering the survey) is getting older. I've noticed this too when attending. There are new and younger people attending, but generally the audience has been getting older. In the UK, this was identified in an technical article I read a few years ago (sadly I don't have a link to the source), which highlighted a shift in the late 80s/early 90s away from writing computer games on Spectrums, Dragons and Beebs to just playing consoles. I suspect the age of attendees at other technical conferences are also seeing a shift.

As noted in a previous post, I'm going to be looking at the Conference Survey software over the summer, and hopefully integrate it more with the Act software. I'm hoping this may encourage more to respond. I'll also be reviewing the survey itself, and looking at better and more relevant questions to include. If you have ideas of how to improve the survey, please feel free to drop me an email.

Enjoy :)

Die Mensch-Maschine

Posted on 10th November 2011

German Perl Workshop 2011 - Speaker Evaluations

I have now sent out all the talk evaluations from this year's German Perl Workshop or more correctly Der 13. Deutsche Perl-Workshop. If you were a speaker and haven't received an email, please check your spam folders first, and let me know (barbie at cpan . org) if you don't find it. The mail will have come from barbie at birmingham . pm . org.

My thanks to all the organisers of GPW2011 and everyone who took the time to respond to the evaluations. From previous experience the speakers have very much appreciated your feedback. I would also like to extend extra special thanks to Max Maischein aka "Corion", who took the time to translate all the questions, templates and emails into German for me.

The results of the main survey will be published soon on the YAPC Conference Surveys site.

This is the first survey that I have undertaken in a non-English language, and for the most part it has been very successful. While there have been some slight problems due to byte vs character lengths (I'll save my 'why-oh-why did we ever start with ASCII and not UTF-8' rant for another day), the work Max has done to provide all the translations has started me on a path to be able to accommodate other languages.

At the moment the plan is to create a GitHub repository of all the necessary files, with language branches containing the appropriate translations. Then should anyone wish to request a survey instance in the future in a non-English language, their first step will then be to provide the necessary translations for me. It currently takes roughly a day to set-up an instance, so drop-in replacements for these files will ease the set-up process. It will also mean that as time goes on and questions get added, refined or deleted, we can replicate these changes across all languages.

I'd like to see the survey site get more use in the future, and although I'm happy to run the survey sites, with the support of Birmingham Perl Mongers, the longer term goal has always been to allow others to create their own instances. With the official release of Labyrinth this year, much of the tool set is now Open Source. I still need to release the Survey Plugin for Labyrinth and the additional command-line tools used, but getting the language translations moving will be a big step forward. Hopefully I'll have more news in the new year.