For my first post in the Enlightened Perl “Iron Man” get-more-people-to blog-outside-of-the-Perl-echo-chamber initiative, I thought it’d be fun to do a follow-up to my recent WebGUI Survey Goodies post, comparing our humble Perl powered Survey tool to the industry giant: SurveyMonkey.
According to their official website, SurveyMonkey is “the leading survey tool on the web (ranked by Alexa), with over 80% of the Fortune 100 currently using SurveyMonkey”. Impressive stuff! They must have an awesome tool. Let’s see how Survey stacks up. I’m comparing Survey to the free version of SurveyMonkey, so if someone has a paid account and wants to let us know what extra features we’re missing out on that’d be great.
Let’s start with the SurveyMonkey features page.
Design
Select From Over 20 Types of Questions – We support everything from multiple choice to rating scales to open-ended text. You can customize the layout of every question type for the ultimate in design flexibility.
20 question types sounds impressive. I don’t think we have that many. Looking closer though, their list actually only numbers 15 in total, and some of them are redundant items like “Multiple Choice (only one answer)”, “Multiple Choice (Multiple Answers)”. Ok, first win to SurveyMonkey! – we don’t have a drop-down list question type. But that’s only because no-one has asked for one yet – Kaleb could probably commit one faster than it took me to write this blog post. Digging deeper, most of the other question types could be recreated by playing with Survey’s per-question/answer settings such as horizontal/vertical display, etc.. Preconfigured bundles of questions are an interesting concept.. Survey has the concept of a multi-choice question bundle, which is a question plus all of its answers – these can be copied, and also saved so that you can deploy them at will anywhere in your survey, but it might be interesting to apply that concept to the Section level (SurveyMonkey doesn’t let you create your own bundles by the way).
Missing from SurveyMonkey’s arsenal are “Date Range”, draggable sliders (and dual-sliders), and a calendar-style date picker. So let’s call it even for now.
Use Our Survey Templates, Create custom themes
Yup, we’re fully templated, using the same template system (HTML::Template) that powers the rest of WebGUI. There appears to be some skinning you can do in SurveyMonkey, but the “Survey Template Library” is only listed for the pro version on their Pricing Page. I think we’re edging ahead at this point.
Add a Logo to Your Survey
Um..?? Well let’s see.. You can run Survey on your own domain, within a WebGUI site, however you please. And yes, that includes putting your logo wherever you like. (Looks like the logo feature is disabled in the free version of SurveyMonkey. bummer.).
Validate Your Survey Responses
SurveyMonkey has some nice per-question validation options that go beyond what Survey is currently capable of. They definitely beat us on this front. However that will change when we make the planned switch to inputEx for rendering html form elements – inputEx has a huge array of sexy looking form elements that can be easily wired up with validation logic.
Randomize or Sort Your Question Choices
Can do.
Save Your Survey as a PDF
Nice! That’d be a great feature to have. No doubt a solution is only one CPAN module away, but right now SurveyMonkey has it and we don’t.
Collection Features
I won’t list all the features here but they mostly pertain to controlling who can access your questionnaires. Survey runs inside of WebGUI, so you have the full power of WebGUI Authentication/Authorization at your disposal. You have fine-grained control over who can edit surveys, take surveys and view reports. Want to use LDAP users? Easy. WebGUI Auth is fully pluggable too, so there’s nothing stopping you from integrating your site with whatever other infrastructure you already have. Survey also lets you set time windows on how long users have to complete a survey, which can be useful if data validity is important to you. I’m confident we win hands down here, although of course you need to be competent enough to set it up.
Analysis Features
I think SurveyMonkey has more web-based features here, since our current focus is to do only basic online reporting and let users export their data to something more powerful like SPSS for proper statistical analysis. There’s definitely a lot we can improve on here, although we’re completely driven by the features that end-users want, so only time will tell what we decide to implement on this front.
OK, my turn
Off the top of my head, here are some features that Survey has but appear to be missing from the free version of SurveyMonkey:
- Jump/Branching expression. The front page mentions “control the flow with custom skip logic”. Presumably that’s similar to our Jump Expressions (see my previous post). I’d love to do a detailed comparison of the relative power of their expression engine vs ours, but sadly it’s only available in the paid version. Ours gives you a very large subset of the Perl language to play with, so we’re probably winning here.
- I hardly consider this a “feature”, but it looks like the free version of SurveyMonkey only lets you create 10 questions per survey, and you can only have 100 responses. And the free version doesn’t let you export your data to a spreadsheet program. Ouch!
- Scoring, and branching based on scores
- No sign of quiz mode with tabular and graphical feedback for the user to let them know how they scored (see my previous post)
- Visualisation (see my previous post)
- Drag and drop section/question/answer re-ordering. I can’t imagine see myself being very productive if I was trying to work on the 500+ question survey I’m building inside of the menu-driven SurveyMonkey interface.
- The ability to modify the survey engine to do whatever custom magic you like
- Complete control over your data ffs!This is the biggie as far as I’m concerned. Surveys built with SurveyMonkey store their data on the SurveyMonkey servers. Australia’s privacy laws are very strict about sending private data overseas, and I assume the same is true for other countries. You could land yourself in a world of trouble if you created supposedly “private” surveys where people were asked to enter personally identifying information (particularly if the data was medical related) and that data was stored overseas. You can pay to add SSL to your SurveyMonkey account, but there is no mention of database encryption, which is also something to ponder. At the end of the day, if you’re doing anything remotely private you should be self-hosting the data.
- Have access to the code, which, among other things, means that you can build a comprehensive test suite for your complex 500+ question survey. For any complex Survey involving non-trivial branching logic, it’s invaluable to be able to build an automated test suite that independently verifies your branching and scoring logic (and makes sure that any future changes don’t break things). In Survey you can write this in Perl at the code level, interacting with Survey and the database via the WebGUI API, or using any programming language to create an automated “virtual user” that interacts with the Survey via the web interface (simulating page clicks etc..).
So…
Overall that was a bit disappointing. I mean, there’s are definitely areas we can improve on, and I’m sure that the SurveyMonkey documentation and support is great, but for something that purports to be the best of breed, … well let’s just that that as far as the free version goes you’re pretty limited in what you can do. No doubt the paid version is better, and of course most people out there just want something simple that they can jump right in and start using with minimal fuss. But it seems to me that pretty soon you start running into limitations, especially with the free version.
Ok, enough of that, let’s get back to what’s new in Survey. Here are a couple of new features that have been added in the past week:
Firstly, Kaleb committed some code a few days ago that allows you to flag Sections as “logical”. What this means is that the section is hidden from the user, but appears in the admin screen and can have branching expressions defined for it. This is really handy if you have sophisticated branching logic and you run into a situation where you need an intermediate jump target. For example, it’s a common scenario to have a bunch of “pre-section” rules that you want to evaluate before letting the user start the section. Rather than duplicating these rules on every other node that jumps to the section, you can have those nodes jump to a pre-section node that evaluates the rules in one place.
Another awesome feature that was recently committed to the Survey repo (and that won’t cost you a banana) is tagging. The Survey expression engine now allows you to tag data as you go, which allows you to do all sorts of interesting things. For instance, say you’re doing one of those magazine pop-quizzes where you (very scientifically) classify people into personality types. After asking a bunch of dog-related questions, you can now say something like this in your branch expression:
tag(“personality type”, “dog lover”) if score(“dog questions section”) > 5
That creates a new data field called “personality type” that is stored against the user’s response (in this case the stored data is the string “dog lover”). Later, you can actually use that tag inside another branch expression to only show a bunch of questions to the dog lovers:
jump { tag(“personality type”) eq “dog lover” } “more dog questions section”
Or you could use that tag inside the text of another question as templated text:
Clearly you’re a [[personality type]]!
..which would appear to the user as:
Clearly you’re a dog lover!
And finally, the tagged data is stored as part of the user’s response data, meaning that you can pull it out of the database later (using another WebGUI asset like SQLReport or your own custom code) and do whatever you like with it.
