Skip to main content

The Madding Crowd : Some Thoughts About Crowdsourcing

Sounds like a great idea. You've got a bunch of labour-intensive work to get done. Someone says: "Let's turn it into a game and let the crowd do it for us." So you knock up an app and the idle hordes get to work. You save money while building your online reputation. What could go wrong?

Clearly the strategy can work. Wikipedia. SETI@home. Galaxy Zoo. Foldit. The hordes are out there and willing to support projects that they care about. But for every one of those high profile success stories there are plenty of low profile examples of crowdsourcing that has gone nowhere.

So before you jump onto the crowdsourcing bandwagon, here are some questions you should be asking:

- Why should people work for you? People engage with problems that they care about. Editing items on Wikipedia is a by-product, for example, of their interest in some topic, they start with the interest and showcase their expertise via Wikipedia. Or they use Galaxy Zoo to engage their interest in the universe. And in each case they get a sense that they're contributing to some bigger purpose. How are you going to offer similar engagement and purpose?

- Will people enjoy working for you? That sense of purpose can wear off pretty quickly if you make it too difficult for people to do the work. After the initial surge of goodwill, people have to enjoy the task for its own sake. So you need to build an app that's easy and fun to use. Quick and functional design works for a utility app, where people care about the result rather than the task, but for crowdsourcing you need more. This takes time to design, build and test. Knocking together something in your spare time won't cut it.

- Will it really be that much fun? Let's face it, most of us can't offer that much purpose. Helping XYZ insurance company price its policies a little more sharply just doesn't compare with getting in touch with ET. So your app had better offer some genuine fun. Now look at the games market - there's some pretty compelling stuff out there, created by people whose sole objective is fun. They have to be good to survive. Can you really compete?

- How much support will people need? Viral doesn't just happen. It takes effort to recruit any sizeable crowd, and more effort to support them. Look at Wikipedia - there's an awful lot of work going on around conflict resolution, for example. It's very easy to underestimate the cost of maintaining any sort of online community. And if you don't do it properly, things can backfire badly - bad experiences also go viral and can damage reputations very quickly.

- How much work will you need to do to use the crowd's outputs? Crowds are made of people. Some of them will make mistakes. Some of them will even try to warp whatever results you're looking for (either for the sheer fun of it, or because they have some ulterior motive).

- How will you check their work, remove outliers and aggregate the results into a useful output? I've seen situations where it would take someone longer to check and integrate the crowd's analysis than it would take them to simply do the analysis from the ground up themeselves. Of course, you can build such cross-checking into the app, but this has costs too; it complicates app development, and requires you to recruit an even bigger crowd.

Crowdsourcing works best when you have the right public image and a good relationship with potential members of the crowd. Even then, you need to have a suitable problem, good application design and the right support model. The rewards can be high, but you need to be prepared to invest time and credibility to achieve them. Get it wrong, and the downside can be significant.

Are you willing to do that work? If you are, then maybe you should just knuckle down and apply your efforts to the original, labour-intensive task?