When Microsoft released the product strategy for Office 2010, the stakes were higher than ever. Months before they actually released it, however, they began crowd sourcing software, enabling 9 million individuals to download the beta version and provide feedback. This allowed them to receive 2 million individual insights and comments. It also gave them an understanding of how their product was used, thereby making sure that people would be able to get the most out it. This was the largest crowd sourcing project of its kind to date.
Why Crowd Sourcing Software Is the Way Forward
If you wish to understand why testing by the crowd work, you have to first understand the biases that test managers and testers around the world have. This is commonly described as the “curse of knowledge”, a phrase coined by the Journal of Political Economy in 1989. What it means is that once someone is an expert in a certain subject, they can no longer look past what they predict or know. Hence, thinking outside the box becomes nearly impossible, and true innovative thinking has been lost.
Elizabeth Newton, a graduate psychology student from Stanford University conducted an infamous experiment to prove the curse of knowledge phenomenon. In her test, she created a game where people would be assigned a role of either listener or tapper. Tappers would be assigned a famous and easy tune, like Happy Birthday, tapping it on the table. Tappers were asked how likely they thought it was that the listeners would guess the song, and they believed it would be 50-50. In reality, however, after tapping 120 songs, listeners only guess a total of three songs, equating to 2.5%. What this proved was that tappers will always hear the tune they are tapping inside their own heads. But the listeners heard nothing but some strange code. It proved that people find it impossible to consider someone doesn’t know something, when we do.
This experiment applies strongly to software testing, or any form of testing, as well. When software is developed, it is put through a range of tests that developers feel are representative of how end users will actually use the software. But in reality, they may use it completely differently. It is impossible for someone who knows the software to imagine how people who don’t know it will use it. The result of this is that various elements of the program do not get tested at all, leading to debugging, customer escalation, system crashes and malfunctioning, and generally unhappy customers.
By asking the crowd to test packages, however, all these issues are avoided. It ensures that product developers know that their software really works for the people they want it to work for. It means that it is truly viewed by the very people who will use it once it is launched. This information is invaluable and cannot be obtained without actually putting things out there and listening to individuals from a range of backgrounds.