Over the years since I first put out the Norrathian Scrolls, I’ve read many reactions from the player base to the findings presented. On forums and message boards, some players draw out critiques that they argue invalidates the entire study. While some of these critiques are valid, their severity is often overestimated. More importantly, it may be hard for non-statisticians to understand why a valid critique may in fact not matter for many of the findings presented here at The Daedalus Project. In retrospect, I should have provided a primer to these potential weaknesses long ago, but it’s better late than never. I’m going to start with the more simple critiques and move onto critiques that appear harder to resolve.

Small sample size. Some players argue that a survey of 2000 gamers from a player base that numbers in the millions won’t show anything, but sampling is inherent to almost all surveys. For example, the Gallup Poll survey less than 1% of the total US population. A small sample size in and of itself doesn’t say anything about the representativeness of the findings. Oftentimes, people on forums will say that a sample of 2k is unreliable and then they will go on to expound their own opinion. Please call them out on their sample size of 1.

Players can’t possibly be that old. Some balk at the finding that the average age of MMORPG players could possibly be as high as 26 and thus all other results must be wrong. Oftentimes, they claim that the average age must be closer to 18-22 because they always feel like they’re interacting with adolescents in the game. But industry reports have shown that the average age of video game players in general is 30, so the MMO average from the surveys is actually lower than the overall average.

>> [Next Page]