This bit of Internet discourse has made its rounds today in the gaming world (or at least my world on Twitter which heavily consists of gaming journalists), and I wanted to make sure you saw the two prominent posts and give my thoughts on the subject.
The Rotting Cartridge is an indie game developer working on a Game Boy-styled platformer for the iPhone called Kale in Dinoland, and they, like many other small dev teams, entered the Independent Games Festival in hopes of being recognized and garner some attention and/or praise. However, they had a poor experience with the whole event and took to where words of any sort can always find an audience: the Internet.
They went into specifics about how the IGF had judges use TestFlight—an app that tracks analytics for other app usage on iOS devices—but was unaware this would provide information back to them about when or if their game was installed and how long it was played. Out of their eight assigned judges, one didn’t install it and two others didn’t play it. Those aren’t great numbers and definitely say something towards the points brought up by the Rotting Cartridge, but unfortunately, this is where things fall apart for me.
Being angry that your game didn’t get a fair shake is fine as it’s totally justifiable, but fluffing statistics to make your point doesn’t make it right. Your sample size is eight. Your outliers will not be robust. You are throwing out one data point based on ostensibly subjective qualities as an outlier (i.e. you want to make your point) and yet you are including three zero values (one of which should not even be included) to drive your average playtime down. This immediately discredits the later stat of “a large group of anonymous gamers” averaging 30 minutes of playtime. This lacks transparency and legitimacy.
This is not how you make a valid argument.
And bringing up when the judges played the game seems wholly irrelevant. There is no substantive proof, no direct relationship that can link how close a game is played to the judging deadline to how this impacts a judge’s perception of the game. This should not even be an issue. Partaking in the process alone requires faith and trust in the judges to be impartial to any outside influences.
This is then followed up by quoting in its entirety an e-mail from GDC events head Simon Carless saying that he and IGF chairman Brandon Boyer would like to discuss any inequities those at the Rotting Catridge may have been victims of. Aside from the connotations liberally strewn about, this also, in fact, does nothing for their argument. Publishing a transcription that is saying “hey, we’d like to figure out where things went wrong and maybe fix them for you” with a reply of, well, never mind, because there was none does not have the effect that was intended. And saying you have the decency to not release the names of your judges and yet call out both Simon Carless and Brandon Boyer is not the classiest thing you could have done.
The last bit is a link to an entirely unrelated 2-year-old post from IGF judge Anna Anthropy. The Rotting Catridge and Antropy are arguing two entirely different points. The developers are saying the judges and their integrity are broken. Anthropy is saying the judging process is broken. Talk about misappropriation.
Of course, this is all being said by someone who has never been involved in the IGF or any gaming festival as an entrant or a judge. Good thing then that Jenn Frank, gaming journalist and former IGF judge, caught wind of this post from the Rotting Cartridge and decided to retort.
Frank owns up to certain points such as not playing every game in the festival (which would be ridiculous, by the way), but she does play her assignments and then some. But why does she skip some and play others? Simple: “shit was broke.” Plain and simple, some games simply don’t work. Either they fail to install, can’t run, or are so labyrinthine in setup that it’s nigh impossible to make happen.
Sometimes it’s not the system. Sometimes it’s just you.
She also makes the point that in attacking judges, you are also attacking the few proponents of the legitimacy of the industry. Everyone else in the world is indifferent or malignant towards video games and somehow attacking the minuscule percentage of people who view these products as truly viable art seemed like a good idea.
These are just my thoughts, though, so I suggest you read both parts of the argument and make up your mind on the matter. I can understand both sides as developers (especially small one- or two-man teams) want to be given their fair share of time against other bigger and better marketed titles since that is one of the fundamental concepts of indie games. However, game critics do what they do because they are passionate about video games. If they weren’t, they wouldn’t be spending almost all of their time playing them. So when a game goes unplayed, it is for a good reason: it doesn’t work.