Back in January of 2011, I wrote a piece on Games and Geekery discussing the length of time it would take for someone to really get a sufficient impression of a game to have an opinion on it. I had no answers then, and I still don’t really have any answers now, as I do not think one could definitively quantify for everyone else how long someone would need to play an online game to give that MMO a worthwhile review or impression.
I’m mentioning this now because I’ve spent the past month reading the four-part review of Darkfall: Unholy Wars by George Dimmock and observing the comments of people on MMORPG.com and on the Darkfall forums. It seems that, while new games have sprung up since 2010, the talk about spending adequate time to play an MMO for review still occurs.
Let’s discuss that, shall we?
The Bell Curve?
The “bell curve of play” is a term I used back in 2011 to refer to “the idea of writers needing a predetermined number of hours or a set point in the game met to achieve what could be considered an acceptable impression.”
Back then I found it preposterous to institute something like this for every game, or for a game company to “suggest” having a minimum number of hours actively played before one could properly discuss a game. I still think this: people who play games pick up the nuances of a game at different speeds, so it seems counterintuitive to force a requirement just to have an opinion.
The Unholy Wars Review
What I noticed when it came to our Darkfall: Unholy Wars review was that people seemed to be taking offense at the amount of prowess gained by Mr. Dimmock or the total game time he spent in Darkfall to make the review. As I understand it, he spent approximately 20 to 30 hours in the game over the course of his review period and acquired around 2000 or so prowess, which he primarily used to enhance crafting and gathering skills.
)
In the comments of the third in-progress review article, Itchmon perhaps phrased most diplomatically what Darkfall players were thinking when they read the review. Itchmon wrote,
“If the site's review contains no group play and only 2k prowess of xp, all from crafting, that's not really an in depth review- its more of a first impressions. A nicely written one but nonetheless a first impression. I certainly don't play a ton but I have 5k prowess and some experience pve grouping in non safe area with clanmates.
Tldr version... your reviewers should review after playing the game the way players play the game...”
I’ll give Itchmon the benefit of the doubt when it comes to seeing the final review piece, but the context that paints the last line of the comment is telling: reviewers should review after playing the game the way players play the game.
I do not think that is how reviews work, and the idea that a particular way of playing a game is needed to make a proper review is a notion that needs to be corrected to some extent.
For Whom the Review Tolls?
I can assure you, it tolls not for thee, hardcore gamer.
Gamers who read reviews hoping to see their opinions of a game justified (or really, anyone invested significantly in a product looking for validation of their choice) are looking at game reviews in the wrong light. Reviews are meant to inform prospective buyers of a product or service of the things they can expect from that product or service based on the word of an individual who is, in ideal situations, looking out for the consumer’s benefit.
Taking into account the shifting nature of an online game, reviews of MMORPGs are generally not aimed towards people who’ve already purchased an MMO or subscribed to it. Instead, they aim to provide people who’ve heard something about a game with more information to provide them with a better idea of the strengths and weaknesses of a product.
Allegations of reviewers being corporate shills aside, the best MMORPG reviewers (in my opinion) speak plainly about their experiences with a game. They offer facts and opinions relevant to how they gauged their experience with a game but they also know that, with the shift in the gamer demographic and with the occurrences in that reviewer’s personal life, their review will not satisfy everyone. More to the point, a review by one person is not meant for consumption by all gamer types.
Looking at the four-part Darkfall: Unholy Wars review, the more hardcore players of Darkfall likely latched onto his prowess rate and time played. Mr. Dimmock, however, has his own life and he played the game as his life allowed him to properly do so. People whose lives allow for casual play may likely see more of what they expect to experience if they play Darkfall by reading the review.
Here’s the problem that faces Mr. Dimmock and every other game reviewer out there, and this is a problem that I salute game reviewers for being willing to take heat for. To speak enthusiastically about all aspects of a game without having ever experienced it is disrespectful to people who trust a reviewer’s judgement. For a reviewer to remain silent on one’s experiences because a reviewer’s play style doesn’t fit the expected player demographic of a game, however, is cowardice, and such cowardice also disrespects anyone looking for a well-formed opinion. Finding that middle ground of offering your experiential opinion with information to back up your claim is perhaps one of the tougher jobs a games media writer has to face today.
)
Reviewing Games, Reviewing Jam
Back in 2010, Jamie Madigan of The Psychology of Games had a brilliant piece on the psychology behind game reviews, likening it to reviewing jam.
In his piece, Madigan asks the following:
“When video game reviewers ruminate over the merits of a particular title, they are often asked to consider standardized lists of features – graphics, sound, fun factor, multiplayer, value, extendibility, controls, and so on. Should they always try to analyze decisions across every possible variable? Is that the right way to review a game?”
He continues by telling the story of two researchers who were intrigued by a Consumer Reports ranking of 45 strawberry jam brands by professional food tasters. They recreated the test using the 1st, 11tth, 24th, 32nd, and 44th best jams from the report and had college students rank them. The control group, who was not asked to explain why their rankings were the way they were, ranked the jams closely to the professional taste testers’ rankings. The experimental group, who was asked to write reasons for their ratings, “started focusing on factors that didn’t really matter,” did worse than the control.
Madigan explains the theory behind it: people who are asked why they they preferred something over another feel “obligated to include the most salient (that is, apparent) and plausible explanations. Even if we would have otherwise ignored them.”
He adds, “Jelly or game review guidelines that require us to over analyze our decisions or check them off against a standardized list of factors (graphics, sound, etc.) can exacerbate this limitation and lead us to consider what should be irrelevant information when making our ratings. This corrupts the rating process and takes us farther from our ‘true’ feelings or evaluations.”
)
Reviews, Experiences, and Honesty
Reviews aren’t perfect, and one review or opinion on a topic isn’t always going to be helpful to a given person. While having a standardized list of factors to consider in a review can force reviewers to consider things they don’t feel are important for their impressions, they are also helpful as a guide in introspection for those who want to think about a game.
With some guidelines for reviewing in hand, MMORPG reviewers who are honest about what they did while playing will likely be the most effective in giving an well-thought out, yet personally relevant opinion.
Using both well by being open about what one likes and dislikes in games can lead to a hybridized sort of review, allowing someone who’s experiencing a game for a review to further think about why he liked certain factors and then write about them truthfully.
Sure, that sort of review still won’t please everyone, and people will still claim you’re a game company shill. That said however, people who are receptive to a game reviewer’s openness with his experience with a game will likely find themselves learning a lot about whether a reviewer shares their way of approaching a game, giving them a better lens with which to view something they’re on the fence about buying.
Victor Barreiro Jr. / Victor Barreiro Jr. maintains The Devil’s Advocate and ArcheAge columns for MMORPG.com. He also writes for news website Rappler as a technology reporter. You can find more of his writings on Games and Geekery and on Twitter at @vbarreirojr.
- Part 1 of the MMORPG.com Darkfall: Unholy Wars Review
- Part 2 of the MMORPG.com Darkfall: Unholy Wars Review
- Part 3 of the MMORPG.com Darkfall: Unholy Wars Review
- The Final Score on the MMORPG.com Darkfall: UW Review
