Metacritic, the popular aggregator website has been making the news lately since the co-founder and editor of game reviews, Marc Doyle, spoke publicly via Skype on the A Jump, B Shoots Podcast. Metacritic is an aggregator that collects critical review scores for different types of media, such as games, music and movies. Then Metacritic applies a weighted average and posts that average in an easy to read, color coded format called a Metascore along with a blurb of the article and a link to the original review. The website has often been under fire for becoming the de-facto standard ratings system on the internet and that Metacritic ratings are incomplete as they do not incorporate every reviewing website, only the ones they choose. Looking at the overall ratings for games in general and then picking out smaller samplings of ratings particular to MMORPG’s I discovered just how flawed and incomplete Metacritics ratings are for this genre in particular.
The Power of the Meta
When one is considering on making a new purchase for themselves, especially a new MMORPG, they are immediately faced with the question, “Which is the best and what is worth the money?” For some of us that are veterans of the gaming online gaming arena, we know what to look for, who to talk to and how to go about it with the most limited risk. Yet, for some new to MMORPG’s can easily become confused and inundated with opinions and advertisements. Enter Metacritic, who single handedly collects reviews from some large, well known websites and some smaller ones as well. Now the consumer has a one-stop-shop for all skinny on a new MMORPG or an older one. Many consumers can be easily swayed by a metascore, these scores have to be correct in order for a consumer to make the most informed decision when approaching a new purchase.
Lack of Evolution
One of the problem’s with Metacritic’s scoring system for MMORPG’s is the fact that they do not incorporate re-reviews of existing games, just post the immediate reviews that were released simultaneously with the game. MMORPG’s have the nasty habit of changing over time, especially with the new trend of subscription based games adopting the new Free-to-play subscription model. Take into consideration Dungeons & Dragons Online, by Turbine Inc. was one of the first games to make the power move to free-to-play status, many changes were made to the game. The latest reviews posted on Metacritic are dated 2006, that’s quite a few years ago and Turbine Inc. has made many improvements to this game. The latest re-review on our own MMORPG.com rates this 8 for great! This stands in line with some of the older reviews on Metacritic, yet the weighted score is still a respectable 74, dragged down by some reviewers that gave D & D Online scores of 60 or lower
. Now if those other review sites and Metacritic were as insightful and intuitive as MMORPG.com and re-reviewed the new, improved Dungeons & Dragons Online, then we might see a higher overall weighted score of 80, or above. As time goes on, many MMORPG developers make sweeping changes to their existing games by ways of patches and expansions; many have reviews of their expansions, yet not applied to the original games score. Re-review and averaging in new scores for new content for MMORPG’s would be something for Metacritic to look at.
The Darkfall Dilemma and the Case of the Missing Scores
When looking through Metacritic for scoring of popular MMORPG’s I discovered some games that were missing metascores and that disabled the game from showing weighted average. Darkfall was one of the games that I found to have 2 scores missing from their average and one of the reviews used was the controversial Eurogamer review
that has been retracted and issued a re-review. To bring it up to speed in short form: a game reviewer from Eurogamer gave Darkfall a score of a 2; Tasos Flamboras, associate producer of Darkfall Online, looked through log records and announced that the reviewer was logged in for a total of 3 hours with most of that time be in the character creation screen. Eurogamer then issued an apology and offered a re-review that Tasos turned down. Eurogamer then re-reviewed Darkfall anyway and issued it a 4. You can read both reviews on the Eurogamer website. I also know for a fact that some of their accepted websites have issued reviews for Darkfall and those haven’t been added to the average. This inconsistency has also affected a game such as Aion as the new Assault on Balaurea expansion hasn’t been issued any additional review scores to enable it to have a proper weighted average. These types of holes need to be filled before Metacritic announces to the game review industry what it needs to review, because some reviews are not added to the average and that gives consumers an incomplete picture of the product.
WE do need to review EVERYTHING
One thing I have to agree with Marc Doyle (picture right) that he mentioned in the interview with the A Jumps, B Shoots podcast is that he said, “We need to review all the S*&t.” I tend to agree with him for the fact that some games listed here and are functional MMORPG’s such as Xsyon and Wurm Online don’t even have entries on Metacritic. The game Valkyrie Sky by Yolim Entertainment has an entry, but not one single review listed. It is important to give a complete picture of ALL games in order to have a correct weighted average. Not to say that any of these games are bad, but it sometimes takes one game to either have the worst score to determine the average. Due to this, Metacritics average scores page is incomplete, inconclusive and the review sites such as our own MMORPG.com need to fill in those blanks. There is also a side column for user scores, we as players, need to step up and look at everything as a whole. Endure the hours it takes to give something a real review, post an honest point of view and fill in the blanks with some professional content where the pro’s fall short.
Fallen Earth IS NOT LIKE Fallout 3
MMORPG’s, MMOG’s and online games in general need their own category listed on Metacritic. If you look through all the MMORPG’s that are listed on Metacritic, they all fall into the same category: Role-playing PC games. That is a correct assumption, that’s exactly what MMORPG’s are in essence, but they are a vastly different. MMORPG’s count on players to be part of the content, while single-player games count on only one player to interact with the AI of the game. MMORPG’s evolve over time and change, while a single player game might have some additional DLC content, more often those types of games have the developing studio release a sequel and stop development for the previous game. Single player role-playing games more often than not have a beginning and an end, meanwhile MMORPG’s are an ongoing process of character development and do not ever end, unless the player stops paying for the subscription and paying for the game. That brings up the last point that most MMORPG’s have a subscription cost or an item mall, some type of method that the player will have to pay additional money to play the game. This is a big difference between both the single player and online genre. One type of game will continuously require additional investment of the player to enjoy the game, meanwhile one does not. This is really important when a player is making the decision to buy into a new online game as they need to know they are getting the most for their money.
If someone chooses to use Metacritic as a reference guide, then they should be able to compare all the games of the same genre together in one category. A consumer has the right to see complete and updated metascores for game reviews, since as time goes on many online games go through deep changes. All games in the same space need to be reviewed so the consumer can have a proper average, not just a top heavy average weighted by all the most popular games. If Metacritic was able to implement all of these changes then it would give that their aggregated scores as well as the game review industry more credibility as a whole. As a consumer, I want to see the whole complete picture, all the pretty and the ugly too, in full, when making my decisions.