|
Recently, I've been perusing metacritic quite a bit to check out what the critic's were saying about whatever game I'm interested in. It's always interesting to see what the metascore is vs. the user score. Initially, I've kind of generally regarded the former as having more weight than the user score because it's an average of the professional critics, so surely it must be more reflective of a game than what a bunch of random joes think? They're "professionals", they know more than me because they do it for a living and they're generally more articulate, their reviews more thoughtful, they're backed by companies, they get the game earlier, etc. Back in the days of Atari, Nintendo, SNES, what have you, when reviews were mostly gotten from magazines and small articles (I was around), I'd say this was the case. So the reasoning if they did the public well back then, they'll still do good now, right?
Reviewers of then, compared to the reviewers of today, were in a very different atmosphere. Gaming was a very niche thing, not the huge, mainstream, money market it is today; nor were reviews so easily accessible at the tip of a finger, at home, on your phone. Word of mouth was perhaps the biggest factor to a games success, so a game would actually have to be good to sell. Does that still apply today? Or do you just need good reviews to do well? Now, people just hit up IGN or Giantbomb or Metacritic because these websites employ people to review games, they should be good at it. They have the connotation of the word professional attached to them, do they need any further validation? How many people read full reviews versus a quick skim through, followed by a stern look at the score?
"This game got rave reviews from multiple critics, it must be good. I can't go wrong buying it."
Here's the problem. Say I go to a website for a review on Company of Heroes 2, but it has CoH2 advertisements plastered everywhere, am I really going to get an honest review? Is it going to look good to the publisher when a review blasts their game that they pay to have advertised there? When I go to check out some video review on Diablo 3, are major flaws going to be pointed out when the reviewer relies on Blizzard for his/her review copy? How inclined will Blizzard be to give them review copies of their future games after a less-than-stellar review? I guess you see where I'm going with this, quid pro quo happens, media blacklisting happens.
"Oh, that's the company that gave us a bad review, don't invite them to our next media event. They get no coverage."
I asked myself, why do such discrepancies exist between the metascore and user scores on games like Company of Heroes 2, Dragon Age 2, SWTOR, Diablo 3, Modern Warfare 3, GTA 4, etc.? It was easy to just brush off the user scores as troll garbage, but after playing all of these games, and many more, finding that many of my experiences reflected the user score closer, I couldn't really ignore that fact as far as actual game quality goes. Over-time, I've come to view that as a better metric than the metascore. Sure, I can continue to dismiss the user ratings on metacritic because of how polarizing or knee-jerk they can be, but I feel they balance out when you get enough of them (see: bell curve). And can you really dismiss the consumers? Aren't they actually a better representation of me? Are we, the consumer, still the ones who ultimately decide if something is good or bad? Or are we told by the critics? They can tell us a game is a groundbreaking masterpiece, but in the end, doesn't make it so. Look at Modern Warfare 3 (Xbox), which has an 88/100 metascore, but 8300+ people rated it to an average of 3.3! Are the critics this out of touch with the consumers, are the consumers confused about what they want, confused about the definition of good, or is something else going on?
Think about it, the metascore is an aggregate score from a handful of "professional" critics who may or may not be influenced by more than their opinion, while the user score is the average from thousands of CONSUMERS. You. Me. The ones that actually buy the game and aren't handed a review copy along with whatever baggage it comes with. We might not be able to hide behind a wall of rhetoric, discuss all of our points at-length, or clearly justify our criticisms, but most of us still know whether we like or dislike something!
Related links...
http://trickznstuff.net/articles/?id=1 (original)
http://kotaku.com/5892903/maybe-video-game-reviewers-are-con-men-maybe/
http://www.joystiq.com/2007/11/30/rumor-gamespots-editorial-director-fired-over-kane-and-lynch-rev/
http://bf3blog.com/2011/10/ea-tries-to-manipulate-battlefield-3-scores-in-norway/
http://arstechnica.com/gaming/2011/06/duke-nukems-pr-threatens-to-punish-sites-that-run-negative-reviews/
http://gregorulm.com/metacritic-video-game-journalism-part-i/
 
|
One of the biggest issues with video game reviews is that few reviewers actually play through an entire game, or spend time doing things that an actual player is supposed to do. Or they do, but they've already released their article a week ago so that their website could be relevant with game release day.
Hence why games like MW3, D3 and SWTOR have such good professional reviews, but so much consumer backlash. The initial game impression and the first several hours of novelty are cool, but once you actually settle down to play the games for the long-hall, the major flaws start showing.
|
I guess the issue is some publishers hand out their games to reviewers who are guaranteed to praise their game.
And there is also the problem where reviewers do not finish or play the games long enough. Some people might review RPGs based on the first few hours of gameplay but I believe you have to finish the game to be able to provide an accurate review.
Like if we based The Witcher 2 on how the prologue plays out, we can easily call it as a pretty boring game. Although the game gets a lot better than the moment the storyline starts branching. (In particular I think Chapter 2 was easily the best part of the game, regardless of which path you took).
I guess there might also be pressure on the reviewers from their superiors. It could be possible that the writer does not think too highly of the game but is under instruction from his boss to give the game a positive review.
|
I've always wondered about that quid pro issue you mentioned and wonder how common it really is. It certainly does create for some potential disingenuous reviews.
I also don't think this is exclusive to the video game industry as well. It's one of those things that consumers should hopefully always keep in mind.
|
One large problem with relying on user scores from meta critic is that many people give a game a zero or a 1 because it doesn't live up to their expectations, and thus screw up the aggregate. I think the majority of people who rate games lowly on metacrtic haven't played terrible games, just disappointing ones.
Realistically, I think both aggregated user reviews and professional reviews will always work better hand in that separated, however when forced to choose, I would much prefer the critical analysis of john walkers and tom francis's of this world, rather than the boo or hurrah mentality of bunch of pooled users on the internet.
In short, if you use video games journalism as a buyers guide, use your head and critique both the professional reviewers who benefit from sites wrapped in adds, and codphan420 who might be just sad that the ending sucked or that the left out their favourite gun from the last game.
-edit
I hope you like reading quality games journalism from totally unbiased consumers who totally don't call the developer of the game Nazi's
|
For better or worse, reviewers are becoming increasingly obsolete by the year. There are so many ways of getting a first look at the game nowadays that is not colored by personal bias or company interests that people no longer need to rely on someone's rambling and a numeric rating to determine whether they'll like the game or not.
|
I try to refrain from raging at people who use scores of any kind to judge whether or not a game is worth purchasing. I personally don't believe that you can distill someone's opinion down to a numeric value, and I implore people to read the actual reviews -- as many of them as they can. I realize that I can't really change peoples' minds on this, though, and that human beings will always look for the quick and easy answer to any one question, especially when it's regarding something they just do as a hobby or as entertainment and don't want to think too much about.
What I will do, however, is rage at people who rely on Metacritic. It's a very flawed system and it's bad for the industry.
For starters, every site has their own value range wherein a game is considered good -- 60% on one site is a solid title, and on another it's considered mediocre. Even discounting this fact, Metacritic's conversion method is vastly flawed. For instance, Giant Bomb has a rating system of 1-5 / 5. A game that GB would consider a rock solid 3/5 is given a score of 60% by Metacritic, which is not the same thing. Some other sites use A/B/B-/C, and the way Metacritic handles a lot of these values is obfuscated.
Second, Metacritic never alter their scores. Ever. Even when a site posts a fucking abysmal review full of factual inaccuracies, and then later pull and replace that review with one less full of shit, the original score remains on Metacritic, forever tainting the game's actual score. At that stage the entire point of Metacritic becomes invalid.
Third, and most frustrating to me, Metacritic uses weighted average, which means that the way individual scores affect the average is based on the popularity of the site in question.
Thing is, all of this wouldn't even bother me so much, except publishers are using Metacritic scores to determine how a developer gets paid. This is some absolute bullshit, and I admit that it clouds my opinion. It makes it hard for me to support the site, even if its abuse is not actually the fault of Metacritic itself.
Their arbitrary and non-transparent nonsense doesn't help, though.
|
1001 YEARS KESPAJAIL22272 Posts
As the guy who keeps trying to improve Recommended Game Polls, let me tell you, public polls where the rater has no incentive to vote properly are usually useless, and at best, inaccurate. Just read as many reviews as you can, 'try the game', then buy it if you like it.
|
Never ever rely on, or even read, the user score on metacritic. CoD games are a great example, reviews love them and players love them (which is why they sell so well) but there's a VERY vocal minority who hates them with a passion and they make sure every single cod game has terrible user scores which reflect reality by about 0%.
|
Very good games can get a terrible score on metacritic from the (extremely) vocal minority.
It's always some irrational people rating something either 0/10 for a bad PR move by a company or 10/10 from people who are blindly loyal to franchises.
|
On August 19 2013 16:53 Aylear wrote:I try to refrain from raging at people who use scores of any kind to judge whether or not a game is worth purchasing. I personally don't believe that you can distill someone's opinion down to a numeric value, and I implore people to read the actual reviews -- as many of them as they can. I realize that I can't really change peoples' minds on this, though, and that human beings will always look for the quick and easy answer to any one question, especially when it's regarding something they just do as a hobby or as entertainment and don't want to think too much about. What I will do, however, is rage at people who rely on Metacritic. It's a very flawed system and it's bad for the industry. For starters, every site has their own value range wherein a game is considered good -- 60% on one site is a solid title, and on another it's considered mediocre. Even discounting this fact, Metacritic's conversion method is vastly flawed. For instance, Giant Bomb has a rating system of 1-5 / 5. A game that GB would consider a rock solid 3/5 is given a score of 60% by Metacritic, which is not the same thing. Some other sites use A/B/B-/C, and the way Metacritic handles a lot of these values is obfuscated. Second, Metacritic never alter their scores. Ever. Even when a site posts a fucking abysmal review full of factual inaccuracies, and then later pull and replace that review with one less full of shit, the original score remains on Metacritic, forever tainting the game's actual score. At that stage the entire point of Metacritic becomes invalid. Third, and most frustrating to me, Metacritic uses weighted average, which means that the way individual scores affect the average is based on the popularity of the site in question. Thing is, all of this wouldn't even bother me so much, except publishers are using Metacritic scores to determine how a developer gets paid. This is some absolute bullshit, and I admit that it clouds my opinion. It makes it hard for me to support the site, even if its abuse is not actually the fault of Metacritic itself. Their arbitrary and non-transparent nonsense doesn't help, though.
Thanks for the post, I didn't even know they used weighted averages! I know of people who just discount numerical scores altogether. If not sites like GameRankings or Metacritic, as convenient as they are, what metrics should we be using?
|
People are idiots. People who write reviews are most often idiots or jump bandwagons (hype / hate).
So what do you do?
You _read_ the reviews and dont give a shit about some random number.
Oh that guy gave a 1 and his review is "shit game", well you should probably ignore that.
If someone writes "there's no pvp in the endgame, some classes are fucking OP, and it gets boring really fast after you reached Inferno" (i.e. D3) you have a better idea.
You dont care about PvP. You dont care that other people are more efficient. You didnt plan to play the game for months.
Well - this just might be a game you might enjoy. While apparently not a 10/10 it will be a decent & enjoyable time for you.
Of course if you enjoy PvP etc etc you should skip the game.
And if not - you will survive. You have to play some shit games to really appreciate the good games.
|
For the launch of BF3, EA sent out invites with a form to fill in to journalists for a sneak peak at the game close to release. On the form were questions about the CoD vs BF series. Along the lines of "Do you like BF more then CoD?" "Have you played CoD and what did u think of it?" "Have you played BF? Was it better then CoD?" those who showed criticism towards BF or leaned towards liking CoD better were not invited.
Gaming journalists are bought or strong armed into writing positive reviews just about all the time for big releases tbh. Play DA2 for 10 minutes and you see that it is not "RPG of the year, one of all time´s greatest RPGs" and there have been reported several instances of game journalists sitting somewhere on camera, reviewing a game with advertisement for the same publisher in the background. Hell i remember PC-Gamer's review of Diablo 2 years back, the screencaps they showed the player had the level 1 potions well into act 3 and 4(far into the game) so i assumed he played with godmode hack. And i was right, they admitted it a few months later citing time as an issue. How do you get an even remotely accurate review of a hack n slash aRPG, where dying is a huge part of the challenge and skill usage and character development - if you play with freaking godmode on?
Tbh anyone can be a game journalist, as long as you have a pulse and can type somewhat decently you qualify. Reviews are bogus to begin with since it's just your opinion so you can write whatever you want. That opinion is however, from what i have seen again and again - usually very 'lobbied' or how you might want to call it. It is at the very least very skewed and does not reflect how an actual consumer would experience the product.
|
I rarely ever depend on reviews these days...most of the time I buy something it's because of recommendations from a trusted forum/friends, or seeing something in a letsplay. Reviews,even user score based reviews I feel dont really cut it.
|
On August 19 2013 16:53 Aylear wrote:
Second, Metacritic never alter their scores. Ever.
good point, i've noticed that myself as well.
On August 19 2013 16:53 Aylear wrote:
Third, and most frustrating to me, Metacritic uses weighted average, which means that the way individual scores affect the average is based on the popularity of the site in question.
weighted average HAHAHAHA
i didn't know that. 
![[image loading]](http://i.imgur.com/A7WkIkB.jpg)
As far as CoH2 goes.
The "User Score" started off at 5.9 from users about 3 weeks after the game was released. This score included a few CoH2 fan-boys with no other metacritic user reviews other than a single "10" for CoH2.
Then, 3 weeks later a bunch of Russian citizens got angry about the content of the campaign. I do not want to debate the MERIT of their complaints. I only use this fact to show why the Metacritic user score went from 5.9 to 1.6 after a bunch of people posted 0s and said nothing in their "review" about the game itself. The metacritic score dropped from 5.9 to 1.6 in something like 3 days.
So...
is 5.9 truly the accurate assessement of the "users"?
or is it 1.6?
I'd go with 5.9. CoH2 is missing many features most expect from a $60 RTS game in 2013. What the game does have it does fairly well.
As was mentioned earlier in this thread; do not take my word for it. Head on over to Twitch.tv and check out CoH2 for yourself.
|
reviewers are awful shills, nothing more. crowd sourced gamer reviews arent much better though. consumer reviews are filled with stupid shit like 1stars to boycott some kind of thing in the game instead of actual reviews
|
Giant Bomb covered this a while ago and has said that you have to pick your reviewers and know where they stand. Just like everything else in the world.
Also, Metacritic is shit and their user reviews are shit. People actively review bomb that site and small groups of people make large numbers of acocunts just to bring down the score. Users scores are worthless and you are better off just asking a friend if they liked the game.
|
I haven't touched a video game magazine in years. I think I was about twelve when I made this realization. They're like picture books for little kids.
|
On August 20 2013 01:40 StarStruck wrote:
I haven't touched a video game magazine in years. I think I was about twelve when I made this realization. They're like picture books for little kids.
I use Giant Bomb, TB and a couple of other people on the internet to get my information about games and if they are good. In the era of Youtube and twitch, you pretty much can figure out if a game is good for you on your own.
|
On August 20 2013 01:24 Plansix wrote:
Also, Metacritic is shit and their user reviews are shit. People actively review bomb that site and small groups of people make large numbers of acocunts just to bring down the score. Users scores are worthless and you are better off just asking a friend if they liked the game.
I personally find the user scores to be decent if you're looking up some niche games that get ignored by the masses and shat on by the critics that don't personally enjoy the genre.
Do indeed stay the fuck away when it comes to AAA western games though.
But yeah I make my purchases based on what I hear from friends if they got the game and by watching gameplay footage, doesn't matter to me whether it gets a 6 or 10 from IGN.
|
|
|
|
|
|
EPT![[image loading]](http://i.imgur.com/Q7pHHy3.jpg)
