Metacritic ruining the industry

Is Metacritic Ruining The Games Industry?

The truth about Metacritic's pervasive influence on games development and publishing.



by Keza MacDonald

July 16, 2012

Let’s say there’s a new film out in a few weeks that was filmed under the following unusual contractual condition: if the Rotten Tomatoes rating isn’t above 80% after a week or two, the writer and director don’t get paid royalties. Or say a new band signs to a record label, and the label refuses to pay a signing bonus unless their first album gets 5 stars in NME, the Guardian and Kerrang as well as an 85+ on Pitchfork.

Imagine how patently absurd this would be – and then reflect that this is exactly the situation that some game developers find themselves in. Metacritic, the review aggregator that collects scores from major sites, newspapers and magazines and swirls them together with blogs and smaller outlets to create an average number and give you an at-a-glance assessment, has become such a powerful force in the games industry that it doesn’t just impact the nature of games reviews, it directly influences development and marketing, as well as how developers are paid.

None of the following is necessarily the fault of Metacritic itself, nor the people who run it. There’s nothing inherently wrong with an aggregate review site for a quick summary of critical opinion, and it’s a useful service for film and game fans despite its limitations. What is wrong is the way that Metacritic averages are used by the games industry to determine how games are made and sold, and the negative effect that they are having on criticism.

Metacritic rates movies, TV and music as well as games, but only in our industry is that arbitrary average held in such high regard. It’s difficult to overstate how much most large publishers, from EA to Bethesda and beyond, care about that Metacritic average, and how it affects every aspect of game development, marketing and PR. Ideas – we’re talking creative ideas about what to put in a new game, here - are pitched and green-lit based on how they might affect a projected Metacritic rating. PR people are evaluated on how effectively they’re doing their jobs by looking at the aggregate number they’ve managed to ‘achieve’ for games they’re publicising.

It’s difficult to overstate how much most large publishers, from EA to Bethesda and beyond, care about that Metacritic average, and how it affects every aspect of game development, marketing and PR.

You don’t need to cast around for long to find evidence of the publishing world’s obsession with the Metascore. In 2011, Take-Two’s CEO Strauss Zelnick said that, “unlike many other entertainment business[es], ratings by Metacritic and others' reviews really can influence the success of a newly-released title… if your ratings go below a certain level, it can really hurt your ability to sell the title, and above a certain level can make a real difference in your success.” Both Telltale and Bioware have been caught messing with the user scores. EA’s Peter Moore, head of EA Sports at the time, set the FIFA 10 team a Metascore target of 90 – and then went on to say that basing success on Metacritic scores is a “slippery slope”.

For understandable reasons, it’s extremely difficult to get people to go on the record about this stuff, but anybody who’s worked in the games industry at any large publisher or as a journalist has stories. Obsidian’s Chris Avellone (perhaps inadvertently) broke the silence in March this year, revealing in a since-deleted tweet that Obsidian missed out on being paid a bonus by Bethesda for Fallout: New Vegas because it failed to hit a Metacritic average of 85. (Tragically, it currently sits at 84.)

This doesn’t apply to all publishers and all contracts, of course. But it shouldn’t apply to any. It also creates a very weird problem for games critics, who suddenly find themselves potentially responsible for a developer’s livelihood – as commenters who feel you have underscored a game will sometimes point out. This absolutely should not be a consideration in the critical process. Any reviewer who ups a score out of sympathy for the developer is not doing their job properly, and is not doing their readers justice.

This ties into another problem with judging a game’s success based on an aggregate score: the reviews listed on Metacritic come from sites with varying standards of professionalism. Metacritic’s weighting metric does give more established, respected outlets more clout, but it’s still a broken system. Eager, inexperienced writers from smaller sites have been known to give very high scores knowing that their review will appear near the top of the listings and refer traffic. This is all fine when you’re just using Metacritic as a reviews resource, but when that score determines whether a game gets a sequel, it’s a serious problem.

I’ve been told stories of PR executives working on particular games being directed to specifically target these smaller sites in the hope of raising that average score and covering up a less enthusiastic reception from the big outlets.

It also leads to some very dodgy behaviour from the publisher side of things. I’ve been told stories of PR executives working on particular games being directed to specifically target these smaller sites in the hope of raising that average score and covering up a less enthusiastic reception from the big outlets. For most people working in games marketing, Metacritic will come up during their annual review process, and a less-than-ideal average score for a game will sometimes result in a severe bollocking.

A Metacritic average undermines the whole concept of what a review is supposed to be: an experienced critic’s informed and entertaining opinion. Instead it turns reviews into a crowd-sourced number, an average. You can’t average out opinions. If you adore the new Muse album and your Radiohead-loving friend hates it, that doesn’t make it an average album. And yet this is exactly how Metacritic scores are treated by publishers. It punishes divisive games – and honestly, most interesting things are at least a bit divisive.

Limbo’s creator Arnt Jensen once said to me that “you’re never going to create anything interesting by listening to what people think”. If you’ve ever wondered why certain publishers are so conservative when it comes to their big franchises, this is why. When any change that you make to something successful might make that Metacritic average drop a few points and make everybody look to their bosses like they’re not doing their job properly, it becomes very hard to justify the experimentation.

.