And from Gamespot (8.5/10): From Eurogamer (8/10): And GameSpy's rather odd verdict (4/5): Interestingly, every review accurately mentions AoC's flaws and qualities, they all have a caveat -- that the game is a work in progress -- and they all state that Funcom is working hard on patches. In fact, they're all good at representing what the game is and what it could be. But they all fall over as reviews. These reviews tell gamers to go out and play it (greater than 80% is a strong recommendation in my book), but AoC was poor when it launched and it's still poor now, which makes these reviews poor recommendations at best. As reviews, they simply don't work. Why?Mainstream games journalism is a perplexing little industry, what with those mysterious scoring systems, PR-saturated stories, and the baffling amount of screen time given to previews. But perhaps the strangest phenomena is the industry-standard MMO review model. Lets take a quick look at how Age of Conan, which is now becoming synonymous with failure, somehow scored overwhelmingly positive reviews.
And before you read the verdicts, just take a quick moment to reflect on what a review actually is, i.e., a recommendation on whether you should invest in something -- money, time, or both -- delivered by an independent expert.
First, here's a snip from Jolt UK (8.4/10): "Provided you’re not in a race to see the end of the game, Age of Conan comes as a highly recommendable MMORPG. It’s bold, brash, visceral in nature and for the first 20 levels it’s highly compelling. It’s also unfinished, somewhat unpolished and requires a fairly hefty PC to run."
"You'd do well to look past these imperfections, though, because Age of Conan is the most brutal and immediately satisfying MMOG on the market, thanks to its unique slant on combat, resonant quest writing, and uncompromising maturity."
"In the next six to twelve months, Age of Conan will live or die on how successfully Funcom can address these issues. Right now, though, the game as it stands is a rough diamond - very good, bordering on great thanks to a regular stream of comprehensive patches from Funcom."
"The game's major problems stem from missing or incomplete PvP and PvE content, not any lack of solid infrastructure or poor game design."
First, most reviews gave Funcom the benefit of the doubt (something I've also been guilty of in the past for my pet developers -- in fact I'm probably guilty of most of the things I'm criticising here), which turned out to be an epic mistake. What voodoo did Funcom pull to enjoy such a privileged position? How did it get its halo? Funcom misrepresented its product, under delivered in very important areas, and then had the gall to take money off people. There doesn't seem to be any good reason why Funcom should have been trusted, yet they were. If Funcom had a history of delivering on their promises and making cracking good MMOs, then you can absolutely make a prediction based on that -- just be upfront about why.
But more importantly, one-shot reviews just aren't appropriate for an MMO. To accurately recommend a MMO a review must make a call on the future, yet there are so many things that can change from month to month. Any review made at a point in time -- and for all time -- is too much of a gamble. MMOs aren't final, complete products, they're services, and they should be reviewed on an ongoing basis.
So here's a suggestion. If you're going to review a MMO at launch, be honest about it. If it's crappy and not worth the investment at the time of review, then say that. Most MMOs will probably get a poor rating at first, but that's okay. That's what a review should be saying. But don't stop there. Instead of endless caveats that a developer may or may not fix the game in the future, place a footnote at the end that states the game will be reviewed every month, or week, or after every patch, or at some stage appropriate for the game.
This may not be perfect, but it's a far better solution than the one-shot review model. Gamers deserve more appropriate editorial for MMOs and a representative gaming press that provides more accurate and fair recommendations. Anything less is a disservice.
31 August, 2008
Broken Games Journalism: The MMO Review
Posted by David at 12:32 AM
Tags: age of conan, mmo
Subscribe to:
Post Comments (Atom)
Posts
14 comments:
If you ask me, the reviewers are pretty credible sources of information. Whether or not you like the game isn't really under their control. They did their job, they gave you a snap shot of the game.
You have money and decide what to spend it on, so funcom didn't tell you they were going to kill your parents if you didn't buy their game. You have the ultimate yes or no, so you should have exercised more discretion when you bought the game if you even had an inkling that you wouldn't like it.
I'm sorry you didn't like the game but it's not that big of a deal. The time you did spend in the game you had to have enjoyed a little bit, otherwise you would have put it down after the first few levels. If that were the case, then you don't really have a leg to stand on besides the path to tortage. So you sort of did get what you paid for, you paid probably $100 (if you're in the US) total for the game and sub and if you're like me, upgraded your hardware (which would need to be done eventually anyways). So for all that, you got some solid hours of gaming in. All in all, you didn't do too bad, you get what you pay for.
Interesting thoughts, Early, but with respect, I disagree with your apologist nature to games reviewers, and to AoC.
I agree that the ultimate decision to buy a game rests with the individual. But that individual is swayed by the advice and recommendations of others - hence the massive industries that have grown based around 'reviewing' products of one kind or another.
Simply saying "you have the ultimate yes or no, so you should have exercised more discretion when you bought the game if you even had an inkling that you wouldn't like it" is like apologising on behalf of a car reviewer who says a poorly-built car is as good as a Rolls Royce, or a Mc Donalds burger is as good as a 5-star meal.
A reviewer is delivering more than just a subjective opinion on something. They have a job - a responsibility - to be analytical and mindful of the potential audience and give them appropriate advice. A good review should tell you whether you are likely to enjoy a game or not, and thus, whether you should buy it. A bad review just tells you whether the reviewer liked it or not. An even worse review paints a misleading picture of the product.
As such, Dave's criticism of games reviews is not because they held a *subjectively* different point of view, but because they *objectively* misrepresented the game through their very high scores.
The game reviews Dave mentions give me the impression the reviewers didn't actually play the game in its entirety either. That's like reviewing a car by driving it around the block. And that's just lazy and unprofessional.
I think Dave's ultimate point is that, as gamers, we expect more. We expect a high calibre of critical reviews that have exacting standards and that call a spade a spade.
At the moment, many games reviews sites are guided by PR and marketing dollars, sponsored trips to developer studios, conventions etc and have weak journalistic integrity. And that's not good enough. We demand more - and it's only through vigilance that we can remind the games journos that they need to up their game, so to speak...
no you dont get what you pay for with this game. It hyped so much , and delivered so little. you really cant judge Conan based off a few levels. you dont see all the broken mechanics till i'd say your atleast lvl 50 and above, and by then , you've waisted time and money.
All thought no MMO is perfect at launch, most atleast deliver whats promised on the box, not make you pay to play beta. These people need to be sued for false adevertising, and should give their remaining players free play time until the game is atleast stable.
This company makes a promise for a june patch , and we are now heading into september without the content promised in June.
Promised a Mature game, Ive seen pg-13 movies with more blood gore and sex in them.
Customers are not getting what they paid for. Only reason I still play is because ive drop money on the box, a new graphics card, and my sub and am to cheap to throw that cash away.
all in all , the issues with AoC is that not that people do not like it, its that people do not like being lied to and are fed up with funcoms crap.
bugs that severely affect gameplay ( like gem duping) took weeks on end to fix. which is ridiculous. or watch out you dont get banned for breaking line of sight, or having a gm see you use the terrain to your advantage. thats right, players are getting banned for things the game boasted about players being able to do.
AoC has the potential to be a great game, Funcom just needs to deliver on their promises.
A credible review would be upfront and honest, saying the game is incredibly broken ( not just missing some stuff here and there) have a look at the AoC forums and read for yourself.
and i know this sounds like a huge rant and is a reply to the first comment and really has nothing to do with the OP's post. so to be on topic I totally agree that game reviews should be on going, and not just done once.
Age of Conan is a huge failure.
You can take a look at their forums, the game has become a joke, a testament to poor game design which we havent seen in ages.
Never before has an MMO been this poorly thought out.
Its as if the developers purposely fooled the fans into thinking:
a.) We have boobs, gore and foul language! b.) We have working epic seige combat!
c.) We have amazing PVP and PVE content!
d.) We support DX10!
When infact they didnt at launch, and still dont.
They sold almost a million copies on LIES.
And furthermore, they polished the very start of the game (Specifically Tortage) to purposefully fool game reviewers. They know that a professional game analyst will not have the time to play 80 levels of content, so they polish Tortage up, and leave the rest of the game as crap.
Pathetic really, and I hope the industry learns from this.
I also hope the consumers learn from this, and we can run FuncoN out of business, for good.
Interesting thoughts.
I can agree and disagree with you, to a certain extent.
MMO's, especially of the RPG variety, need to be reviewed with the way they at the review, with thoughts on promised changes, and then of course potential. To go further, RPG's like AoC should have thoughts about how good the game is at certain levels.
Since AoC is in the spotlight here: I'd rate it
Overall: 5/10 pretty bad
Levels 1-40: 8.5/10 very polished
post 40: 3/10 obviously incomplete and needing work
Potential: 9/10 overall good system. great world, amazing graphics.
can that "potential" be realized?: 3/10 doubtful at this point that it will be able to keep a solid player base.
@early
"They did their job, they gave you a snap shot of the game."
That's not their job. Their job is to recommend whether AoC is worth someone's time and investment.
@anon
I see what you're saying re: commenting on a MMO's potential, but it relies too much on the prescience of the game reviewer. They become a speculator, where their recommendation is based on the 'odds' of a game getting better or worse in the future. That so many AoC reviews appeared to get it wrong shows how difficult this is.
Further (as you implied) a game's potential can change anyway, so even if it did influence the recommendation, you would still need subsequent reviews.
Alright, here it is. btw I have a level 77 guardian on one of the lowest server population in the AOC game (pvE). I also rerolled and pvp and have a level 18 POM. sorry about all the abreviations. Here goes this game is somewhat broken. but addictive. I played WOW up until maybe level 25 until my internet died on me one day. I have to say this AOC game is much better. If you play it u better have time on your hands. I bought it when my wife was away for six months. I would rate it a 10/10 for potential. yes there is memory leaks. yes there our bugs. but if the community sticks them out and the losers and complains leave this game may just stand teh test of time. even if it is only a cult classic game. i am down with the cult at least till my wife comes home.
further more... alot people are bitching because they love the game. think failcome knows their product shines like a diamond but does not cut glass. its a big fat forgazzEE. like in the show donnie brasco lol. the thing is people bought it. some will stay some will go. i will stay. but lets look at this. if they fixed up major areas. lets say they had no to very few memory leaks. balanced out the pve and pvp so all sides were content to the level content can make itself. u would have the best mmo ever created the question is will funcom deliver or will failcom shine like the forgayzee it is now.
Quick comment. This post isn't about AoC, as such. It's about MMO reviews.
I don't doubt that some people will like AoC, despite the bugs etc. But the point is that the game reviewers failed in their duty to provide an accurate depiction of the state of the game. And that has let a lot of other gamers down.
And Dave's final point is that for this and other reasons, MMOs should be reviewed differently to the way they are today.
If you like AoC, I bare you no grudge. I don't though. And I resent that the AoC reviews I read made it sound to me like something worth buying. (As it happens I played the beta and had a dire premonition about the game's future, so I never bought it when it went retail.)
Nice post guys; have pointed to it from MMOCrunch and Geek Critique. I think it's a topic well worth talking about.
I fundamentally disagree with your opening premise:
"a review actually is ... a recommendation on whether you should invest in something -- money, time, or both -- delivered by an independent expert".
A review should, first and foremost, be an entertaining piece of prose. The review is the end, not the means to an end. It should tell the reader as much about the author of the review as it does about the product being reviewed, and above all it should reward the time spent reading it in and of itself.
Consumer advice is an entirely different matter from a review. I agree that in gaming these two functions have become conflated, but they shouldn't be.
As for MMOs, if anyone actually wants to know what a new MMO is like before they spend money on it, it is very easy indeed to find out without reading reviews. Join the beta yourself, either early in the process or in the almost inevitable "open beta" stage; browse the relevant forums/blogs when the NDA drops a few days before launch;have some patience and wait a couple of weeks after launch, reading the official and fansite forums the while.
@Bhagpuss
A review, by definition, is an evaluation, and in the case of a consumer product, a recommendation on whether one should invest in it. Of course, a review can *also* be entertaining to read, and knowing as much as possible about the author is certainly valuable to the reader, but these factors are not "first and foremost" what a review is for.
If a review should be, first and foremost, entertaining prose - then the bad game reviewers (if not the terrorists) have already won. For what, then, is to distinguish an entertaining bit of prose about wine or cars from a review about wine or cars? Or games, for that matter?
So I'm going to stick my neck out and say a necessary (but not sufficient) condition of something being a review is that it makes an evaluation about its subject. In the context of a purchasable product (as opposed to, say, art) the evaluation is to guide a potential purchaser of that product.
Pretty prose about a game without evaluation is merely prose about a game. There's nothing wrong with that - in fact, I encourage it - but it's misleading to call it a 'review'.
And I agree that there are ways to find out about a game without reading reviews. But therein lies one of the reasons we have reviewers - they do the hard yards for us.
For example, I used to review computers and peripherals. The reason people paid to read those reviews is I actually spent the time *using* those products, as well as running tests on them. Many of those tests were public domain, and people could go to a store or a mate's place and them themselves. But that would take more time than most people devote. So I'd do the legwork for them.
The other thing a reviewer (supposedly) brings to the party is *expertise*. In the context of this discussion, a game reviewer should have experience playing games of a similar ilk and be equipped with an understanding of the gaming concepts that make games enjoyable to certain people. I find this skill, in particular, lacking in many games reviews today.
I mean, how many reviews have you seen that kick off with "I don't normally play RTS/sports/MMOs etc, but...". That's usually where I stop reading.
And looking at the length of this comment, this is where I'll stop writing...
The problem with MMO's that they cannot be reviewed (not in time at least) there are just to many variables. Playing a specific class can make e huge difference. A MMO can be enjoyable while playing until you hit a specific level (and turn into a grindfest). Also the amount of people playing can make a big difference.
If you take into account that the timing of an ideaal review is in the week the game is released, you will understand that reviewing a MMO-game is impossible.
The development of a MMO doesn't stop after launch so a game can be broken while it is released. Six months later the situation can be completely different. No one can tell which direction the game is going after release.
Post a Comment