Board 8 > And this is why Metacritic and review scores are bulls***.

Topic List
Page List: 1, 2
WazzupGenius00
03/15/12 8:13:00 AM
#1:


external image

Also, shame on Bethesda for the way they did this.

--
[NO BARKLEY NO PEACE]
If you can't slam with the best, then jam with the rest.
... Copied to Clipboard!
SHINE GET 64
03/15/12 8:17:00 AM
#2:


that's so terrible

--
http://i.imgur.com/4KfnE.png
http://www.yourgamercards.net/trophy/a/CaptainRockman.png
... Copied to Clipboard!
paperwarior
03/15/12 8:19:00 AM
#3:


Basing payment on an aggregator of already-arbitrary review scores is definitely bull****.

--
I had a nightmare where Ulti forced me to eat brain matter. ;_;-MegaWentEvil
... Copied to Clipboard!
Achromatic
03/15/12 8:20:00 AM
#4:


They agreed to the contract, it's fair.

Now the review scores may be bs.

--
http://img.imgcake.com/Chris2pngas.png
SuperNiceDog is a fine Guru Champion!
... Copied to Clipboard!
EndOfDiscOne
03/15/12 8:20:00 AM
#5:


XD

Review scores really are serious business

--
http://img.imgcake.com/fitgirljpgta.jpg
... Copied to Clipboard!
KCF0107
03/15/12 8:21:00 AM
#6:


I don't even think that review scores have that big of an impact on sales figures. There always seem to be dozens and dozens of games each year that are critically acclaimed but don't sell well, resulting in a loss for the project or a small profit

--
http://img.imgcake.com/kcftabletpnget.png
... Copied to Clipboard!
Mac Arrowny
03/15/12 8:23:00 AM
#7:


To be fair, it would've easily gotten an 85 if Obsidian's games weren't all so glitchy.

--
All the stars in the sky are waiting for you.
... Copied to Clipboard!
RappinHobo9292
03/15/12 8:28:00 AM
#8:


Wait the South Park RPG got canceled? God damnit.

--
Will not change sig unless the Sens win the Stanley Cup. Started 2/19/07
http://paranatural.net/chapter-1-page-1/ - Read Paranatural
... Copied to Clipboard!
KCF0107
03/15/12 8:30:00 AM
#9:


The article that I read made it sound like the South Park game only had some of the staff removed, not the entire project being scrapped

--
http://img.imgcake.com/kcftabletpnget.png
... Copied to Clipboard!
The Real Truth
03/15/12 8:30:00 AM
#10:


From: RappinHobo9292 | #008
Wait the South Park RPG got canceled? God damnit.


No, the next-gen Obsidian game got canceled. South Park is hurt by this though which is incredibly irritating.

--
GameFAQs isn't going to be merged in with GameSpot or any other site. We're not going to strip out the soul of the site. -CJayC
... Copied to Clipboard!
TheKoolAidShoto
03/15/12 8:32:00 AM
#11:


Hey, any of you guys seen the metacritic "trophies" in EA's studios? S*** is crazy

--
You know who's sexy? Black Turtle, the 2010 Games Contest Guru Winner, that's who!
... Copied to Clipboard!
EndOfDiscOne
03/15/12 8:36:00 AM
#12:


The Real Truth posted...
From: RappinHobo9292 | #008
Wait the South Park RPG got canceled? God damnit.
No, the next-gen Obsidian game got canceled. South Park is hurt by this though which is incredibly irritating.


This better not be the Wheel of Time game.

--
http://img.imgcake.com/fitgirljpgta.jpg
... Copied to Clipboard!
voltch
03/15/12 8:36:00 AM
#13:


So some reviewer might have cost a few people their jobs huh.

--
Dr voltchball - best in the world.
... Copied to Clipboard!
WazzupGenius00
03/15/12 8:39:00 AM
#14:


well it wouldn't be the first time reviewers cost people their jobs, but it would probably be one of the first times they did so while still giving said people's game a high review score.

--
(10:17:09 PM) Ed Bellis: i think pac-man just keeps repeating, for instance
(10:17:26 PM) ertyu: ya well tell that to billy michtlle
... Copied to Clipboard!
SHINE GET 64
03/15/12 8:40:00 AM
#15:


Guess who gave it a 4/5 thus lowering the average?


external image

--
http://i.imgur.com/4KfnE.png
http://www.yourgamercards.net/trophy/a/CaptainRockman.png
... Copied to Clipboard!
WazzupGenius00
03/15/12 8:42:00 AM
#16:


part of why Metacritic is garbage. A 4/5 automatically counts as 80/100 to them.

--
SuperNiceDog is a vicious breed and should be banned from further Guru contests.
... Copied to Clipboard!
EndOfDiscOne
03/15/12 8:45:00 AM
#17:


SHINE GET 64 posted...
Guess who gave it a 4/5 thus lowering the average?


external image


He's just bitter because he lost his job over a review score

--
http://img.imgcake.com/fitgirljpgta.jpg
... Copied to Clipboard!
Ry Senkari
03/15/12 8:46:00 AM
#18:


Guess that's the reason this trope is so prevalent:

http://tvtropes.org/pmwiki/pmwiki.php/Main/HePannedItNowHeSucks

--
Kid: She wrote a bad word!
Jade: I write what I feel.
... Copied to Clipboard!
DigitalIncision
03/15/12 8:48:00 AM
#19:


This is fair in every way, really.

--
Digi
Shark swims ashore in new jersey ... wonder where he bought it.
... Copied to Clipboard!
baron von toast
03/15/12 8:50:00 AM
#20:


Review aggregators are dumb, but not because of this. The only dumb thing here is Obsidian.

--
.
... Copied to Clipboard!
PrivateBiscuit1
03/15/12 9:30:00 AM
#21:


This is with a lot of companies. It's not like it's just this company. It makes me very sad to see something like this happen, and if game developers are just going to be this anal about review scores, I'd much rather just see everyone get a great score on their game to prevent crap like this. You can barely even trust reviews any longer, so you might as well just go ahead and give them all high scores to help them out.

--
Xbox GT: PrivateBiscuit1
Down with Zhang. Let there be Biscuit!
... Copied to Clipboard!
pjbasis
03/15/12 10:15:00 AM
#22:


From: WazzupGenius00 | #016
A 4/5 automatically counts as 80/100 to them.


Uhhhhhhh

--
http://i498.photobucket.com/albums/rr345/Rakaputra/pjbasis.png
SuperNiceDog - 1, pjbasis - 0
... Copied to Clipboard!
th3l3fty
03/15/12 10:17:00 AM
#23:


believe it or not, 4/5 is not equivalent to 80/100 when your scale involves rounding

--
thelefty for analysis crew 2008 imo -tranny
I have a third degree burn in flame-o-nomics -Sir Chris
... Copied to Clipboard!
WazzupGenius00
03/15/12 10:19:00 AM
#24:


my point being that a 4/5 from most sites that use that scale does NOT mean the same thing that an 80/100 means from sites that use a 100-scale.

Mathematically, sure, the two numbers are equal, but the meaning behind the scores is not.

Plus there's the fact that even some publications that use the same number scale assign scores differently. 8/10 from Eurogamer means a lot more than 8/10 from IGN.

--
http://i49.tinypic.com/anio39.gif
... Copied to Clipboard!
pjbasis
03/15/12 10:21:00 AM
#25:


From: WazzupGenius00 | #024
8/10 from Eurogamer means a lot more than 8/10 from IGN.


And what do you suggest they do in cases like this?

What else is 4/5 equivalent to?

It's an aggregate site meant to average a bunch of data. The conclusions you draw after are up to you.

--
http://i498.photobucket.com/albums/rr345/Rakaputra/pjbasis.png
SuperNiceDog - 1, pjbasis - 0
... Copied to Clipboard!
TRE Public Account
03/15/12 10:23:00 AM
#26:


Why does either side want to tie something like this to review scores and not something like sales? It would prevent something like handing out a bonus for a game that bombs saleswise despite high reviews as well as not handing out a bonus in the reverse scenario.

--
SuperNiceDog was not so nice to my bracket.
Congrats to SuperNiceDog for winning the Rivalry Rumble Guru Contest!
... Copied to Clipboard!
WazzupGenius00
03/15/12 10:24:00 AM
#27:


From: pjbasis | #025
And what do you suggest they do in cases like this?


they stop existing because Metacritic is f***ing dumb

--
http://i49.tinypic.com/anio39.gif
... Copied to Clipboard!
pjbasis
03/15/12 10:28:00 AM
#28:


You're mad at the conclusions and actions people take because of metacritic.

The data is useful though.

As useful as the scores it's based on anyway.

--
http://i498.photobucket.com/albums/rr345/Rakaputra/pjbasis.png
SuperNiceDog - 1, pjbasis - 0
... Copied to Clipboard!
XIII_rocks
03/15/12 10:28:00 AM
#29:


Aren't the majority of reviews pretty much bought and paid for anyway? Jeez

--
"To be fair to Herman Cain, we can't all be as wise as Slowking when he's wearing the Shellder of knowledge. Look it up, b****es." - Jon Stewart
... Copied to Clipboard!
KamikazePotato
03/15/12 10:29:00 AM
#30:


Huh. I kinda hate Bethesda now more than anything else, really.

--
Black Turtle did a pretty good job.
... Copied to Clipboard!
DigitalIncision
03/15/12 10:31:00 AM
#31:


From: KamikazePotato | #030
Huh. I kinda hate Bethesda now more than anything else, really.


Why? What do you want them to do? Give them more money than the contract specified? Give them money so they don't have to fire people?

--
Digi
“They took out my left ventricle and atrium,” James said half-heatedly.
... Copied to Clipboard!
CalvinbalI
03/15/12 10:31:00 AM
#32:


well now that Giant Bomb has been bought by Gamespot, it looks like we won't have to worry about any of those pesky, low 4/5 reviews anymore! 5/5's for everyone! (...that purchases $50,000 of advertising space on Giant Bomb!)

--
"It's a magical world, Hobbes ol' buddy...let's go exploring!" - Calvin
http://img.imgcake.com/calvinfinalpngpy.png
... Copied to Clipboard!
pjbasis
03/15/12 10:32:00 AM
#33:


I think he means don't make contracts based on professional reviews.

--
http://i498.photobucket.com/albums/rr345/Rakaputra/pjbasis.png
SuperNiceDog - 1, pjbasis - 0
... Copied to Clipboard!
KamikazePotato
03/15/12 10:32:00 AM
#34:


Between intentionally releasing a bugged version of Skyrim and "no 85+, no money", yeah.

--
Black Turtle did a pretty good job.
... Copied to Clipboard!
foolm0ron
03/15/12 10:37:00 AM
#35:


From: DigitalIncision | #031
Why? What do you want them to do? Give them more money than the contract specified? Give them money so they don't have to fire people?


How about NOT give bonuses based on an extremely inaccurate review aggregator?

This is pretty damn disgusting, I literally gagged when I saw that. As someone who is trying to get into the games industry, I really hope this isn't a common practice.

--
_foolmo_
mobile computer
... Copied to Clipboard!
KamikazePotato
03/15/12 10:41:00 AM
#36:


foolm0ron posted...
From: DigitalIncision | #031
Why? What do you want them to do? Give them more money than the contract specified? Give them money so they don't have to fire people?
How about NOT give bonuses based on an extremely inaccurate review aggregator?

This is pretty damn disgusting, I literally gagged when I saw that. As someone who is trying to get into the games industry, I really hope this isn't a common practice.


Don't know if it's common, but I have heard of bonuses for great review scores. I have not heard of only getting payment upon great review scores, what the hell.

--
Black Turtle did a pretty good job.
... Copied to Clipboard!
Grand Kirby
03/15/12 10:46:00 AM
#37:


From: pjbasis | #033
I think he means don't make contracts based on professional reviews.


The ironic thing, wouldn't this normally benefit the staff more than anything? I would imagine most bonuses in the video game industry are based on either reviews or sales. Both of which are very high for Bethesda. This is only a theory, but it's possible someone thought they would have higher chances of reaching a review goal than a sales goal (considering sales can sometimes be fickle, but good reviews from popular, well-respected franchises are easy to get), and intentionally bargained for a contract like this, and it backfired.

Like I said, it's only a theory, I have nothing to back it up. But I do think if they had a choice, a lot of people would probably see a review goal as being a better bet for them.

--
Okay, I rolled a 14. What's that mean? Hsu
That you're a cheater. This is a 12-sided die. Chan
... Copied to Clipboard!
tereziWright
03/15/12 10:49:00 AM
#38:


pjbasis posted...
You're mad at the conclusions and actions people take because of metacritic.

The data is useful though.

As useful as the scores it's based on anyway.


The data is not useful at all, it's counter intuitive. It's a flawed system.

--
http://img.imgcake.com/1327084525516jpgra.jpg
http://img.imgcake.com/1326165368699gifme.gif
... Copied to Clipboard!
pjbasis
03/15/12 10:50:00 AM
#39:


What are these flaws exactly that aren't inherent in the review process itself?

Someone else called it inaccurate too, and I can't see the reason.

--
http://i498.photobucket.com/albums/rr345/Rakaputra/pjbasis.png
SuperNiceDog - 1, pjbasis - 0
... Copied to Clipboard!
TRE Public Account
03/15/12 10:55:00 AM
#40:


Don't know if it's common, but I have heard of bonuses for great review scores. I have not heard of only getting payment upon great review scores, what the hell.

It was a bonus in this case as well. They did get a normal payment.

--
SuperNiceDog was not so nice to my bracket.
Congrats to SuperNiceDog for winning the Rivalry Rumble Guru Contest!
... Copied to Clipboard!
tereziWright
03/15/12 10:56:00 AM
#41:


Because everyone scales things differently. And nobody is truly objective.

Even taking scores from two sites and averaging them is already ****ing up.

No review is worth anything without context. Gerstmann put out best when he said anyone who puts all their faith into a number score is a moron.

--
http://img.imgcake.com/1327084525516jpgra.jpg
http://img.imgcake.com/1326165368699gifme.gif
... Copied to Clipboard!
redrocket
03/15/12 10:56:00 AM
#42:


pjbasis posted...
What are these flaws exactly that aren't inherent in the review process itself?

Someone else called it inaccurate too, and I can't see the reason.


How about that literally anyone can make a blog, start reviewing games, and get added to metacritic.

--
From his looks Magus is Macho Man Randy Savage as an anime zombie. The black wind howls, and one of you will snap into a Slim Jim ooh yeeeah! -sonicblastpunch
... Copied to Clipboard!
foolm0ron
03/15/12 10:59:00 AM
#43:


From: pjbasis | #039
Someone else called it inaccurate too, and I can't see the reason.


They linearly convert 5-point and 10-point and letter-grade rating systems into the 100-point scale, when usually the scales aren't linear like that at all.

--
_foolmo_
mobile computer
... Copied to Clipboard!
EmDubyaSee
03/15/12 10:59:00 AM
#44:


From: tereziWright | #041
Because everyone scales things differently. And nobody is truly objective.

Even taking scores from two sites and averaging them is already ****ing up.

No review is worth anything without context. Gerstmann put out best when he said anyone who puts all their faith into a number score is a moron.




Because we should listen to that worthless piece of **** moron...

--
From: Regaro_Ukiera |
MWC is god-tier
... Copied to Clipboard!
DigitalIncision
03/15/12 11:02:00 AM
#45:


I've got some extra hosting and love startin up some new sites. Anybody here interested in starting a metacritic-esque site and creating a panel that decides, based on some sort of algorithm, to weigh sites and their reviews? as in, as someone noted, an eight from eurogamer being different than an eight from IGN? This is definitely something that's a possibility, and not something that I want to take on by myself.

--
Digi
And then God created Saturn, and he liked it, so he put a ring on it.
... Copied to Clipboard!
tereziWright
03/15/12 11:04:00 AM
#46:


MWC you are one creepy dude.

--
http://img.imgcake.com/1327084525516jpgra.jpg
http://img.imgcake.com/1326165368699gifme.gif
... Copied to Clipboard!
pjbasis
03/15/12 11:04:00 AM
#47:


From: foolm0ron | #043
They linearly convert 5-point and 10-point and letter-grade rating systems into the 100-point scale, when usually the scales aren't linear like that at all.


They're not linear? What are they then? How can you compare what a 5 is compared to a 4 or a 3?

And if all these things can't be compared to each other, why are we using numbers at all?
I agree that context is the most important thing, but if they weren't trying to make a universal statement with numbers, they wouldn't do it at all.

Aggregates just take raw data and spit out raw data. I agree that it shouldn't be taken as anything too important, but a pattern does become available.

Anyways, I think most of the gripes I hear are just misdirected complaints about the actual conclusions people draw.

--
http://i498.photobucket.com/albums/rr345/Rakaputra/pjbasis.png
SuperNiceDog - 1, pjbasis - 0
... Copied to Clipboard!
redrocket
03/15/12 11:06:00 AM
#48:


DigitalIncision posted...
I've got some extra hosting and love startin up some new sites. Anybody here interested in starting a metacritic-esque site and creating a panel that decides, based on some sort of algorithm, to weigh sites and their reviews? as in, as someone noted, an eight from eurogamer being different than an eight from IGN? This is definitely something that's a possibility, and not something that I want to take on by myself.

Any 'algorithm' you use would just be something that someone literally pulled straight out of their ass. This wouldn't be any better than metacritic and wouldn't help anything.

--
From his looks Magus is Macho Man Randy Savage as an anime zombie. The black wind howls, and one of you will snap into a Slim Jim ooh yeeeah! -sonicblastpunch
... Copied to Clipboard!
DigitalIncision
03/15/12 11:07:00 AM
#49:


From: redrocket | #048
Any 'algorithm' you use would just be something that someone literally pulled straight out of their ass. This wouldn't be any better than metacritic and wouldn't help anything.


I don't mean a hard algorithm that we objectively put numbers into and pump out a score. I mean people who actually pay attention, gather feedback from users and even companies.

EDIT: The biggeest problem with Metacritic, to me, is the fact that there's no face there. There's nobody that can evaluate the process. They created a terrible way to do things years ago and it hasn't changed even the slightest bit.

--
Digi
It must be very hard to prove Viagra works.
... Copied to Clipboard!
redrocket
03/15/12 11:08:00 AM
#50:


pjbasis posted...
They're not linear? What are they then? How can you compare what a 5 is compared to a 4 or a 3?

Are you being intentionally dense? Or are you playing the "internet devil's advocate"?

--
From his looks Magus is Macho Man Randy Savage as an anime zombie. The black wind howls, and one of you will snap into a Slim Jim ooh yeeeah! -sonicblastpunch
... Copied to Clipboard!
Topic List
Page List: 1, 2