Not true.
Value of a game can be affected by it's rarity and not reflect how good it is. Again, BBB!

Thats one of the things I like about MM.

Ranking s can change or stay the same, not matter to me. I will play them and make my own decisions. FYI, I think the specialist rating top 50 is the most accurate rating list but other my not agree. It is based off of people on here with 50+ ratings and it seems to work well.

What I think are top suggestions for improvement:

-Pins need more than 10 rankings in order to be considered as part of the top 100 list in order to limit impact of homer rankings followed by trollings. Maybe 30, maybe 50?

-Remove all rankings from members that have not signed in to pinside in the past 12 months. Makes rankings system more relevant to current members.

-Games must be on market for atleast 6 months >> again limits the owner inflation and subsequent trolling.

-Limit the number of overal rankings for a specific range that an individual can give out >> i.e. a forced distribution so you can only give one 10.0, 9.9, 9.8, 9.7, etc... This makes individuals more consiously rank games against each other rather than saying they are all the best.

-Pins with less than 500 machines made are kept to their own rankings page >> "the botique pins rankings"

those ratings should be thrown out, IMHO. same with 10's from folks that have never played it but simply like TRON.

Done. And in fact, I'm really glad I stumbled across this thread because I really haven't rated any games on Pinside yet. It'll be nice to get my first 6 done

This makes sense in a few ways.

1) The way things are going each new machine will have 2 or 3+ different versions. The rankings might get pretty messy in a couple years.
2) Not many people actually own the limited ones outright. So there is a rankings battle of the "haves" vs. the "have-nots."
3) People who don't own them probably haven't had time to log solid play time on them to make a decent ranking anyway.

Goota love this thread...every one just has great points .
It wasnt long ago there was a thread that evolved into the manipulation of the ratings.
Perhaps some of the really high and low ratings should be dismissed...as sometimes there is trolling that takes place just to manipulate the rating of a certain pin. And just a thought..no doubt that those who own a pin, tend to rate them higher...right or wrong, those ratings can be too subjective.....so this perhaps should be taken into consideration.
I dont think the idea about karma should weigh on ratings...to me, that would not be fair. You dont have to have high karma to be a "better" rater.

Just a quick question...

If it does not show the little pinside emblem thing next to your piside avatar does that mean you have not ranked any games or does that mean you have set rankings to private?

IMO, I think the Top 100 list will get cluttered with the same games after a while with all the new LE's, Premium, Decepticons, Autobots, etc...

I would hate to see the top 10 as only 5 games and their respective LE models. Just seems like after a while it will be a lot of the same games and their other versions.

Although, I do agree that each version should have a separate "page", maybe the rankings should just go together...?

I wonder how many of the raters preordered the LE. I would hope they would abstain from rating their machine as it creates too much justification bias. Alternatively, the non-biased folks will have very little opportunity to properly rate this game due to scarcity. All I can say is that it is a #1 worthy theme.

You defy your own argument, as you have rated 95 games, yet own only3. Plenty of data left over there.

I maintain that LE rankings are doomed to be completely biased.

Who cares how high a game ranks, really?

If your favorite game is #1, good for you - pat yourself on your back that everyone agrees with you.

If your favorite game is #100, looks like you know something everyone else doesn't.

That is a shame. I do not have a problem with Tron LE being # 1 If it was rated fair which I think it was,although I have not played it yet.The ratings work themselves out after a while,but if some one is purposely trying to skew the ratings because there is a new #1 then that is TOTALLY WRONG And should not be allowed!

I echo the sentiments of those who say 'who cares'. Seriously, who cares what the Pinside ranking is? Someone that cares deeply for the Pinside ranking, please help us understand why you care so deeply, I'm honestly curious. Any rankings like this are a cluster of biased subjectivity. Why on earth would anyone take that recipe seriously?

with all due respect, i can't get on board with that. i pre-ordered TRON LE. i ranked TRON LE. i don't think there's the slightest bit of justification going on....i could not possibly care any less how much a game is "worth" or how much i paid for it. if it's fun, it's fun. period.

you'll never find me bashing MM because it costs a bazillion dollars either. i think it's one of the best pins i've ever played.

i only rank pins that i've played.....bet ya there's gonna be a lot of down-ranking from people who have never played TRON LE though.

a big +1 on that.

Wow, talk about an upset! I thought the 15-20 spot was just fine IMHO...
Sounds like CLU is up to his old tricks again.

CLU.jpg

"I executed the plan!"

...So does that mean my LE is now worth $12,000?!?
Any takers?

i doubt clu likes TRON all that much...what with the betrayal and all.

You guys do know that TRON made it to the Elite Eight of the Pinball March Madness Tournament done by the Pinball Podcast guys last month, right? It missed making the Final Four by one vote, if I recall correctly. That was voted on here, by e-mail, on KLOV, and on their website. I don't think it is justifiably at #1, but I think it is clearly a top 10-12 game at a minimum.

When WOZ is released Tron LE and MM won't matter anyway.

I think that is a given, in fact I am surprised some have not already rated it a 10.

Now that's a sensible remark.

For example, #143 is Baywatch - a surprisingly fun game with quite a good rulesheet too. Artwork is top notch (well except for the big 'Hoff' backglass maybe) and the DMD animations are beautiful. Playfield layout is very clever and it's even a widebody. Fortunately no one else seems to know this and I got my Baywatch dirt cheap - shhht don't tell anyone

If you REALLY TRUELLY Think you rated it fair then I have no problem with that.

I rated it the same way I have rated all pins thus far >> subjectively after I have played the title enough to 'get it' and without the intent to artificially move it up or down but rather how I feel it performs relative to all other games I have ranked.

Granted I only have ranked a handful thus far but they all fit with my honest opinion of how I like them.

I think it is fun to see how it could be made less biased and more accurately a true rating system rather than one skewed due to artificial inflation or deflation. I also just like data and numbers...

On a more utility perspective >> I use it as a good way to figure out what other pins I should seek out to play. I also really value the personal comments in ratings and find it a great way to see what others truly think about a game. For example, If I see that someone else REALLY likes TAF then I go read their comments and ratings to see what else they like and why.

Bias is a real phenomenon. We pinheads are not immune.

Here is my somewhat tongue-in-cheek proposal for ratings adjustment:

-Owned a game, sold it, then rated it- no bias, no adjustment necessary
-Played a game a lot and rated prior to owning- no bias, no adjustment necessary
-Played a game a lot and rated after owning- small bias, reduce rating by 2%
-Bought a game with little to no prior playing experience then rated it- moderate bias, reduce rating by 5%
-Bought a game on preorder before anyone had even seen it, much less played it- maximum bias, reduce rating by 10%

just some food for thought/discussion

Haha, I love that! I'm a little less enthusiastic about the time it would take to build this, but it is a really creative idea!

That I get, I do that same thing with movie and book reviews. Good point!

I wonder how reviews could be made less biased, as reviews are based on completely subjective criteria. Adding objective criteria would negate the spirit of a review, but would be fun to see I think. For example, quantifying rule depth; how many of X need be accomplished before Y happens which is necessary to get to Z. Or ramp count, or mechanical toy count / category; bash toy, magnet toy, lock toy, etc. If you liked a particular type of playfield feature that might be interesting.

Problem is each machine is such a random assortment of features, meshed together with lights, art and sounds, that you end up coming back to the subjective when attempting to evaluate. Ah well, good times.

I dont even look at the scores. I just read what folks write. based on that I can get a good idea of how relevant the game is, what they think and what I may like.

i think people are capable of being intellectually honest despite the fact that they pre-ordered a pin.

yup. i'm the same way.

hell, i'd be in favor of throwing out some of the 10's and 3's that don't explain why the ranking was given.

I see what you are saying. However, there is a flip side to this. Owners usually get more play time on the pin in question and thus have an opportunity to see all that the pin has to offer gameplay wise. This may also be a valid reason why they rate it higher. So to take out or reduce the potency of opinions by those that have the most experience on a pin would make little sense IMO.

Capable? Certainly. But not reliably so, or even particularly likely. Even clinical researchers have to design studies that minimize the potential or their own bias to be considered rigorous.

I like the Karma points or if they owned game. To me owning it may be biased but probably more realistic to assess.
heck I even look at what games folks own on the forum and as bad as it sounds that dicates how much I give credit to their posts. I am pinshallow : )

Yes, but as a lover of pinball, I would imagine that I would rate most titles higher with the added experience/familiarity, thus ownership is still a net positive bias. For me, at least.

ok. we're obviously not gonna agree on this one.

i tend to lend more credence to high rankings by owners than i do low rankings by haters. to me, there's not a whole lot that's less admirable on here than someone coming in and giving a game a 3.5 just to "adjust" a ranking they don't agree with.

i dunno, man.....too bad we can't all trust people to rank a game based on how they actually feel about it.

Agreed. Haters are a bigger problem. I didn't include negative bias in my formula because I can't think of a mechanism to predict which "haters" are going to hate which titles. Which isn't to say that the haters effects are random and evenly distributed.

I love Pinball,so I do not see how any pinhead could rate any real(working) pinball machine below a 5.

You have to realise some people built their collection around being told what to like Like the most expensive Pokemon derivative ever seen.

Hooray for Tron Legacy!

All this talk about Tron and I still cant even find one to try. There is not a single one listed on the Pinside map to play in public in my entire state.

Don't worry, many people will rate WOZ a 5 something, whether they played it or not, and self justify it through cognitive dissidence just to ensure it can never pass the Bally/Williams's Grail games.

Lots of really good ideas have come out in this thread. This was a very useful discussion. Another possible thought, though this one is going to be more controversial, is:

Only let the members with the little heart by their names' votes count...that ends trolling through creating fake accounts...to a large extent anyway unless people start using stolen credit cards...and if someone is that desperate then by golly let them manipulate the rankings.

It's a top 20 game to me.
It's not my top game though...

Good idea, but I don't have a heart, because they didn't add that I donated.

"Down voting". Haha. Kind of reminds me of an article I read about the Xbox 360 vs PS3 fanboy wars. As soon as a big exclusive game comes out for one machine or the other, the fanboys go on Metacritic.com and vote all zeros for the other console or all tens for their console. They do so using many accounts per person too.

Oh well. I have only played a few games of Tron and have seen but not played Tron LE. I think it is a great game. I put it in the top 40, but not anywhere near the top ten. Maybe my opinion will change when I get a chance to play it a lot more. Until then I will not rate it officially.

John

I guess I am going to have to find a Tron LE somewhere to play!

I just found an ACTUAL problem with the rating system.

This deals with older ratings compared to new ratings. It may have to do with the changes Robin implemented to the ratings system late last year.

Seeing this thread, I realized that my Tron rating was for the Pro and not the LE. Deciding to rectify this I reviewed my previous rating and decided I still felt the same way about it, so I went ahead and rated the LE... The EXACT same way that I had rated it before.

The problem is that if I look at the ratings on my personal list the rating listed under the Pro was logged on 8/8/11 and is 9.223. The rating I submitted just now for the LE (with the exact same settings) comes in at 8.840... You can see them both on my personal top whatever list, although I don't think you'll be able to see the ratings breakdown showing that they are in fact rated identically.

I suspect there is a flaw in the system somewhere related to when the ratings system changes went into effect last year. Any ratings submitted prior to the changes will be artificially higher than those submitted after the changes were implemented. Or the ratings submitted after the changes were put in place are artificially lower. Take your pick.

There could be another cause entirely, but I have limited data to go on at this point. I'm sure Robin will be able to investigate this more deeply and find the root cause.

One other question for Robin... Is there a way to delete ratings? I've only played Tron:LE and never had a chance to play a Pro thus making my Pro rating invalid. Not to make more work for you, but it would be nice if there was a way for me to remove it.

Same request here, if it's possible.

I think these are all good points. Also, I think you require discloser of how people rank AND only allow those who have contributed to the website to rank (or run two lists for payers and non payers) and you will have as good of a rating system as is reasonably possible.