The problem there is the inversion of your prior paragraph...many people rank games they do not own extra low (in an effort to lower the value of them, increase the prestige of the games they do own, or for whatever other reason).

Subjective ratings will always have issues with bias. Very hard to avoid. I'd like to see Pinside get rid of legacy votes (votes by people who have not logged on to Pinside in the past year)...that would make rankings reflect the current community and not the community of 2007.

Well what if people buy the games they like because they generally have a high appreciation for them and aren't just a bunch of masochistic market manipulating sociopaths?

That also happens, no doubt about it. Hard to distinguish bias from honesty on subjective matters though.

Theme drives a lot as well, if someone dislikes a theme they will rate a game low regardless of merit often.

There is no real ranking

It's all subjective

Think of it this way:it's like the horoscope,it's fun when it matches what's going on with you,but you don't live your life by it.
And you don't build your collection off the list.

Well that's logical. Most people don't buy a game unless they've played it first and like it (unless they preorder). So of course they won't give it a bad score. And if they do buy a game and dislike it, they'll sell it, in which case they will no longer own the poorly scored game.

Yeah agree with the comments a above , u play a game ,fall in love with it, need to own it because u love it an want to figure the game out in great depth . Realise it's even better than what u originally thought an then once played enough to give an opinion rank it.

how does liking a game so much that they bought it invalidate their opinion?

i doubt anyone really believes they can affect the resale value of their game in such a way. it's one rating among thousands, and Pinside rating is only loosely related to resale value anyway. even if you assume owners ranking their own game is somehow unfair, the effect is more or less offset by all the owners of all the other games on the list doing the same thing.

it's really not worth worrying about.

Gilligan's Island, number one.
No other rankings need apply.
Thank you

I don't rank games on Pinside (or anywhere) because I don't believe in the system. I think it is nothing but manipulated rankings. Anyone that ranks them honestly is lost in the noise of manipulation. There is no way to make them valid. I think it would be better to have just the mod staff write reviews rather than the rabble with an agenda.

I've abandoned pinside rankings. I don't look at them, and I don't contribute to them anymore.

Unlike movies, where after you've paid your money, you are done with it, and are happy to declare that the movie sucks, with pinball we have heavy investment in machines and that will make some people pump and others dump their reviews.

Then you have the fact that people will review games on day 1, and never revise their review.

Then how games you reviewed 5-10 years ago, may not be the way you feel about them today compared to a newer game, but your old review stands.

And of course the old reviews by people that don't even visit pinside anymore, so the 10s they gave a game years ago will always stand.

I don't honestly know how to fix it, so I choose to ignore them now.

The problem is still there though, mods have the same prejudices as everyone else (confirmation bias and what not). Nine of my top ten rated games are in my collection...of course, I bought them because I really like them (so that's why I have them ranked high) but that would look like I was manipulating the rankings based on the 9 out of 10 fact alone.

So it's easier to believe there is a large conspiracy to alter prices by fudging ratings instead of people generally spending more time and owning games they enjoy the most?

I'd like to see people rank games in order of their preferences rather than in an absolute sense. That way if you own 9 out of your top 10, you'd still be forced to pick among your own children. Doesn't solve the manipulation problem but I think that's not as big an issue as an 8/10 means different things to different people.

Its all statistics and they can be manipulated any way you want to with presentation. MM is only .4 points above the 11 spot. Does that mean MM is 10 times better than TOTAN? Or does it mean it is only 5% better? Both are true depending on how you present the info.
The people who look at this list like it is the 10 Commandments or something make me shake my head. It is just a tool. Like anything You have to dig a little deeper to get to the "WHY?" Why is it rated poorly? Or well? Is the artwork great but the gameplay is flat? Yadda yadda. The comments are what matters. Best game ever gets passed over immediately. I try to be as objective as possible with my ratings, but I know my comments are not the best or very detailed. That's ok, I use the ratings more for my own notes than anything else. So when I see a game I don't remember playing for sale at possibly a good price, I can go in and see if I played it, and liked it or not.

To not put your rankings in because you think it will get drowned out by the noise of the false votes is the same as saying you don't vote. You are not helping.

I've ranked 14 games that I own honestly, and they were all rejected by pinside so none of them count. Apparently some were never even reviewed.

So do pinside top 100 lists and ranking really matter? Not really when some are approved and others dismissed.

I believe the system, whatever it is, sets it up the way it wants it to be.

I checked out your ratings O-D. It says they have not been approved yet. How long ago did you enter these ratings?

I added comments with all of my ratings. Can you go back into them and add some brief comments.
Several of your rating scores look very legit and match up similar to how others have rated some of those same machines.
I do see that you have several that are rated at 10.0, which may or may not be a red flag.
It makes sense that you would buy/collect the best of the best if that is your style, so having some 10.0 scores (or high 9's) is reasonable. Perhaps review them all and see if any need tweaking.

Over a month.

None of them need tweaking. I've played or owned just about every game I could in the 50 years I've been playing pinball so I know by now what I like.

I may add some comments later, but for now as long as my ratings are rejected, not reviewed, not relevant or whatever, I'm not going to waste any more time on this pointless venture.

That is a pretty long time frame.
If it had only been a couple days, I would say wait for a full week. But a month is more than enough time for them to be reviewed.

o-din,
Do you sometimes feel like you are sort of like Otto in Repo Man, and Pinside mgmt is sort of like store mgr Mr Humphries?
http://www.tcm.com/mediaroom/video/352887/Repo-Man-Movie-Clip-Young-Men-Of-Your-Age.html

Perhaps. I have discussed it in a moderator feedback thread and the only thing I can get out of it is my ratings have been flagged as "fake" by the algorithm.

So in the end it is some computer that decides which ratings are relevant and which aren't. Which is probably why the pinside top 100 lists are arranged like they are. Totally bogus.

I've ranked 14 games that I own honestly, and they were all rejected by pinside so none of them count. Apparently some were never even reviewed.

Hard to believe considering the ratings that have several words or short sentences saying how lousy the pin is and rate it a 1 or 2.

I didn't give any negative ratings trying to bring games down.

I only gave good ratings to games that I am keeping and have passed the test of time for me. Not what Pinside's looking for I guess.

People who rate game 1's have a heck of a lot more pull over a rating compared to those who rate games a 10.

For example MM is rated around a 9 and if one fool rates it a 1 then 9 people have to rate MM a 10 to keep the 1 rating from lowering the overall avg rating.

The 1 rating haters of games are a lot more of a bother than the 10's. They are the real problem with the rating system.

If I was to give MM a 1 (which I wouldn't because it's at least a 6), does that mean my other ratings will count too?

From what I can see, you gave 9 games perfect 10's. I have never given a perfect 10 to any game. Or a zero, for that matter. No game is a perfect 10, IMHO.

Honestly, you're getting yourself worked up into a frenzy over nothing. I'm sure Robin has it on his (lengthy) to-do list to investigate. He's been away on vacation for several weeks, and only just recently returned. It's going to take him a little bit to review everything that has happened while he was away and prioritize what needs to be addressed.

In any case, when you go around rating a bunch of games perfect 10's or zeros, then yes, the software is going to think there's some funny business going on and mark the ratings as bogus. For 99% of the users on the site, the feature works properly.

In the end, ratings are always going to be subjective, so any raking system based on opinion isn't going to be perfect.

Absolutely. The ratings tool is what it has always been: a mechanism to rate and (hopefully) review the games you have played. The main goal is to allow Pinsiders to create their own top x games list. Allowing them to keep track of games they might want to own in the future (or possibly avoid buying). And of course it's also about letting others know what games you like the most.

That's the main goal.

And then we use the cumulative of the ratings of all people on Pinside (currently over 75,000 ratings) to create our "Top 100" list. Which really is just that: a list calculated by averaging all those individual ratings. Nothing more, nothing less. It's not the ultimate truth about pinball. Like Sealclubber already noted: it's just a tool. I think this is stated everywhere throughout the site. Taking the Top 100 as gospel is silly, really - how could one list fit everyone's opinion?

The ratings are what you make of them yourself. And the review part of a rating is the most important one: it allows you, the reader, to give weight to a rating. I even contemplated making the review part of a rating mandatory but then decided against it: not everyone is good with words or even the English language.

When the system flags something as bogus it is most often right. When all your ratings are perfect 10's without any explanation it is not considered a "serious" rating. Does that invalidate them? Absolutely not! They will show on your profile page and still be your ratings, for all to see. We are not dictating what your opinion on games should be. However, we do reserve the right to choose which ratings we want to have part of the Pinside ranking and (as our how to rate guide clearly outlines) rating all your games perfect 10's is not considered serious rating.

People are looking into this ratings system way too much, and have been for a while. Keep in mind that this was system was built by yours truly in 2003 (the first rating in the database dates from February 19 2003) and little has changed about the system since. I would love to overhaul it all and actually have some pretty cool ideas for it - but currently all my time is divided over other aspects of the site. In other words: it does not have high priority.

If there would be a bit more spread in your ratings, you would probably be okay. Surely not all games you have currently rated as 10 are really equally as good as one another? Is Struggle Buggies really just as good as Heat Wave? Of course you can have 2 or 3 games a 10 but having 9 games a 10 really makes it look like you are not seriously rating these games but simply trying to doctor the system. And that's fine with me, you can have a personal top 100 list of 100 games all rated a perfect 10 - I don't care- but the system will not accept it for the top 100 list.

Don't look at the ranking ....buy what you like without looking at the rankings ....for me my favorite pinball in my collection is The Avengers and its ranked around the #250 and I own the pinball #1 MM in my collection too ....

You're stating it yourself: In your humble opinion.
My opinion is different, I know at least one game that is a perfect 10 to me.

There is no wrong or right, these kind of rankings/ratings are strictly about personal opinions.

But I don't really care either as I don't rate games in a public forum.

It is perfect. It reflects the collected average opinions about any game in a perfect way.

I don't understand the fuss that's constantly made about this rating system.
Why would anyone care if his favorite game is rated No.1 or No. 300?
It's just a fun collection of opinions that has no other goal than pure entertainment.

I would be seriously interested in the arguments on why it matters so much (or at all) for certain pinheads which rating a game has. Really boggles my mind, maybe I'm lacking imagination, but it just don't get the level of commotion and emotion involved in this topic at all.

the problem is ratings are virtually linked with pinball values. so ratings can be perverted in this way. maybe cutting 10% of the highest and the lowest to get a fairer ranking?

and i don't understand the final score of each rating. when i calculate the average of the 4 categories (game design, artwork, sound, other aspect), i never get this final rate.

Honestly, I don't take average ratings into account, I read comments - and only comments - I don't even check if the associated ratings are consistent. And I tend to trust the opinion of some pinsiders more than others - my own personal bias...

(Hey, BTW: congrats for the Portuguese victory... Was in Lisbon last week, gorgeous city: loved it!)

Is that really the case? I never saw any connection between price and rating.

Of course more expensive games tend to be ranked higher in many cases, but to me that effect is due to those games being more desirable to more people, not because of the rating. It's a bit of a chicken and egg problem.

The really expensive games (ST LE, BBB, TRON LE) are actually ranked rather low. TAF or TZ are much cheaper than CV or WoZ which are ranked much lower. BBB could be the last game on the list and still be one of the most expensive. I've also never heard that MET or AC/DC are priced higher because they have climbed up the rating list, they were always quite expensive.

I don't think it has to fair as it's just opinion. It's not supposed to be an objective analysis of the "true value" of any game.

From a user point of view I would make some changes, though:
Delete all ratings that are not commented and delete all ratings that are commented with general formulas like "worst game ever" or likewise.
There's no value in any rating if there is no argument attached to it. I enjoy reading why people like or dislike a game, even more so if they have a different opinion than mine. I can happily accept any "0" or perfect "10" rating if it is supported by reasonable arguments.

It's for fun. We can share our opinion and see what others think.

The longer I'm in the hobby, the less I look at it. It's nice to look at when I'm considering buying a game. I like to see people's impression of it. I don't tend to rate games much.

I don't take ratings into consideration at all. I do read reviews and try to deviate statements from them that may be useful to my decision making, but that's something very different than basing a decision on the rating itself.

Or, to put it differently - I would not buy a Justin Biber album even if it's No.1 in the Billboard charts.

And congrats to the EM victory! Thanks so much for kicking out France!

i think you can't rate a pin 10 or 0 without getting in mind to raise up the pin value or lower it. some of raters are gambling with that.
i never said it was fair or objective but i am sure it has an influence on pinball value.

so I sort when I look at the opinions of each other. some italian pinball fellow here rate with 10 saying the pin is great or 0/1 saying the pin is ugly. this is not a rate according to me

The real ranking should be your own ranking. Open excel and start at #1

Can we discuss ranking then? I liked Robin's statement that it is really my rankings, so I'd like to try a discussion about "X", a game I randomly picked from my collection. It is probably my 3rd or 4th favorite game right now. It is a fantastic game with things that could be considered minor flaws, but in the context of this game, it all works. Where "should" it land? I can start at 7 because it is near the top, but does that mean game "Y" needs to be an 8 because it's a bit better?

That is why we run into issues. I can't even rank the games I own. At any given time my favorite ge in my collection may be something ranked very low by others, yet I'm having a great time on it. I realize it isn't a 10, but it sure is giving me fun.

Maybe it is simply the number causing the issue. The "better/worse" system suggested above could remove the numbers. It makes it so that you could only have 1 "best game ever made" (currently a 10), and only one "worst game ever made" (currently Judge Dredd).

I feel that there should be two parts to the game rankings & ratings.

Part one is a comprehensive list of where the game ranks on your personal list. The Pinside 100 can be based on the average spot a game takes on everyone's list.

So the number 1 game on the Pinside 100 would be the game that has the best average spot across all Pinsiders. In order to prevent manipulation, a DMD Modern machine would need to be ranked by at least 20 members. An EM would need to be ranked by at least 5 members to join the EM Top 100.

Since the system will not allow for ties, a member will be forced to order his/her games from best to worst.

It would also require that someone rank a lot of games before their last place would affect a popular machine. Someone who wants to keep Star Wars LE from taking the top spot from their precious MM would need to dedicate of effort to put a dent into the rankings.

The second part would be the ratings. This is where every game in your collection can be rated a 10 if you choose. These ratings would not be used by Pinside to rank the games. It would just be additional data that a member can reference. Pinside could provide an average rating for each game based on members' input.

Before a member's rankings can impact the Pinside 100, both rankings and ratings need to be added to the collection.

This would also allow Pinside to sunset rankings & ratings by users that have not visited Pinside in over a year. Only users who are active members would count towards the Pinside 100. This would help eliminate the one time ratings by one time visitors (I.e duplicate accounts, fake accounts, etc)

Just my 0.02

Marcus

History writes itself.Even without rankings there will always be a #1 and there will always be a game that's dead last.

Whaddya gonna do?

I just follow the ratings of and read the comments from caucasian2step. Saves me a lot of time.
-mof

Yes, getting rid of legacy votes is really key. It would clean out the one time only dupe accounts.