there is something to be said about sample size but we are never going to get the same size sample when there are almost 10X as many mm's as tron le's out there.

There are more than 38X more TZs than TRON LEs, too.

Which was my point in the other thread. MM with 4400 units out in the wild for 15 years versus Tron LE owned by very few in private homes for a few months. There is MUCH more inherent biased baked into the Tron ratings.

With all due respect the rating system is automatically flawed because there is ZERO scientific data used to determine position. The rating system is 100% based upon *opinion*. It's truly a simple popularity contest. Same with IPDB's rating system as well.

Right, but if you rate a $5000 pin you bought for your home, versus playing one that someone else bought at a pizza place....well, the former is going to win that popularity contest every time. In other words, the entire universe of "informed" LE raters own that machine for their home.

The same thing happens on IPDB but reversed...POTC and Tron are supposedly worse than Zaccaria's 'Clown' if you believe those ratings...because Clown has been rated 9 times and the other two have been rated 80-150 times...the newer games get all the ratings and the old ones sit in the top 100 with no sign of going anywhere

...but then again thats the IPDB, the place where SM is rated the same as Diner

Looking at IPDB ratings makes us realise how sophisticated and accurate Pinside ratings can actually be in the long run.

My point is that if the data is not hard data, only soft (opinion, feelings of the moment, etcetera) than why on earth does anyone care? You simply cannot make a purchasing or even something as innocuous as a playing decision based on other people's feelings! Well, you CAN but I'm just sayin'... why? Has society so bought into American Idol, et al - which are merely glorified popularity contests - that it even extends into our pinball hobby? Really?

Variety is the spice of life and one day perhaps Cyclone is the fave of the day for me but then a couple days later maybe it's Batman Forever or the next day Star Wars. See what I'm saying? Some of you guys put too much stock in what other people think... or what they DON'T think of YOUR choice of games. Who cares!? Play whatcha like and like whatcha play! I have an MM and a buddies Tron LE sitting here next to my TZ. I'm telling you any day of the week either of them could be my #1. Or none! And I'll wager your choice would likely be something else entirely. The way it *should* be

cross post from other thread since this one is on the topic:

What I think are top suggestions for improvement to the pinside rankings system:

-Pins need more than 10 rankings in order to be considered as part of the top 100 list in order to limit impact of homer rankings followed by trollings. Maybe 30, maybe 50?

-Remove all rankings from members that have not signed in to pinside in the past 12 months. Makes rankings system more relevant to current members.

-Games must be on market for atleast 6 months >> again limits the owner inflation and subsequent trolling.

-Limit the number of overall rankings for a specific range that an individual can give out >> i.e. a forced distribution so you can only give one 10.0, 9.9, 9.8, 9.7, etc... This makes individuals more consiously rank games against each other rather than saying they are all the best.

-Pins with less than 500 machines made are kept to their own rankings page >> "the botique pins rankings"

Some good ideas, and some might get implemented but... in the end it won't matter - some people will always disagree with "the list", no matter how it is compiled.

The Pinside Top 100 is meant to be a fun and informative list, it's not set in stone!

Please read http://pinside.com/pinball/help/the-pinside-top-100-faq where you can read this:

Please try to keep in mind that we're talking about a popularity list here. Now this is important enough to repeat! This list only represents the average taste of the users of this site. This means that for a machine to be on this list, it must be known enough amongst our visitors to have been rated a certain amount of times (10 times).

You must understand that we don't assume that our list is 'the ultimate list of pinball machines' in any respect. Nor do we pretend the machines on this list to be the greatest ones ever designed. The list is merely a reflection of the opinions of the visitors of the Pinside and it should be used this way. This means that you should never buy a machine just because it's on this list. Play it first! Reach your own conclusion! And if you like a machine after playing it, you may come to realize that you seem to agree or disagree with a lot of other people.

BRAVO

those that don't like it, sell it. if they don't like it and sell it, i doubt they rank it as a perfect 10. pretty simple stuff here, folks.

i don't believe there's a vast TRON LE conspiracy from the owner's of the pin.

Not arguing with you Cliffy, just saying how a pinball popularity contest (ratings) can be wildly biased based on limited production units, time, and ownership.

i don't believe there's a vast TRON LE conspiracy from the owner's of the pin.

Maybe a little bit... But I called it back in August.

Mr.D.jpg

"I am very very sneaky, sir."

Never said it was a conspiracy, just more biased introduced for LE models due to a number of factors.

So when all the magic girl owners rate their machines as a 10, no issue right?

The top 100 is based off pinsiders rating. I dont think your a pinsider until you have been on the site for at least 4 month with participation (karma). Then you should be able to get the privilege of rating. If a member goes inactive for period of time there ratings should be put on hold IMO (for inactivity). When they come back I think they should have to wait a few months for their ratings to get applied again. The top 100 in my op is for the "active" forum members. Not the trolls or fly by nighters. The only people that will have a problem with this "THEY ARE THE PROBLEM". I dont care what some old forum member who is no longer part of the forum rating games good or bad. I care about the current members who I chat with on the forums, the people who make the contents of this website so great. I also think that Robin runs a great site and he will do the best for us to get fair changes. I also think (lets use tron bc that how this all started) that if a game has superior sound, someone cant rate is a 1. They can only rate 3-5 if we had a base rate. So what im saying is that all games should have a base rate that cant be graded lower. This should be set by Robin or delegated to a "rate referee" so to speak. This would also eliminate some people rating great looking games like CV or TOTAN a 1 on cab art when in fact I dont care if you dont like the art and its not your taste it is still great art.

perhaps.

if we're gonna factor that in, we probably need to factor in all the haters as well. lot of backlash against popular titles.

lot of price backlash too. i think it's silly to think along those lines when ranking a pin, but we see it all the time.

MM has some of the same issues.

you edited your post.

if magic girl <cough>vaporware<cough> ever sees the light of day, then those dozen folks are more than welcome to rank their game a perfect 10 if they want.

i'm gonna stick with ranking games that i've played and ranking them according to how fun they are.

If it is they good and the owners really think it is I do not have a problem with that.I doubt that will happen.

I agree with this part.

You also have to factor in how long a game has been around for people to get the chance to play it a few times and feel like they can give a review.

I totally disagree with removing someone's rating because he is not active on pinside. The fact that someone logs into the site or not in a given time should not validate or invitation their data.

There is definitely a problem with LE ranking. That is the majority that rank it own it. People are biased towards the games they own, otherwise they wouldn't own them. This is a unique sampling to the rest of the list. And that throws off the accuracy.

Personally, I think tron LE is a 9.0 game. But I don't think it's the best game of all time. I definitely think spiderman is better. But that's just my opinion.

Cliffy is 100 percent correct. all ratings are subjective. It is more art than science. It will always be controversial. Just wait till the olympics start this year and people complain about the professional judges. We are all just anonymous amatures in a hobby.

Yeah I would not rely on the rankings on this site too much. Theres NO WAY that Tron is better than:

HS2, RS, CP, WCS, BK2K, T2, TAFG, etc, etc

I disagree on all accounts.

I like TAF better, but TRoN is superior to all of those others.

I concur TDA!

Please tell me what else to expect in today's market. Pinball is no longer a big hit in arcades or anywhere for that matter. This is the reason a le exist in the first place. I would have to drive a 100 miles just to find a pinball on location and I would bet its a piece of junk. Pinball is a collectors market going forward, so expect all ratings of new games to come from future home owners. I own 6 games and I'm not blasting a 10 on all of them. My ratings as an owner are far more accurate than someone who slips a coin into MM and rates it a 10 or 1. Get used to owners rating their pins.

+1...from someone who doesn't even own Tron.

So if some 1 does not log in for 2 years you think there thoughts still count? Look what has happened to pinball over the last 2 years. Why would you care what some one 2 years or 5 years ago thought about a certin pin? I mean it doesnt matters to me. I adjust my ratings from time to time on pins or maybe write a better write up on them. This would also eliminate trolls. I dont know about you but I check the site everyday (well 5x a day at least). What I was getting to is the "current community" rate on a game. That would keep the rating current.. no? If you want an ongoing rating go to the IPDB and check that out. Many 1x ers and old rating. Im looking for active community ratings IMO.

Shakes head, shakes head... I thought this was a pinball forum where *players* knew what was a good game... Saying that Tron is better then Getaway, TAF and T2.. Ok sorry to the guys who think otherwise... nah I cant say it

The pinball hobby changes as the technology changes. Hopefully LCD is a big leap forward. Is it fair for someone's rating from 2008, before many of today's manufacturers entered the market, to still count now that the hobby has changed and new great games are out? If the poster was still around he could update his rankings. If the poster has been gone since 2008 then his ranking should have no weight.

If only people put this much effort into rating and writing reviews of said games in the ranking system...

...

...

...

...

...oh

Nah. It is just a place where people know what they like and aren't afraid to say it.

Heck, I enjoy playing HS2 and TAF. Just not as much as Tron. T2 doesn't even deserve to be mentioned in the same breath as any of those three IMO.

Well....I don't see how you can have a "perfect" rating system anyhow....too many variables and factors to consider and everybody interprets the rating categories differently...

Regardless, the pins at the top are highly regarded by a consensus of people and they must be damn good no matter what....

Yeah sure, as you are prob aware I have no probs with being honest, lol but in this thread I cant be too honest back regarding other peoples opinions but I will say that Tron should not be in the top 50 for a start. Atleast 6 reasons why....

Fair enough. We are all entitled to our opinions after all. Likewise, I could find a number of reasons why TZ doesn't belong in the Top 50. I've tried to like it, but just can't seem to. Different strokes for different folks brotha!

tron now at #1... price of the machine went up another $1000, some re-sellers will be happy.

i take the top 100 not too seriously, i see pins that should be higher and others lower.

They should put games in catagories. By system, company, era or all of the above. That might narrow the ratings. It's apples and oranges, 90's Cowboys or 2000's Patriots.

In general I think the Pinside pinball machine rating system is pretty good.
When you look at the top 200 ranked machines, in general, most seem to make reasonable sense where they ended up on the list.
Pinside has 4 main categories w/ 16 total aspects to address when rating a machine, and each aspect has an assigned weight. Then an average rating for a machine is calc'd. Not bad imho.
I'm sure the rating/ranking system can be improved slightly. Very low ratings do affect the avg.
Changes to consider implementing:
* increase the min required ratings from 10 to 20, or at least to 15.
* drop the high & low individual ratings, or top 2 and bottom 2 scores.
* increase importance of lighting (Note: sounds/music has 4 aspects to rate @ avg 6x, lighting has 1 @ 5x).
As of now, I have not adjusted any of the rating settings eventhough we have that option.
Now that we are allowed to rate the older e-m machines, there could be some more temporary "problem" rankings w/ some e-m's that have only 10 to 15 ratings done, but eventually these should also settle out in time as more people rating the e-m's, or have a completely separate top 100/200/300 for e-m's.

It seems to me like Tron LE will eventually settle in somwhere around at # 6 to #15, definitely in the top 20.

There are a lot of new people getting into the hobby.... and a lot of them Stern fanboys. If you look at many collections you will see a high percentage of Stern (or all stern) in many lineups.
it's highly possible many have not even played any of these older games. The lack of older pins on location and the desire to have newer (lower maintenance) machines is slowly and steadily driving new pinheads in this direction.
I'm not trying to insult anyone, but it's reasonable to assume that some newer collectors may have a very small frame of reference when rating pins.

Edit:
lastly,
Let's just leave the rating system alone to work itself out, but please, can we stop splitting pins according to their versions?
If a manufacturer wants to release multiple versions and have some of them full featured and others that are lacking, let the overall rating of the project reflect this. Let's rate the entire effort as one (inclusive of pros, premiums and LEs). Anything else is just getting confusing.
If Robin wanted to give people the option to select what version they are rating and then create a weighted and aggregated single score (maybe based on estimated production numbers for each version), that might be fair. For example, LE ratings count for less because there are fewer of the out there. Maybe people could still view the individual scores for each version within the game's rating summary... but the master score would still be a single combination of all of them.

on the other side of that coin......LOT of bally/williams fanboys out there.

i'd say the two sides cancel each other out, if not a slightly higher B/W fanboy base.

Wtf of course. Why would them not logging in have anything to do with their opinion on a game that existed at the time they logged in last?

Because the pin existed at that time and has probably not changed since. The rating is on the pin not on the pin related to current events or the pin compared to other pins.

Given what I found this evening, there is a definite problem with the older ratings. We'll wait to see what Robin's response is. Ideally he writes a SQL update query to resolve the issue and bring those ratings in line with any that are added going forward. It would be interesting to see how that would shift the rankings since a lot of ratings would be affected (including my own).

Markmon What im saying is taste changes. When I bought my TZ years ago I thought it was the best game ever. After playing it at a local spot I bought one. After a few years of owning it my rating went down. Also TAF. Loved it owned it sold it 3 years later hating it. This is the point I was making. Non active members will not be keep there ratings accurate. I chang my rating from time to time. Am I the only 1 that does this? Also some games I don't like when I first played them. And now after it grew on me Im looking to get one. Also game rules more with the newer games. Let's say TF gets a great update. They make the code great and now its a top 5 pin. Inactive members will still have it rated as a dog when infact they may love the game now and well, didn't update there rating bc they are inactive.

This statement above is fair and true.

Though I would say there are few pins on location these days anyway . . . my parish/county has three on rout that I know of . . . Tron, IJ (Stern version), and South Park. I doubt the location/rout pins are playing much of a role . . . if anything more people may be aware of the older games due to them being in more video game applications now (granted, I know RIBON gained popularity this way as well).

Also, I think less maintenance is a bona fide benefit of the Stern pins that shouldn't (and frankly can't) be ignored. Would you prefer a NIB SM or a routed project AFM? Assuming you are not in the hobby for the fun of repairing the machines, I assume the NIB SM would win out in most people's mind despite a similar cost at the moment.

This whole Pinside rating reminds me of the BCS in college football.

People are taking this whole thing too seriously. When that happens it ruins the spirit of why the top 100 was created.

Let's please dispense of talk on having qualifiers for voting on a pinball site where people just want to have fun and rate a game they have played. It's the helicopter parents ruining Little League syndrome.

Qualifiers?

Karma points = smiley face in a post in 20 OT posts, now I can vote.
Remove older votes = Censorship, which always works well. (Being sarcastic)

I'm sure the top 100 was created to solicit opinions and not repress them.

i can only answer for myself, but yes....i do change my ratings if/when i get more plays on a pin.

i only rate pins that i've played and i try to only rate them after i've played them for a while.....but sometimes, as you've hinted at, the way i feel about a pin changes after time. sometimes it's a ratings nudge upwards...sometimes it's downwards.

i would like to think that a lot of the community that takes it seriously does the same. i have no idea if that's the case or not...