I've noticed there are roughly 160 to 180 members that log onto the Pinside everyday. There have been more then 1 thread started about the ratings and they go on and on and this made me think (smell something burning?)

So here is a very simple question: Should the Pinside even have a ratings system or not?

It's a simple question that I ask purely out of curiosity...

So please cast a vote! Thanks.

We should get rid of voting:)

How about an option for who cares?

The rating system is what first attracted me

This rating system is the best around and I like it...

+1 for who gives a flying fuck.

Keep the ratings, what first attracted me also, also read the comments

I see no reason to get rid of it. It's not exactly "in the way" if you don't like it so it's very easy to ignore. Personally, I like it.
Scott

I like the rating system, it's what got me to join pinside and really helped me decide on my first machine. It’s easy to decipher who has a legitimate write up and ranking compared to the guy who throws out a 10 or a 1 just because they can. It's not the bible, but I think its a cool feature.

Pinheads get mad at the ratings because they are afraid that the value on a machine they want to buy will go up. It's the same argument over and over. Rating is an opinion which you can choose to accept or not. It's pretty ridiculous.

who tilts these posts? some of you's don't like the foul language?
i feel we're all adults here and have all heard these words before.

Tilts don't mean shit, sooner people learn that the better...

This poll is stupid as well...I vote it closed forever...and moved to the very bottom of the forum where no one can find it again...

PinBill said:

who tilts these posts? some of you's don't like the foul language?
i feel we're all adults here and have all heard these words before.

Because some of these post are simply unnecessary to the question at hand. Tom, if you think so badly of it why waste your time replying to it?

shakethatmachine said:

I see no reason to get rid of it. It's not exactly "in the way" if you don't like it so it's very easy to ignore. Personally, I like it.
Scott

This make sense,For those that don't like it don't look at it.
I don't like chicken livers and I don't eat them.

Be nice everybody

You gunna eat that.....

Cause I like talking about pinball

The vote shows I was right, sorry if I came off harsh hahah...

We could get rid of it, but then what are people that create threads to complain about it going to talk about now? That is like a third of new threads here, gone!

It's a system that is for shits n' giggles and somewhat works. It isn't broken just because someone's favorite/overhyped/overpriced game is not number 1.

I want a tilt! I feel left out. I guess all I have to do is say a bad word. I would say "shit", but I think they allow that on cable TV now. haha

scooter said:

I don't like chicken livers and I don't eat them.

Me either!

tomdotcom said:

sorry if I came off harsh hahah...

I understand.

Thanks for voting. I'm a bit surprised by the count so far...

TheNoTrashCougar said:

I want a tilt! I feel left out. I guess all I have to do is say a bad word. I would say "shit", but I think they allow that on cable TV now. haha

I tilted you but not because you said "shit". I tilted you because I liked your name better as Garythenotrashcougar than just Thenotrashcougar. Care to explain what it all means?

Yeah! I liked my full name better too, but there is a 16 character limit on the names. Robin changed it for me.

Here are the origins of the screen name:

» YouTube video

» YouTube video

I vote we get rid of the tilts on posts. We should go old-school and have a "report to moderator" button instead.
--Scott

Looks like this thread has been tilted

scooter said:

Looks like this thread has been tilted

Indeed. Did you know that you can even tilt your own posts? I think I will tilt this one too since it is not about the rating system that people look to be abusing... and I do not like what I have to say most of the time.
--Scott

Yes.

But more weighting once a pin has been out for a while so the honeymoon period has worn off.

You have to remember that there are people out there that have not played a game that give it all zeros. That is crap. It goes the other way too.

They should put in a rating system where you are allowed to sort your favorite top 20 games. Then you can get a solid sample set to create a top 100. Easy.

--Scott

I liked the suggestion someone made in another thread - no more anonymous votes - should expose a lot of the troll ones.

NY2Colorado said:

How about an option for who cares?

this

PinBill said:

+1 for who gives a flying fuck.

+1

system11 said:

I liked the suggestion someone made in another thread - no more anonymous votes - should expose a lot of the troll ones.

I agree but solely to keep trolling out of it. I've recently noticed a member being hammered no matter what is being posted... That's the kinda thing that ruins it for everyone.

It's like democracy, it ain't perfect, but it's better than anything else out there!!!

Keep em. We're people. We have opinions. It's okay. No need to panic here.

Let's face it. The only reason anybody is even talking about the ratings system is because Tron:LE took the top spot away from the chosen one. Poor little MM getting knocked off it's throne is the sole reason for all the ratings chatter lately. People just can't seem to handle the fact that a Stern could knock off one of the holier than thou Bally/Williams offerings. You'll see it again this weekend when the B/W pitchforks come out to knock AC/DC:LE down a number of pegs.

As a Tron:LE owner I don't think it should be #1. While Tron:LE is a great pin, I like others more. So this isn't a case of sour grapes. It is merely an observation.

jarjarisgod said:

The only reason anybody is even talking about the ratings system is because Tron:LE took the top spot away from the chosen one

Incorrect. I think the fact that people are putting in zeros for games they have never played just to bring them (tron:LE) down out of the top spot is crap. That is why I like the idea of having every user allowed a "top 20" list with comments. We will then see which machine is at the top.
--Scott

well I think the numbers speak for themselves 88% of you voted- "keep the rating system "

TheNoTrashCougar said:

Incorrect. I think the fact that people are putting in zeros for games they have never played just to bring them (tron:LE) down out of the top spot is crap. That is why I like the idea of having every user allowed a "top 20" list with comments. We will then see which machine is at the top.
--Scott

I agree that is crap, but that has been going on for a long time. Don't think you can give zeros though. I believe one is as low as any go.

Look at any of the top games. They all have stupidly low ratings mixed in. It just affected Tron:LE more due to the lower sample size. The only reason that happened in the way it did is because it hit #1 on the list and a thread got started about it. The pitchforks promptly came out and it got knocked below the Pro (which I predicted would happen).

I don't think your system would be ideal either. Probably the best tweak to the system (not that one is necessary) is to throw out the bottom 5% and top 5% of the ratings for each pin. That should get rid of most of the issues with absurdly low ratings as well as people blindly throwing 10's at games right and left.

In the end no system will please everybody and the only reason we are talking about this at all is because MM got knocked off it's perch. If Tron:LE had debuted at #5 we wouldn't be seeing all these ratings threads. That is a fact.

All this talk about retooling of the ratings is great...

jarjarisgod said:

Let's face it. The only reason anybody is even talking about the ratings system is because Tron:LE took the top spot away from the chosen one.

Not me, I'm talking about the ratings because of the ratings talk, hence this thread and poll.

PINBELL said:

It's like democracy, it ain't perfect, but it's better than anything else out there!!!

Not to nitpick, but a democracy allows 51% to rule over the other 49%. A Constitutional Republic is actually the best thing out there where the individual's right and liberties are (theoretically) protected by a representative yet segmented form of government.

"A constitutional republic is a state in which the head of state and other officials are representatives of the people and must govern according to existing constitutional law that limits the government's power over all of its citizens"

"No single individual or group is allowed to exercise executive, legislative and judicial powers. Instead, these powers are separated into distinct branches that serve as a check and balance on each other. In a constitutional republic, "no person or group [can] rise to absolute power."[4]"

Jason > you are oversimplfying it. Much of the discussion has been motivated by TronLE being #1 but I think it has very little to do with that particular game and more so with the limitations of any ranking system.

The brunt of the discussion has revolved around the problems of the ratings system as it currently stands. Taking personal subjectivity out of the discussion.

10 ratings is too few a number for a pin to make the top 100 (witnessed 2 weeks in a row by relatively new games having fans/owners bias high for week 1 and then trolls bias low the following week). Neither is good for getting a real feel for a games rank and leads to people then artifically biasing later votes just to move a game up or down the list.

The system needs to have a built in self-limitation device which prevents people that have alterior motives from overhanded rating or trolling. It also needs to guide users how to rate on the spectrum (i.e. some only rate as low as 9.0, others only as low as 5.0, and others use the entire spectrum of 1 to 10.)

On a more subjective note, I do not think botique pins should even be part of the main rankings since just as an artifact of their limited nature they are not available to the general pinball populus. Hypothetically, lets say that all the recent stern LEs, magicgirl, BHZA, etc... really are the best games ever, well there is your cemented in stone top 10 FOREVER and it will never change since the owners are the only ones that can rightfully rank them and they obviously love them.

Many have said that the utility of the top 100 is that sometimes you want to add a new pin to your collection and you find a title for a decent deal but you have never played it. The rankings system used to be a nice way to get a general feel for a consensus on a game and maybe make the decisions to go for it and buy. The strength of this tool has been undermined by the recent split of pro/LE and also the current influx of many more people to pinball and pinside. The law of averages works well when you get a decent sample size (i.e. besides the current LEs, the top 20 pins all have over 300 raters and the vast majority of the top 100 have over 100 raters so the highs and lows average out to get a reasonable assessment of where they all rank).

People that truly think these botique pins are the best ever are really being negatively impacted the most since there are far more people that do not own them/ will never have a real chance to play them enough and hence a greater chance of them being artificially biased downward forever.

I think "1" CAN be a legitimate score. I rate pinball machines like this: 1 is the worst machine I've ever played, 10 would be a perfect machine. Some people seem to rate all machines from 6 to 10. Instead of holding it to pinball it gets held to the world around us. Bad pinball still has to be at least a 6, because it's still pinball. I guess in that case a 1 would be a torturous death, and 10 would be something like eating some Kobe beef while getting a blumkin from Salma (not really my type, but she seems popular around here ).

I also adjusted my ratings to fit my preferences. I really don't care about cabinet art in relation to gameplay. I think it was a good idea to make the ratings adjustable, it gives my list an accurate rank based on my priorities.

Read the ratings and try not to get defensive when a game you own gets knocked down a spot. Let's not rehash the whole thing to try to keep some of the more belligerent among us from trolling, stupid people never disappoint.

Deaconblooze said:

10 would be something like eating some Kobe beef while getting a blumkin from Salma (not really my type, but she seems popular around here ).

hahahahahaha

Can I be dressed as Shaggy and playing some pinball at the same time?

I would also be repetitively saying "Zoiks"

Whysnow said:

10 ratings is too few a number for a pin to make the top 100 (witnessed 2 weeks in a row by relatively new games having fans/owners bias high for week 1 and then trolls bias low the following week). Neither is good for getting a real feel for a games rank and leads to people then artifically biasing later votes just to move a game up or down the list.

Proves my point. Again, had Tron:LE or AC/DC:LE debuted anywhere but #1 this wouldn't even be brought up as an issue. People would have commented on them sure, but if they had debuted at say #8 there wouldn't be an uproar over it.

My point is that this stuff is being discussed because something not called MM has held the #1 spot the last couple of weeks. Not saying the discussion is bad, but wanting to completely overhaul the ratings system (like some have suggested) seems to be a pretty extreme reaction IMO.

Now that more people are on Pinside, the minimum probably could be bumped to 25 instead of 10.

Whysnow said:

On a more subjective note, I do not think botique pins should even be part of the main rankings since just as an artifact of their limited nature they are not available to the general pinball populus. Hypothetically, lets say that all the recent stern LEs, magicgirl, BHZA, etc... really are the best games ever, well there is your cemented in stone top 10 FOREVER and it will never change since the owners are the only ones that can rightfully rank them and they obviously love them.

It seems like all of the fear over the LE and JPop pins centers around them potentially knocking the perennial members of the Top 5 off their perch. If they come in and fall in the lower half of the Top 20 or lower then it seems no one will care. It is only when the MM's, TZ's, etc. get threatened that it becomes an issue.

BBB has been around for years with a limited run and it currently sits at #39. The only one of these on location that I know of is at SS Billiards. So in theory the rest (or at least the overwhelming majority) are HUO and yet it can't even manage to crack the Top 20. Less than 1,000 CC's were made and it ranks #21. I've never had the chance to play one. Don't know if I ever will. This tells me that by and large most collectors are fair when assessing their pins. Probably moreso than somebody who drops a couple quarters in a route machine and moves on with life.

Where does the line get drawn? 124 BHZA's? ~200 BBB's? 400 Tron:LE's? 1,000 WOZ:ECLE's?

Isn't it possible that one of these machines actually COULD be the best pinball machine ever made? Why should it be treated differently just because a limited number were made?

The reality is that we live in a day and age when production numbers of pinball machines are far lower than they were in the 90's. Not sure why the current pins need to get penalized due to the current economy.

Things have a way of working themselves out. If that isn't good enough for some then raise the required rating limit from 10 to 25 and drop the bottom/top 5% and call it a day. Life will go on. It is only an arbitrarily ordered list of pinball machines after all.

Whysnow said:

Many have said that the utility of the top 100 is that sometimes you want to add a new pin to your collection and you find a title for a decent deal but you have never played it. The rankings system used to be a nice way to get a general feel for a consensus on a game and maybe make the decisions to go for it and buy.

How common is this, really? I think the *comments* on the ratings are a good way to get a sense of whether you might like a machine or not, but I have a hard time imagining a scenario where the only thing someone uses to make a decision is a game's position on the Pinside Top 100. Who is going to decide to buy or not buy a game solely because it is #75 on the Pinside Top 100 instead of #60? Or (an even more absurd scenario), #3 instead of #1? Or because a machine has an average rating of 7.789 instead of 7.881?

Jason> the 'fear' is more so that the ranking tool loses all utility when 200 pins are owned by 200 people and they all rank it a perfect 10. Upward bias is just as bad as downward bias.

If I have an investment in a LE pin then I will naturally have a bias upward. LE pins by nature have a smaller population that can even rank them because they are not found in the wild very often. They also fall even more into the collectible category and less in the game category for some that own them. I prefer the ranking to be based on the overall 'game' and less on the commodity of a collectible.

Take AC/DC LE for example, 20% of the rankings think it is the perfect pin and this is an obvious upward bias IMHO. I am not saying their opinion is wrong but saying that since 20% of the votes are what actually put it at number 1 this week, 4 rankers had an ability to alter the top 100 this week, put a target on the back of AC/DC LE, and hence further bias for the long term of the top 100. It will surely eventually end up with a nice average that fits a greater consensus, but the aggregate impact has forever impacted the true accuracy of the system in a negative way.

If I were to draw a line in the sand for a botique pin list which is seperate from the regular list, then I would say anything less than 500. An edition size of 10% of the average normal edition (5000?) seems like a reasonable cutoff. Also with the total number of pinside members I think an edition size of less than 500 reasonably precludes the majority of the population. I would agree that the BBB and CC, etc old school pins should also be removed from the regular list. I actually think an only botique pin list would be a very useful tool also.

Last thing, I actually do use the rankings system to help seek out new pins to play/maybe buy someday.

jarjarisgod said:

throw out the bottom 5% and top 5% of the ratings for each pin

Like this idea.

Whysnow said:

Take AC/DC LE for example, 20% of the rankings think it is the perfect pin and this is an obvious upward bias IMHO. I am not saying their opinion is wrong but saying that since 20% of the votes are what actually put it at number 1 this week, 4 rankers had an ability to alter the top 100 this week, put a target on the back of AC/DC LE, and hence further bias for the long term of the top 100. It will surely eventually end up with a nice average that fits a greater consensus, but the aggregate impact has forever impacted the true accuracy of the system in a negative way.

So you're saying that because 4 people gave the LE perfect 10's (whereas the other 22 raters did not) it invalidates the entire ratings system?

This is the first week that the game is on the list. Give it time. Like you say it will eventually settle in. Historically, pins always debut high and then come down.

Don't worry. MM will be back on top next week.

Whysnow said:

If I were to draw a line in the sand for a botique pin list which is seperate from the regular list, then I would say anything less than 500. An edition size of 10% of the average normal edition (5000?) seems like a reasonable cutoff. Also with the total number of pinside members I think an edition size of less than 500 reasonably precludes the majority of the population. I would agree that the BBB and CC, etc old school pins should also be removed from the regular list. I actually think an only botique pin list would be a very useful tool also.

In that case, CC should be included since 900 of them were produced. At any rate, I completely disagree with this line of thinking. Of course people will think I'm biased because I own an LE, but I don't care.

Here's my reasoning... A lot of this line of thinking seems to be based on the idea that all of these games go into collectors' homes and they are played by a select few people. In some cases this may be true, but I think it is largely a false assumption.

As an example, I have a Tron:LE. I am one person who owns it. Of course I could have made the choice to never let anyone else play it. Instead I would estimate 20-25 people who don't own it have had the chance to play my one example over the last 9 months. I'm sure my case is not the exception, but rather the rule.

Case in point... Earlier this year I went on the Ball Crawl in the Twin Cities and got to visit 3 different collectors' homes and collections. That's where I got to play TF:LE along with a number of other non-LE pins I had never had the chance to play before. There were a TON of people at each stop along the way. Heck, I even got to play Flipper Football at Bryan Kelly's place. Never thought I'd have the chance to play one of those, but it happened.

The point is that, from what I can tell and a number of threads on Pinside seem to support, location pinball is dying. There has been a big shift over the last few years and now it seems that the home is where the majority of pinball is being played. There are pockets like Portland, OR where location pinball thrives, but these cases are sadly the exception rather than the norm.

These days if you want to play pinball you have to network. Sad but true. If I hadn't networked with people I would have only played a handful of machines by this point. Instead I've been fortunate enough to play the majority of pins I want to play. I'd rather go to any of my friend's houses to play pinball than go to any location in Madison to play. Why? Because my friends have better collections than any location does!

Plus, most of the people who come to this site are pinheads who know other pinheads. Odds are in most cases they'll have a chance to play the majority of the pins they want to. I don't think there are many casual players strolling on to Pinside every day.

IMO excluding machines because they are by and large going into home use environments doesn't compute. Heck, the only MM I've ever seen on location is at SS Billiards in the Twin Cities but I've played 4 or 5 of them in various collectors' homes. Better yet, I've only ever played MB in collectors' homes. Should that one be excluded from the list as well?

It all seems very arbitrary to me. The game has changed. This is the state of pinball today. Time takes care of everything. Including these ratings.

Good points Jason. I think maybe I am shifting on the boutique ranking idea, since like you said all pins are no longer really location pins... I just need a better network Would love to come play your pins again if you will have me over. Maybe I will change my mind on Tron with more playtime?

My point with the AC/DC was more-so that upwards bias at onset is bad for the overall stability and utility of the ranking system. Upping the number of needed rankings prior to list-ability in the top 100 would help greatly with this I think.

I still would really like to see a self-limiting aspect to the ratings. I do not know the perfect way to do it, but I would prefer that people are forced to utilize the spectrum more fully and actually make their own individual rankings. In otherwords, not everypin that someone ranks should be between a 9 and 10 IMHO. I also see many examples where someone ranks their #1 at a 10.0 and then #2 through #10 between 7 and 5. If there was a way to deter this behavior you could potentially equalize the voting/ranking so a person can not make their ranking 'worth' more by merely being more extreme and actually pushing ratings up/down in an attempt to move the mean.

kmoore88 said:

PINBELL said:It's like democracy, it ain't perfect, but it's better than anything else out there!!!
Not to nitpick, but a democracy allows 51% to rule over the other 49%. A Constitutional Republic is actually the best thing out there where the individual's right and liberties are (theoretically) protected by a representative yet segmented form of government.
"A constitutional republic is a state in which the head of state and other officials are representatives of the people and must govern according to existing constitutional law that limits the government's power over all of its citizens"
"No single individual or group is allowed to exercise executive, legislative and judicial powers. Instead, these powers are separated into distinct branches that serve as a check and balance on each other. In a constitutional republic, "no person or group [can] rise to absolute power."[4]"

Correct.

Democracy sucks by comparison. But is still better than dictatorship, communism, marxism, socialism, etc.