Unfortunately, the comments attached to the new and probably exaggerated ratings never give much insight into why they think these games are great. But it’s all good fun and not meant to be taken seriously.

This has been gone over several times in other top100 discussion threads.

Whenever there's a new release, it shoots up for a short time, then settles down further down on the list when some more realistic reviews come in.

Um, I think those 5 pins OP mentioned are all damn fine
Today’s pinballs are amazing technical marvels, so I’m pretty happy with those titles sitting in our Top 10

Think of the list more like the Top 20 countdown. New titles usually populate the higher slots for a brief time but they ultimately settle into the list as they are replaced by the "new hotness" and the cycle repeats.

Beating a dead head, trust me. There is no real want to make the top 100 better for some reason.

Very easy to use the median and not the average. Lose the top and bottom 10% of ratings (or something similar). Still wouldn’t be perfect, but it would get rid of a lot of the 10 and 1’s.

I don’t see it as a top 100 of all time list. More of a top 100 of the week list.

I guess it has been a couple months since the last one.

https://pinside.com/pinball/forum/topic/the-top-100-should-be-closef
https://pinside.com/pinball/forum/topic/pinside-top-100-complete-joke
https://pinside.com/pinball/forum/topic/is-pinside-top-100-credible-anymore
https://pinside.com/pinball/forum/topic/bias-in-the-pinside-top-100
https://pinside.com/pinball/forum/topic/i-think-the-top-100-should-be-revamped
https://pinside.com/pinball/forum/topic/top-100-rating-system-broken

I am amazed that those with the data behind the scenes have not seen the new trend. Many don’t care anymore about Pinside ratings so they stop down rating them after the hypers up rate.

Pinside ratings become a bigger fail with each successive new Stern game release. It really comes down to the growth of this hobby has become lots of NIB buyers that don’t work on games. These people tend to buy each new Stern game and rate them highly (despite how many come up for sale in 6 -12 months with only 100 plays on them). Just the reality of the growth in pinball and the new people and their money and skills tend to lean towards NIB.

Taking the median would be the easiest way to fix much of this. Limiting all new games to 6 months hold off for ratings may help also?

CONGRATS! you’re the one millionth person to post about this.

You know, Pinside does have a custom top 100 generator that allows you to filter based on what you feel is important.

pasted_image (resized).png

At least 100 rankings.

Scared stiff and MET. Yuck.

D4043E8B-65EE-4B68-9E97-FF9518025797 (resized).jpeg

They already have. Here’s the top ten you suggested.

63B2C378-E411-42E8-8B99-955703E43E03 (resized).jpeg

Was thinking TMNT might stay up there longer than the normal new release because location play is pretty much shut. So only people rating are early adopter owners....and we know how they’re inclined to vote.

Hi force,
What is an « approuved pinsider rating »?
Only need 25 rating but what i can see for Turtle, only 3 or 4 pinsiders that rated it have around 35-50 ratings. And We can’t see others that do not write a comment. Why?

Why an approuved rating doesn’t come from a pinsider that have rated at least 25-30 games with a fair and detailed comment?
Thanks

Yeah, I’ve always wondered, too, what an approved rater is. In general, though, every time I play around with the custom rankings I get results which I do not understand. It would be great if someone could give us pinheads a bit of insight into how the rankings work.

I don't know what the exact thresholds are for individual members. That might be a question better directed toward @Robin.

For a game to be including in the rankings on the top100 list, it needs to receive 25 ratings from approved raters.

TMNT Premium has not yet received enough ratings to be included, for example.

the issue may be that the Pinside “Top Pinball Machines of All Time” Rankings seems to be one of the “Top 10 things you care about too much”. Do what I do, I couldn’t care less. Problem solved.

Damn right it is. It’s the best table ever made! It hits every note I could ever want in a pinball machine. It’s a world under glass. Every time all the worlds melts away, and I’m transported into the battle. Everyone else should stop making games, because we’ve reached the top of the mountain. I’m very excited to play my first game on it someday. 10 of 10.

You know... I think you're the first to bring this topic up!

Lol.

you should rate it (as you have some playing hours on it to come)!

The reason why TMNT (and similar "new" titles) always get "on top", is that early on, nearly the only people who have played them are the people who bought them early. And these people always rate them very highly for a few reasons...

1) If they didn't like the theme/design they probably wouldn't have bought it (except for those few people who buy EVERY new machine).
2) They want to feel that they "made a good choice" and "got their money's worth".
3) The "new" toy is usually the favorite toy, until that newness wears off.

I would think that a machine should not be listed until like 6 months after release. Just limiting it to 100 ratings or something will not do much (as it will still be the first 100 purchasers that are rating it). It really needs to be on location for a while, so people who haven't invested in a purchase also get to vote on it. Of course, the pandemic has made this even harder for now, as location play has dropped off greatly.

I would like to see an alternate Top 10 (or maybe 25) that is computed by everyone submitting their personal favorites list (in order). This would mean that everyone would be voting using the same "scale", and it would be much harder to try to manipulate the results (as the best you could do is rate 1 machine your favorite, and the worst you could do is not list a machine at all). No more "vote a 10 for all the machines I own and vote a 1 for all the others".

What does it matter? What are you trying to use the list for that necessitates a change? Just keep your own personal top 100 list?

The point is the list is not for yourself...it's for everyone else.
And to see how your list compares to everyone else's.

If some game (say TZ) is listed near the top, then it says to people "this is a great game, you should check it out".
And you may think WH20 is the best game ever, but other people list it as 35th.

Obviously, everyone already knows their own list...it's learning everyone else's list that is interesting.

agree, hard to go wrong with any of those. Not a huge Elvira fan but the rest are pretty darn amazing.

Yes.

Also, is the price bubble about to burst?!

I have found this to come in very handy. I zero out everything and give a full 100% to "The newest Stern LE" or "JJP's CE of-the-century". In a sense of fairness I only go 90% on "The latest Spooky", as I have to draw the line somewhere...

I think the minimum need to be more like 300 to 500 reviews

Frankly, it is amazing the 90's games have held up as well as they have over the years in the face of better technology (such as full RBG lighting, LCD displays, etc.) and the ridiculously deeper code and improved sound and sound packages of modern games. I understand the 90's games had more toys that interacted with the ball and what not...but if you put a stock original MM up next to a stock Stern Jurassic Park LE the differences are readily apparent.

That said, I do personally agree some sort of rating system overhaul would be welcome, I've long argued against legacy votes from years ago from people that are no longer part of the community staying in the system. There are a lot of small things that could make the rating system more "legit" but as chuckwurt already noted these things are built in now...you can make the top 100 take into account many different factors if you do not like the default settings.

I appreciate all the input.

However, the notion of me not caring is quite ridiculous. Of course I care. However, there are some obstacles:

1. Changing the Top 100 would be a huge undertaking and I'm not sure I'm ready to dive into 10 year old code and mess with it at this point. I already have a lot of stuff I'm working on.
2. I agree that some aspects of the system could probably be improved but any change will bring forward another group of people who disagree with the list. The whole concept of rating is kind of... prone to abuse? Will always disappoint people? That's just the nature of top lists.

Some years ago I had a mail exchange with Lyman about this topic and we came up with a great alternative for the top 100, I really liked it. It boils down to this: every Pinsider would be drag and dropping their own top X list together. A forced list, where each position garners points. Top game gets, say, 100 points. Seconds game gets 99, third get 98. Etc. People could easily drag, drop and reorder their ratings as preferences change all the time. I know mine are. You know, after the honeymoon phase wears out, many games slide down my list. Heck, my own current ratings list is outdated as I no longer think MET is my favorite game.

I would very much like to implement that system. Possibly it would replace the current top 100 system. Don't worry, the reviews would remain as they are very valuable. The new system would still have a review option.

Anyways, I will try to dig up that email exchange. But gotta get the upcoming Pinside update done first. It's been waaaaaaay to long already.

I think it would be a shame to scrap the existing system. I like it. That being said, the proposed system you just described sounds like good fun - I would vote for having both. Keep up the great work!

This is a great idea then someone could not vote all of their favourite games as a 9/10 and all others at 1/10. I like it.

I really like this!

this is pretty much verbatim what I suggested to you a few years back.

Great minds think alike I guess

Forced ranking is prefect. Start that forced ranking with also forcing people to rank 10 games. Best becomes a 10. Worst becomes a 1.

Then as they add more rankings, your point scale becomes more granular.
This would help prevent the fan boys from ranking each new Stern a 10 and instead they would need stack rank them.

I would also suggest that people need to have 10 rankings but are only allowed to rate up to 3 games in any given day.

Create user investment in the ranking/rating system, rather than make it simple for abuse.

The issue with this is that if you had someone who only played awesome games or someone who only played crappy games, then their rankings would either artificially bring up or bring down the scores if you just average them. If you had forced-choice ranking data, I think it'd be better to come up with a score that could be used to estimate the probability that it would rank higher in a typical ranking than another machine (along the lines of https://en.wikipedia.org/wiki/Bradley%E2%80%93Terry_model).

Lol go figure Lyman now figures out how to fix our Top 100. What can't the man do (...besides code a future Pro model since Gary now considers him a Premium/LE/SLE feature)?

not needed and creates implicit bias of assumption.

Force a minimum of 10 rankings, stack rank those (you can set a defined range for start point and increment), and the rest will come out in the wash after enough rankings.

Keep in mind the main problem with the currnet system is the wholesale abuse and lack of scale parameters. There are people with only 9-10 for the games they own and 1s for everything else.

The problem with this is that it would take a long time before you had enough pair comparisons to provide useful results (even if you forced a question at each log in). And by the time you do have useful data, a bunch of new machines will have come out - invalidating the data you finally have.

This is a great method if the data being compared is mostly static (i.e. listing the "best" presidents), but not so good when the choices keep changing (and when people tend to change their minds a lot).

I think one of the problems is treating LEs, premiums, CEs and pros seperately. At least premium and Le should be in the same group in terms of score. And Ce and Le. Sometimes LEs and premiums are 8 positions apart in rankings (EHOH). Seems weird when they play the same. This seems to skew things. It also creates a lot of repetition where people post the same review across three different versions. Plus it means that some reviews aren’t seen at all because they are not in the highest ranking category for that pin.

Btw, thanks Robin for the top 100 and all that you do. I like the top 100. Pinside is one of my most pleasant ways to waste time.

The fault isn't in the system or the way its set up, the fault is with the people using it. The custom filters really are the best solution.

They don't always get to the top right away.

Two great pins, TWD and BM66, started out in the 100's for various reasons.

BM66 due to such early code and some Stern hate based on the way they launched that game and SLE send in a video request etc.

TWD, another Lyman game, had really early code too and was difficult to get dialed in.

Both games had potential greatness and finally got there.

THUS, due to the early barrage of negative voting those two pins will never reach the level they deserve based on more recent ratings.

DRAG and DROP!!! Yes, please do this. Here is my version of this idea...

https://pinside.com/pinball/forum/topic/suggestions-for-new-ranking-system

I think if you try to remove the nostalgia bias these new games are objectively better than the oldies in many ways. Iron Maiden and Jurassic Park are true master pieces... for sure deserving of spots in the top 10. TNMT has already fallen since you posted this a month ago.

?

So by this argument the IFPA would be fine ranking players using a system where they just averaged your finishing place in tournaments and your rank would just come out in the wash?