(Topic ID: 285776)

Abuse of rating system.

By rai

3 years ago


Topic Heartbeat

Topic Stats

  • 229 posts
  • 89 Pinsiders participating
  • Latest reply 2 years ago by Doctor6
  • Topic is favorited by 4 Pinsiders

You

Linked Games

No games have been linked to this topic.

    Topic Gallery

    View topic image gallery

    934B57EF-552F-4903-86D9-BE3E432A4159 (resized).png
    8CBF4AC1-0775-40EB-BBB4-D379C84E6ACE.gif
    Keyboard commandos.gif
    04DA4240-C1A2-4B4B-BBD9-1B1FC11F9E4E (resized).jpeg
    IMG_20201231_184528229 (resized).jpg
    05969F5A-388B-4C11-88E0-A6B7419F0ED9 (resized).jpeg
    Let the Hate Flow.png
    BF3689B2-BF92-46C1-9523-47A4BD96D639 (resized).jpeg
    toysrating (resized).png
    5168BF12-AC2F-4403-A3C1-C064798D3E78 (resized).jpeg
    E94AB8F8-EE85-49C1-A898-933DCD69AB8B (resized).png
    31BADAB9-9E73-489A-ACAD-7196BB449B23 (resized).png
    FC93CA11-0D34-4A6B-8D8A-B024454FD023 (resized).png

    There are 229 posts in this topic. You are on page 2 of 5.
    #51 3 years ago
    Quoted from TreyBo69:

    Just for fun I ranked Star Wars Pro with the system. I ended up with it being a little over 3/10. Which...yeah that’s close to my opinion of that game relative to games of the past several years. I’d probably would have gone in saying 4/10. (Well really Id say it’s a 2/5 which sounds a lot better than a 4/10 even if it’s functionally the same)
    I won’t get into why I don’t like the game but I am pretty meh to it overall. It’s not that I don’t get the game, it’s just whatever to me compared to others. But I’ll rate stuff harshly...
    It also reminds me, the top 100 should really break out the early SS games with numerical and alphanumeric scoring. It’s so hard to compare DMD/LCD games with the older ones. Heck it’s hard to compare a lot of 90s DMD games with the latest LCD games. The expectations change so much over time.

    That’s a fair assessment and I think we can agree there is a big difference between a 1.0 and 4.0 and the affect it has on the overall average.

    #52 3 years ago
    Quoted from Psw757:

    You may want to not get so pantsed up and read what I said, I didn’t say throw 5 and 6 out but there is no modern pin that is a 4 or below, period.

    Disagree. Some games are complete and utter dumpster fires. Shaq attack has almost no redeeming features in it whatsoever. The art is terrible. The sounds are repetitive and awful. The layout is terrible. The rules are abysmal. If told I could chose between shaq attack and no pinball for the rest of my life, I'd go without.

    The Beatles is almost as bad to me. It's nails on a chalkboard. It's a recycled layout where I'd rather play the original.

    #53 3 years ago
    Quoted from dung:

    Disagree. Some games are complete and utter dumpster fires. Shaq attack has almost no redeeming features in it whatsoever. The art is terrible. The sounds are repetitive and awful. The layout is terrible. The rules are abysmal. If told I could chose between shaq attack and no pinball for the rest of my life, I'd go without.
    The Beatles is almost as bad to me. It's nails on a chalkboard. It's a recycled layout where I'd rather play the original.

    While I widely disagree on Beatles, neither of these games are a 4 or less. Pretty tough to be that bad.

    #54 3 years ago

    I can't recall the last game I ever bought because of ratings... because I've never bought one based on ratings. Back when it was just IPDB, all I'd go by was the written review. I quickly found the actual number to be useless. That carried over to Pinside ratings as well. They are useless. When I read a review, I can tell what the person was trying to say. The funny part is that if you look at my ratings, I did some manipulation during an experiment on them.

    Step 1. Read what they wrote, not the numbers.
    Step 2. Play the game to see if you like it.
    Step 3. Prophet??? (this means telling people how you bought the game before it was popular, not the making money homonym.) (right after you have to state "No homonym" for it to be ok with everyone).

    #55 3 years ago

    Top and bottom 10% should be deleted from all ratings

    #56 3 years ago

    Modern games I would give a 4 or less:

    Cosmic carnival
    Thunderbirds

    #57 3 years ago
    Quoted from chuckwurt:

    While I widely disagree on Beatles, neither of these games are a 4 or less. Pretty tough to be that bad.

    Disagree, if something can be a 9-10, something has to be a 1-2 otherwise there is no point. I have games I have rated 9-10

    Since this is subjective, you might not feel like shaq or beatles are that bad. Perfectly valid. You might even like them and that is fine too.

    I don't. I hate them. I have owned shaq. Beatles I had to play a lot due to league. I have time on them. I hate them. For me the are terrible. They are not fun. They are not enjoyable experiences. My rating reflects that.

    Both of us are valid.

    How this should work is that if I have a low rating of a popular game, it should get averaged out.

    Quoted from chuckwurt:

    Modern games I would give a 4 or less:
    Cosmic carnival
    Thunderbirds

    Haven't played thunderbirds. Cosmic would be low, should rate it since I owned it. Its saving grace is the art. After that nothing else. Build quality was poor. Code non-existent. Shot layout was ok, not great, but the scoop up the middle was a terrible idea.

    #58 3 years ago
    Quoted from chuckwurt:

    While I widely disagree on Beatles, neither of these games are a 4 or less. Pretty tough to be that bad.

    agree. Beatles is a fun shooter.

    #59 3 years ago
    Quoted from dung:

    Disagree, if something can be a 9-10, something has to be a 1-2 otherwise there is no point.

    Of course. But a lot of the games that are dogs either didn’t see the light of day, or many haven’t played cause you never see them.

    As far as games that are regularly seen and played, it’s rare I would rate something below a 5-6

    #60 3 years ago
    Quoted from chuckwurt:

    Of course. But a lot of the games that are dogs either didn’t see the light of day, or many haven’t played cause you never see them.
    As far as games that are regularly seen and played, it’s rare I would rate something below a 5-6

    But that is you. I seek out the stuff that you rarely see. Just because they made a ton of a title, doesn't mean it was even decent.

    Shaq sucks. Its ugly. It got made because gottlieb was a name at the time. Mystery Castle is an amazing game. Its one of my all time favorites, but because alvin g had a poor distributor network among other things, it barely exists.

    #61 3 years ago
    Quoted from dung:

    Beatles I had to play a lot due to league. I have time on them. I hate them. For me the are terrible.

    I thought it was based on Seawitch, which everybody said was so great.

    -1
    #62 3 years ago

    There's no way that any game with Zombie Yeti, Chris Franchi, or Dirty Donnie artwork (like Cosmic Carnival) should be rated a 1 overall. Even if the gameplay stinks, you can't honestly rate the art a "1". Even if you don't appreciate their art styles, just the fact that it is hand drawn and not photoshop would bring it above a one.

    #63 3 years ago
    Quoted from phil-lee:

    I thought it was based on Seawitch, which everybody said was so great.

    It is. It doesn't have an unnecessary spinning disc. It has art that doesn't look like someone had too much sherbert and threw up all over the pf. It's not stuck with early beatles music which I have heard to death. Seawitch is actually fun.

    Remakes aren't always a bad thing. DE/Sega/Stern has a long history of stealing layouts. JP to me is better than WW. BF isn't better than STTNG but its fun. Tommy and Tron were great reuses of the FH layout. This wasn't one of them. For me its worse and I rate it as such.

    #64 3 years ago
    Quoted from Isochronic_Frost:

    a “most favorite” to “least favorite” slider is better than 1-10 because nobody uses all the numbers.

    What if every slider had the other games you've rated shown on the slider and you have to rate the one you're working on between the ones you've already done?

    toysrating (resized).pngtoysrating (resized).png
    #65 3 years ago
    Quoted from rai:

    This Lowezz 2 days on Pinside actually has unlocked an achievement as ‘ratings expert’ on Pinside. He’s not picky what he rates down either like all the top pins.
    [quoted image]

    If your new on Pinside you may have collected and played many games so rating many ratings is all good. I personaly think your rating are way to high with most in the 9s I have 25 rating between 6.7 and 8.4 I never respect all the 1s and 10s I see on Pinside

    #66 3 years ago
    Quoted from rai:

    Might be there should be a way to toss out any votes of 1.0 or 2.0 or so on as bogus.

    If the same was done with 9s and 10s 21 of your 28 ratings would be tossed

    #67 3 years ago

    Or people could just use discernment.

    #68 3 years ago
    Quoted from John_I:

    Top and bottom 10% should be deleted from all ratings

    But then 11-20% and 80-90% will be argued the top games as they are now and we’d have endless questions on which game really is the best and the worst since the top and bottom games have been censored.

    #69 3 years ago

    I think that if there were 10 and not six spot to fill in it would make a huge difference. If I find 3 or 4 things a 4 and not a 5 or 6 the rating dives on that game.

    #70 3 years ago
    Quoted from Doctor6:

    we are getting pantsed up

    Its my day off, Im not getting pantsed up...

    #71 3 years ago
    Quoted from pinballinreno:

    Its my day off, Im not getting pantsed up...

    The day is early yet

    #72 3 years ago

    Just go read what Caucasian2Step has to say about a particular machine and call it a day...

    #73 3 years ago
    Quoted from fireball2:

    Just go read what caucasian2step has to say about a particular machine and call it a day...

    ... another shout out to caucasian2step

    I pretty much ignore the numerical ratings, and I would not shed a tear if the top 100 disappeared. I do like to read the comments though, and always find some insight or something I had not thought of when I dive deep into a particular title. For me though, caucasian2step is the gold standard of pinside ratings.

    #74 3 years ago

    I've rated a few games over the years, but the ratings never really mattered to me anyway. I know what I like and other people's ratings never factored into my decision on whether or not I like or buy a game.

    #75 3 years ago
    Quoted from pinmister:

    Another major flaw is many peoples ratings are old and need to be updated. Have you ever written off a machine-only to dig deeper and find out your opinions have changed?

    I am definitely guilty of this, and I probably need to go back and edit some of my reviews and ratings from years ago.

    #76 3 years ago

    I’m literally a brand new Pinsider (as of yesterday) who has already realized how to solve this issue for my own purposes. I use the custom configurator for ratings and divide the Solid State machines into 1975-1999 and 2000-Present. This separates the 2 modern eras so they’re not directly compared in the single numerical sequence.

    Like pre-Superbowl era and Superbowl era NFL teams, I don’t consider the machines from the 2 major SS eras as being comparable for many reasons. And I doubt this issue is possible to police into it not being a factor in ratings, especially at this point. The human elements are the problem, not the system itself.

    In the end, I make my own extensive lists for myself. The ratings are just general reminders and guidelines to me. What’s the real functional difference between a 7.867 rating and a 7.779 rating anyway? There are effectively clumps of 3-way to 8-way ties from aggregate mass subjective opinions. I don’t prioritize restaurants, movies, books, cars, or much of anything by such criteria.

    #77 3 years ago
    Quoted from dung:

    Disagree. Some games are complete and utter dumpster fires. Shaq attack has almost no redeeming features in it whatsoever. The art is terrible. The sounds are repetitive and awful. The layout is terrible. The rules are abysmal. If told I could chose between shaq attack and no pinball for the rest of my life, I'd go without.
    The Beatles is almost as bad to me. It's nails on a chalkboard. It's a recycled layout where I'd rather play the original.

    I enjoy shaq attack. Worse than no pinball? No way

    #78 3 years ago
    Quoted from EJS:

    But then 11-20% and 80-90% will be argued the top games as they are now and we’d have endless questions on which game really is the best and the worst since the top and bottom games have been censored.

    I didn't mean to delete the games from the list. I meant the top and bottom ratings from each game. So if a game has eight people rating it as of 10.0 and 8 people rating it as a 1.0 you would delete those and just base the ratings for that game on the rest of what's left because those are more valid than someone just giving all ones or all tens.

    #79 3 years ago

    The moderator gave everyone the answer. I think he was pretty clear that certain reviews are not factored in. There are many ways that bogus reviews can be weeded out. This is done on many sites for various hobbies and interests throughout the internet. Another example would be the Olympics. In certain sports that use judges, the highest and lowest scores are not factored in the scoring to help eliminate biased judges. I am pretty sure the same thing is going on here. The person who posted all those reviews of 10 and 1 probably doesn't factor into the averages.

    #80 3 years ago
    Quoted from TigerLaw:

    The rating system may be flawed due to various biases of the people rating it, but once you find a half dozen or so Pinsiders who have similar taste to yours you can tell a lot about a game by how they review a game in their comments

    Yeah but u still need to play the game. How many films do and ur buddy's love but yet there always those few that u love and they think are shit. I don't think any pin is worse than a 4. make that the lowest and it would prevent a lot of the crap. I mean how do u give a pin 1? I tell ya how. Severe case of Fuckfaceitis.

    #81 3 years ago

    How about only being able to enter 2 or 3 ratings per day. This would cut down on spam abuse. 51 ratings in 1 day? Moderators should remove that.

    #82 3 years ago

    Can we get a rating system for the threads based on the rating system?

    #83 3 years ago

    Ratings is such a tough thing to regulate. With the ability to adjust the weight of each category in the rating, it reduces how comparable each rating is to another. For this reason, I've always left the weighting as the Pinside default. I've rated about 60 games now, and all fall in the 6.8-9.4 range. Each time I rate a game, I put in my full honest gut feeling across the board. If the result is lower or higher than I think it realistically is, then I will adjust as necessary.

    I'm guessing a lot of people don't put much nuance into these ratings. Just like so many things in life, the very vocal fan boys and haters end up sticking out because they are putting ratings that really are far from realistic.

    #84 3 years ago
    Quoted from bangerjay:

    Can we get a rating system for the threads based on the rating system?

    I give this one a 4

    #85 3 years ago
    Quoted from beelzeboob:

    If you give any game a 1, you probably shouldn't be in this hobby.

    Even Thunderturds is a solid 2. Or maybe not solid, but it's definitely #2.

    #86 3 years ago

    I'm very consistent in my rating - if I own a game, I give it all 10s.

    #87 3 years ago
    Quoted from no-extra-balls:

    I enjoy shaq attack. Worse than no pinball? No way

    And you aren't wrong, for yourself. For me, it is one of the biggest wastes of time I have ever experienced. It's fine we don't agree.

    One thing I dislike about the rating system is a game can have meh art, meh sound, and be a blast. Can be your favorite game ever, but it can rate lower than games you hate. Frankly a slider of 0-100 for an overall score would be more effective. Much easier for you to say game A is better than game B, but overall game C is far superior. Besides some of this is stupidly subjective. Do you like the art style of the time period? Do you like the theme? If not good chance you will hate the art.

    #88 3 years ago
    Quoted from dung:And you aren't wrong, for yourself. For me, it is one of the biggest wastes of time I have ever experienced. It's fine we don't agree.
    One thing I dislike about the rating system is a game can have meh art, meh sound, and be a blast. Can be your favorite game ever, but it can rate lower than games you hate. Frankly a slider of 0-100 for an overall score would be more effective. Much easier for you to say game A is better than game B, but overall game C is far superior. Besides some of this is stupidly subjective. Do you like the art style of the time period? Do you like the theme? If not good chance you will hate the art.

    this is actually a great idea.

    #89 3 years ago

    I am not sure how this would work algorithmically, but what if each Pinsider rated his/her games they've played in order from most to least favorite - no numeric scores per game. And then using data from all the Pinsiders, you compile a top to bottom list, again with no numeric 1-10 ranking...just most liked to least liked. Just a thought.

    #90 3 years ago
    Quoted from brucipher:

    I am not sure how this would work algorithmically, but what if each Pinsider rated his/her games they've played in order from most to least favorite - no numeric scores per game. And then using data from all the Pinsiders, you compile a top to bottom list, again with no numeric 1-10 ranking...just most liked to least liked. Just a thought.

    I realize this would need lots of tweaks, like what happens if some people only have one or two games on their list. I imagine it would be weighted based on number of games on your list, pinside cred, etc. Maybe it wouldn't work, but it just popped into my head as an alternative

    #91 3 years ago

    I think part of the problem is it isn't set up as any sort of forced ranking or true 1-10 scale. When you actually rate a game, you choose between one and six "balls" in multiple categories. These are labeled as Terrible, Poor, Not So Good, Decent, Good, and Excellent. In order for someone to rate games low, they have to say that it is terrible or poor in multiple categories (depending on how they have their preference sliders setup). On the flip side, it's easy for a game to get rated higher than necessarily intended if you think it is excellent in multiple categories.

    For example, my ratings show that I think AIQ is a perfect 10 and my top rated game. In actuality, I do not think it is either of those things. I do however think it is excellent in every rated category. But a perfect 10? Nope...I don't think any game is necessarily perfect. But in order for me to "sandbag" it down to the 8.5 or so that I would actually rate it at, I would have to lie and rate multiple categories as poor or "not so good", and I can't in clear conscious say that about anything on that game. I really do think every category deserves better than "good" on that game...and that defaults my rating to a perfect 10.

    On the flip side take a game like Ghostbusters, which I think it one of the worst modern games. No, I have not rated it as I have not put enough time on one to give it a fair rating. But, if I were to rate it, there is no way I could rate every category honestly and have it be a 1 or even something like a 4. There is no way I could call the artwork, sound, or even the code or layout as being terrible or even poor. If I rated it there would be a couple not so goods, a few decents, and even a good and excellent or two. And that's a game I do not care for and if I had to force rank would be on the bottom of modern games.

    The system just isn't set up to use the entire 1-10 scale if you are answering honestly. I'm not saying I know how to make it better, other than throw out ratings that are obvious trolls with an agenda. But even then...how do you define that, at least on the high side? Am I a "good troll"/ratings pumper because I rated AIQ a perfect 10? How do you mitigate that for people that are answering honestly?

    #92 3 years ago
    Quoted from dung:

    And you aren't wrong, for yourself. For me, it is one of the biggest wastes of time I have ever experienced. It's fine we don't agree.
    One thing I dislike about the rating system is a game can have meh art, meh sound, and be a blast. Can be your favorite game ever, but it can rate lower than games you hate. Frankly a slider of 0-100 for an overall score would be more effective. Much easier for you to say game A is better than game B, but overall game C is far superior. Besides some of this is stupidly subjective. Do you like the art style of the time period? Do you like the theme? If not good chance you will hate the art.

    You touched on a major point I was trying to make earlier.

    A game can absolutely blow from a gameplay standpoint but have great art/sound. If this is the case and rated appropriately it would be impossible to be below a 3/4.

    The opposite is true as well, great gameplay but shit art or sound.

    The problem here is the folks that carpet bomb every category as a 1.

    Even Jetsons has good art and hard for it to be rated as a 1 overall.

    I really like the idea of using the trimmed mean system. Keeps everything in check.

    -1
    #93 3 years ago

    Stern Star Wars is the number 2 rated game. I think just that proves ratings are useless.

    -9
    #94 3 years ago
    Quoted from Eightball88:

    caucasian2step is the gold standard of pinside ratings.

    With TZ his highest rating? Come on son.. With all the great games that have come out since. He does have great reviews, but hard to teach an old dog anything.

    It is interesting to look at our own reviews and see how we have changed. IJ is still a 10 but I have to admit there are some games now that are above a 10. That is the problem with perfection, it has a time factor.

    Going for the mean won’t work- people will still add bad reviews to throw out good.

    Like others have said, members picking their best 10 should make a decent ranking. Least entries in top 10s, lowest ranking. Swap out titles as opinions change. People can still write reviews. Only members on site for more then like two months. Maybe extra few points if actually in their collection. Money is the biggest decider. Probably will have to separate between old and new.

    And SW comic becoming #2 will rush in this new computation of Top 100- because no way, NO HOW is that game in the top 20.

    #95 3 years ago

    Family Guy is rated 81, or something close to it, and it’s one of my favorites. Whoa Nellie is ranked close to the bottom and I love it. Don’t let the “rankings” influence what games you like. Those opinions don’t influence me.

    #96 3 years ago

    No game is a perfect 10, and not even Thunderbirds is bad enough for a 1. I'd give 1 to a plain sheet of wood with a marble on it.

    #97 3 years ago
    Quoted from Trogdor:

    With TZ his highest rating? Come on son.. With all the great games that have come out since. He does have great reviews, but...

    I shouldn’t have said ratings, I meant reviews. I don’t pay attention to the numerical ratings.

    “Son”? Lol I guess I’ll take that over being called sir

    #98 3 years ago
    Quoted from beelzeboob:

    If you give any game a 1, you probably shouldn't be in this hobby.

    You should spend some time on the Brunswick pins from the late 70’s.

    #99 3 years ago
    Quoted from 1963BSARGS:

    You should spend some time on the Brunswick pins from the late 70’s.

    Hey there Daddio! I'd take my Alive over whole buncha "real" machines. It ain't killer, but it's pretty fun especially in competition...

    #100 3 years ago
    Quoted from SLAMT1LT:

    No game is a perfect 10, and not even Thunderbirds is bad enough for a 1. I'd give 1 to a plain sheet of wood with a marble on it.

    I did give a 1 to Big Flipper after playing it at Replay last year. That game deserves it.

    There are 229 posts in this topic. You are on page 2 of 5.

    Reply

    Wanna join the discussion? Please sign in to reply to this topic.

    Hey there! Welcome to Pinside!

    Donate to Pinside

    Great to see you're enjoying Pinside! Did you know Pinside is able to run without any 3rd-party banners or ads, thanks to the support from our visitors? Please consider a donation to Pinside and get anext to your username to show for it! Or better yet, subscribe to Pinside+!


    This page was printed from https://pinside.com/pinball/forum/topic/abuse-of-rating-system-/page/2?hl=tractordoc and we tried optimising it for printing. Some page elements may have been deliberately hidden.

    Scan the QR code on the left to jump to the URL this document was printed from.