(Topic ID: 285776)

Abuse of rating system.

By rai

3 years ago


Topic Heartbeat

Topic Stats

  • 229 posts
  • 89 Pinsiders participating
  • Latest reply 2 years ago by Doctor6
  • Topic is favorited by 4 Pinsiders

You

Linked Games

No games have been linked to this topic.

    Topic Gallery

    View topic image gallery

    934B57EF-552F-4903-86D9-BE3E432A4159 (resized).png
    8CBF4AC1-0775-40EB-BBB4-D379C84E6ACE.gif
    Keyboard commandos.gif
    04DA4240-C1A2-4B4B-BBD9-1B1FC11F9E4E (resized).jpeg
    IMG_20201231_184528229 (resized).jpg
    05969F5A-388B-4C11-88E0-A6B7419F0ED9 (resized).jpeg
    Let the Hate Flow.png
    BF3689B2-BF92-46C1-9523-47A4BD96D639 (resized).jpeg
    toysrating (resized).png
    5168BF12-AC2F-4403-A3C1-C064798D3E78 (resized).jpeg
    E94AB8F8-EE85-49C1-A898-933DCD69AB8B (resized).png
    31BADAB9-9E73-489A-ACAD-7196BB449B23 (resized).png
    FC93CA11-0D34-4A6B-8D8A-B024454FD023 (resized).png

    There are 229 posts in this topic. You are on page 5 of 5.
    #201 3 years ago

    This was an interesting read. I'd like to know more about how they realize this from the back end. The last scales at the bottom of the article address what many people here are complaining about in relation to score bombing.

    https://www.metacritic.com/about-metascores

    #202 3 years ago
    Quoted from rai:

    I guess we are making 2 big a deal about this but instead of throwing out all the high and low ratings maybe throw out anything that 20% higher or lower than the average.
    For example if something is at 8.0 than anything above 9.6 or below 6.4 gets tossed.

    I still think using the median instead of the average solves everything. Gives a voice to both the fan boys and the trolls, but doesn’t allow either to have undue influence.

    If there are 200 ratings and 100 are below 8.25 and a 100 are above 8.25, then the rating that is used for the rankings is 8.25 (even though the average may be 8.7 or 7.5).

    robin

    #203 3 years ago

    I normally wouldn't weigh in on something like this but the rating system or more specifically the user comments are maybe the most valuable part of pinside to me. They are kind of hidden in the site and took me a while to find. However, when I think I want to buy a new game I read all the comments on the game. I don't really care about the rating number but the comments are invaluable to me in deciding if I will like having a game long term. Yes there are some goofy comments but they are very easy to pick out and skip. I star good ratings but I wish there was a way to promote on the site people who spend a lot of time giving good constructive comments because it is hugely helpful to me. Maybe also a way to highlight users who have a bunch of bad flags...

    For example I was just thinking of picking up a Johnny Mnemonic because I had heard that people really like it but in reading the reviews it was clear that the fun of the game was the speed and reaction time. That didn't really appeal to me so I skipped it but that doesn't mean it was a bad pin.

    Also, as an aside, in my professional life I have had to design a number of numerical looking evaluation systems (design option studies, employee evaluations, etc) and I always think it is funny that once you introduce numbers people can't help but think it is a quantitative process. It is not. If a process has qualitative input then the output can only be qualitative regardless of how it is setup. You can't do math on opinions. It is worth trying to make a system that makes sense with rules and guides but it will never be more than an interesting gauge of how much people like or dislike something.

    #204 3 years ago
    Quoted from killerrobots:

    I skipped it but that doesn't mean it was a bad pin.

    Well for me JM lasted two weeks in my home, and out of all the machines I have cycled through it was the one that I had zero regrets selling and now despise. Why do I hate it so much? The glove ruins the game, and is the worst feature in all of pinball. I can still hear the super slow motor(I had to rebuild) and the slow movement of the glove mech. Ahhhhh! Hate flow killers

    Come to think of it-I need to start making comments under my ratings-JM is a perfect example.

    #205 3 years ago
    Quoted from pinmister:

    Well for me JM lasted two weeks in my home, and out of all the machines I have cycled through it was the one that I had zero regrets selling and now despise. Why do I hate it so much? The glove ruins the game, and is the worst feature in all of pinball. I can still hear the super slow motor(I had to rebuild) and the slow movement of the glove mech. Ahhhhh! Hate flow killers
    Come to think of it-I need to start making comments under my ratings-JM is a perfect example.

    I have never played it so I can't say but there were people who seemed to really like it and others who didn't. For me the comments are the valuable part. As you say, I didn't like it because of the glove. That is helpful. The scores are just silly fun. Everything is about context and you don't get that without comments.

    For instance, AFM (in my collection) is very highly rated and I think on location or for younger/casual players it is amazing. But for serious players in the long-term it gets boring. If someone had teenagers or if someone just wanted something cool for a man-cave. This would be a great choice but for someone interested in competitive play, terrible.

    #206 3 years ago
    Quoted from rai:

    I guess we are making 2 big a deal about this but instead of throwing out all the high and low ratings maybe throw out anything that 20% higher or lower than the average.
    For example if something is at 8.0 than anything above 9.6 or below 6.4 gets tossed.

    This just neuters your scoring scale and squeezes everything to the middle.

    Doing things like grading ratings and weighting how they are counted like Robin has said he already does is pretty much the best framework if you are going to keep the principle of letting 'everyone' rate games.

    This model allows you to do things like throw out that look like junk... and value others that meet metrics you set (detail, completeness, sanity, etc)

    1 week later
    #207 3 years ago

    One big problem I just realized the current rating system has is people can rate games after playing them a few times at an arcade. Many review comments make it very clear that that they've probably played the game for less than hour total in their lifetime, but didn't enjoy it that much so it *must* be bad even though they haven't even taken the time to learn the rules, or maybe the example they played was just in poor shape.

    When writing a review, users should be required to input am estimate of how many hours they've spent playing the game, and that stat should be shown alongside their review just like how they show whether or not the reviewer owns or owned the game. That stat could then be used to determine how much their review affects the overall score of the game.

    #208 3 years ago
    Quoted from SWfan:

    I still think using the median instead of the average solves everything. Gives a voice to both the fan boys and the trolls, but doesn’t allow either to have undue influence.
    If there are 200 ratings and 100 are below 8.25 and a 100 are above 8.25, then the rating that is used for the rankings is 8.25 (even though the average may be 8.7 or 7.5).
    robin

    I would be very curious to see the list with this implemented. Just to get a view of how something like that would work out. I realize I could put it together myself by scraping all the ratings, but... lazy.

    #209 3 years ago
    Quoted from mystman12:

    One big problem I just realized the current rating system has is people can rate games after playing them a few times at an arcade. Many review comments make it very clear that that they've probably played the game for less than hour total in their lifetime, but didn't enjoy it that much so it *must* be bad even though they haven't even taken the time to learn the rules, or maybe the example they played was just in poor shape.
    When writing a review, users should be required to input am estimate of how many hours they've spent playing the game, and that stat should be shown alongside their review just like how they show whether or not the reviewer owns or owned the game. That stat could then be used to determine how much their review affects the overall score of the game.

    Then they’ll just lie and put their ridiculous rating in anyway.

    #210 3 years ago
    Quoted from chuckwurt:

    Then they’ll just lie and put their ridiculous rating in anyway.

    And once again owning a machine does not add anything to the equation, in fact you probably get a less honest take from owners round these parts.

    #211 3 years ago
    Quoted from TheLaw:

    And once again owning a machine does not add anything to the equation, in fact you probably get a less honest take from owners round these parts.

    Owners should be the best to give reviews as long as they are completely honest about the game. They should know the most about it at least. But yeah, I’m sure some just want their games to be the highest rated games for some crazy reason.

    #212 3 years ago
    Quoted from chuckwurt:

    Owners should be the best to give reviews as long as they are completely honest about the game. They should know the most about it at least. But yeah, I’m sure some just want their games to be the highest rated games for some crazy reason.

    8CBF4AC1-0775-40EB-BBB4-D379C84E6ACE.gif8CBF4AC1-0775-40EB-BBB4-D379C84E6ACE.gif
    #213 3 years ago

    Poor Teenage Mutant Ninja Turtles...

    On the chopping block this week after the Premium version has the audacity to debut at a #2 overall ranking....

    Can’t wait to “read” all of the no comments 4.0 ratings that will fly in this week...

    #214 3 years ago
    Quoted from SWfan:

    Poor Teenage Mutant Ninja Turtles...
    On the chopping block this week after the Premium version has the audacity to debut at a #2 overall ranking....
    Can’t wait to “read” all of the no comments 4.0 ratings that will fly in this week...

    But how else are we going to keep the top 100 list accurate if not with such hastily drawn up ratings?

    #215 3 years ago
    Quoted from SWfan:

    Poor Teenage Mutant Ninja Turtles...
    On the chopping block this week after the Premium version has the audacity to debut at a #2 overall ranking....
    Can’t wait to “read” all of the no comments 4.0 ratings that will fly in this week...

    And there it is. A half dozen or so ratings, without comments, in the 4-6 range within days of it being ranked #2...

    Premium will be rated lower than the pro next week as a result.

    #216 3 years ago

    Well, that's no different than the bunch of people who gave it 10s to get it to the top of the list.

    Every time a new machine is released, it jumps to near the top of the list (usually before most people have even seen one), thanks to the couple of hundred early buyers who rate it as "perfect" while they are still unboxing it.

    Eventually those machines settle down into their actual deserved spot...usually.

    -1
    #217 3 years ago
    Quoted from mbeardsley:

    Well, that's no different than the bunch of people who gave it 10s to get it to the top of the list.
    Every time a new machine is released, it jumps to near the top of the list (usually before most people have even seen one), thanks to the couple of hundred early buyers who rate it as "perfect" while they are still unboxing it.
    Eventually those machines settle down into their actual deserved spot...usually.

    I’ve posted this before, but will say it again... it IS different.

    1. The people who rated it a 10 probably bought it, so they had to at least think it was good enough to buy, which likely means an honest rating of at least 9.0.

    2. The ones who rate it a 4.0 or 5.3 definitely don’t own it, may never have played it, and if they did play it, and hated it that much, likely only played 1-2 games on it.

    3. A 10.0 is 1 point above a 9.0 rating. A 4.0 is 5 points below a 9.0 rating. Thus, the 4.0 rating has 5x the impact on the overall rating for the pin compared to the 10.0 rating.

    #218 3 years ago

    Except there are a lot of people who are rating it highly BECAUSE they bought it. Not buying it because they rate it highly.

    Early on, the only people who have played it are the people who bought it on day one. Those are the people that drive it to the top of the list - temporarily.

    Sorry, but TMNT is NOT "one of the best machines ever made" - even though it is currently #2 on the list. Eventually, it will slide down to where it belongs - with the rest of the games that are "pretty good".

    #219 3 years ago
    Quoted from SWfan:

    1. The people who rated it a 10 probably bought it, so they had to at least think it was good enough to buy...

    This is the #1 fallacy you have to get over. Many NIB buyers grab a game as soon as it's out, either never playing it before or playing it little; Like I did with SW.

    Quoted from SWfan:

    2. The ones who rate it a 4.0 or 5.3 definitely don’t own it, may never have played it, and if they did play it, and hated it that much, likely only played 1-2 games on it.

    Actually, this should be the #1 fallacy; it's just ignorant. People like diff'rent things, assuming they must love a game like you did is silly.
    EDIT: But this did remind me to go back and edit my GB rating. The 2nd worst game I've ever owned, bought NIB without playing it, and put hundreds of games on.

    #220 3 years ago
    Quoted from SWfan:

    And there it is. A half dozen or so ratings, without comments, in the 4-6 range within days of it being ranked #2...
    Premium will be rated lower than the pro next week as a result.

    Well the boys over there in the tmnt thread are requesting that we all give it 10s to keep it up. The battle of people abusing the ratings rages on.

    #221 3 years ago
    Quoted from Doctor6:

    Well the boys over there in the tmnt thread are requesting that we all give it 10s to keep it up. The battle of people abusing the ratings rages on.

    You are abusing yourself by caring about this.

    3 weeks later
    #222 3 years ago
    Quoted from mystman12:

    One big problem I just realized the current rating system has is people can rate games after playing them a few times at an arcade. Many review comments make it very clear that that they've probably played the game for less than hour total in their lifetime, but didn't enjoy it that much so it *must* be bad even though they haven't even taken the time to learn the rules, or maybe the example they played was just in poor shape.
    When writing a review, users should be required to input am estimate of how many hours they've spent playing the game, and that stat should be shown alongside their review just like how they show whether or not the reviewer owns or owned the game. That stat could then be used to determine how much their review affects the overall score of the game.

    i see atleast 1 rating on a game the rater thinks will be the best.I can see hoping so but not rating it until you have played that model

    4 months later
    #223 2 years ago

    Why rate over 40 pins all 10 whats the point ?

    #224 2 years ago

    I will say the top 100 is a helpful, relatively accurate ish guide. I still have no idea how some games are ranked so "low" such as batman 66 and Twd. I guess it is because the software sucked when they came out and were never able to overcome it. Wish there was a way we could sort of "start over" on those games.

    #225 2 years ago

    You have to look at it as just one of the handful of tools you can use to gain insight into a particular game. Where they place isnt entirely that important. What people say about a game is more inportant. You’ll find some really good info sometimes but you’ll also find a lot of one line word junk like, “OMG this game is awesome, best ever, a perfect 10.” I am surprised we don’t see more about a games build. Electrical achitecture etc beyond, “This game is built like a tank.” Especially with all the new guys on the scene.

    #226 2 years ago
    Quoted from Jamesays:

    Why rate over 40 pins all 10 whats the point ?

    I seen one guy who voted every single pin as 10. He said like played this at bowling alley and it was fun... Rating 10. I'm talking South Park included.

    Here it is, he rated like 50 pins all 10 out of 10.

    https://pinside.com/pinball/community/pinsiders/fr0zenf00d/ratings

    #227 2 years ago
    Quoted from rai:

    I seen one guy who voted every single pin as 10. He said like played this at bowling alley and it was fun... Rating 10. I'm talking South Park included.
    Here it is, he rated like 50 pins all 10 out of 10.
    https://pinside.com/pinball/community/pinsiders/fr0zenf00d/ratings

    Lol - he gave all 10s except Primus:

    934B57EF-552F-4903-86D9-BE3E432A4159 (resized).png934B57EF-552F-4903-86D9-BE3E432A4159 (resized).png

    #228 2 years ago
    Quoted from rai:

    I seen one guy who voted every single pin as 10. He said like played this at bowling alley and it was fun... Rating 10. I'm talking South Park included.
    Here it is, he rated like 50 pins all 10 out of 10.
    https://pinside.com/pinball/community/pinsiders/fr0zenf00d/ratings

    Except for Primus. Lol

    Quoted from Damonator:

    Lol - all 10s except Primus:
    [quoted image]

    Edit: beat me by 10 seconds

    #229 2 years ago
    Quoted from Damonator:

    Lol - he gave all 10s except Primus:
    [quoted image]

    I mean.... in fairness they're all pretty awesome games lol.

    There are 229 posts in this topic. You are on page 5 of 5.

    Reply

    Wanna join the discussion? Please sign in to reply to this topic.

    Hey there! Welcome to Pinside!

    Donate to Pinside

    Great to see you're enjoying Pinside! Did you know Pinside is able to run without any 3rd-party banners or ads, thanks to the support from our visitors? Please consider a donation to Pinside and get anext to your username to show for it! Or better yet, subscribe to Pinside+!


    This page was printed from https://pinside.com/pinball/forum/topic/abuse-of-rating-system-/page/5?hl=dr-pin and we tried optimising it for printing. Some page elements may have been deliberately hidden.

    Scan the QR code on the left to jump to the URL this document was printed from.