How to rate a machine

How to rate a machine

By robin

March 13, 2008


9 years ago

What is it?

Did someone ever ask you to list your top 10 favorite machines? Did you immediately know the answer? You probably didn't, right? Rating a pinball machine can be quite difficult, there are many aspects that should be considered. Well, if you want the rating to be fair, that is!

The Pinside came up with a system that allows you to rate a machine simply by judging all the different aspects that are important for a pin: design, artworks, sounds etcetera. In other words, you should reach a fair rating just by telling us what you think about the various aspects of a particular game. Your opinion will lead to your own personal pinball top 10 but also be part our Pinside Pinball Top 100 as well.

Why do I want this?

What's more fun than having a say into a list of the greatest pinball machines ever made, while at the same time discovering how your own pinball top 10 turns out? Well, judging by the close to 10.000 ratings we already have processed at the time of writing this text, people seem to enjoy rating machines quite a bit. Until now more than 500 Pinside users took the trouble of rating 6 or more pinball machines and there are users who have rated more than 100 machines! And because of the rating efforts of our users and the devotion of the Pinside.com team to keep the list clean, the resulting Pinside Top 100 has grown into a respected reflection of what are the most popular pinball machines around. Making the rating of machines a great way to reflect your opinions into the Pinside Top 100. As a bonus every pinball machine you rate will gain you 15 karma points.

First things first

This help text assumes that you've already created a Pinside account. Go here to read how you can create one.

How does it work?

The Pinsiders have created a rating system in which the user not only gives one grade to a machine, but instead a whole array of individual grades, separated into four categories. The four categories are Game Design, Artwork, Sound & Music and the General Aspects. Each one of these four categories contribute equally to the final rating of a machine.

Screenshot: the categories explained

Each of the categories is made up of 4 questions. These questions can be answered as Terrible, Poor, Not so good, Decent, Good or Excellent. The questions are weighted differently within their category. For example: Sound Quality is less important than the Sound Effects and the playfield artwork is more important than the cabinet artwork. So what are the different categories? Let us first introduce them to you, so you have an idea of what you're judging. Don't worry if you can't remember them all. During the rating process all categories will be explained as well.

Game Design

Layout - Placement of parts on the playfield.
The game layout is basically the look and feel of the machine without taking the artwork into account. Are all the areas visible? Are the various items places well on the playfield?

Rules - Clearness of rules and scoring logic.
Are the rules obvious? Does the machine tell you what do with using its lights and sounds? Is the scoring logic done well, or does scoring a jackpot instantly triple your score?

Gimmicks - Playfield toys and special features.
Are there cool toys on the machines that immediately grab the attention. Do the special features make the machine stand out from the rest?

Gameplay - The overall game play.
Is it too fast or too slow? Does it keep you going or is every shot bound to die on a post? In other words, how does the machine make you play?

Artwork

Playfield - The artwork on the playfield
How do like the playfield artwork? Think also about the way the artwork integrates with the light inserts. Think of the signs and plastics.

Backglass - The artwork on the backglass
The artwork on the upright box on the machine.

Cabinet - The artwork on the cabinet sides
The artwork on the front and sides of the machine cabinet.

Animations - The dot matrix display animations
The animations on the Dot Matrix display (if not applicable leave blank).

Sound and music

Sound - The sound effects and speech samples
The creativity, and placement of sound effects in the game. Does the sound reflect the gameplay? Does it fit the theme? Are the sound bites done well if the machine has a licensed theme?

Music - What do you think of the in game music?
Does the music fit the theme? Does it reflect current gameplay? Is it composed well?

Quality - How is the sound quality?
How does the machine sound? Like an old AM radio, or crystal clear?

Variation - Enough variation in the sound and music?
Is there enough variation in the sound and music or do you have to listen to the same samples again and again?

General aspects

Game Lighting (Added April 2004) - The backglass and playfield illumination, the in-game light shows
Is the game well lit? Can you see what to shoot for because of clearly lit inserts? Are the light effects devastating? Don't forget to think of the attract mode illumination sequences as well.

Lastability - Do you want to play over and over again?
Does the machine make you say "One last game" time and time again? Does it keep your interest after 20 games?

Theming - Does the game work out its theme well?
Is the theme done well? Think about the artwork, sound & music and the overall feeling you get from playing the machine.

Fun - The overall fun rating
The overall fun rating of the machine lets you give a final and total judgment over the machine. Maybe you think the graphics and sounds suck, but you still love playing the machine? Then this is the place to let us know!

Start rating!

To rate a machine, go to the 'rating' page of any machine you wish to rate in the 'archive' section. Now find the 'Add your rating'-link and click it to start rating. Note that you can always change a rating by visiting the same page and clicking the 'Edit your rating'-link.

We understand you must be eager to start rating right away, but before you do so, please first have a look at the section about 'Keeping the pinside Top 100 clean' below.

Keeping the pinside Top 100 clean

An unfortunate side effect of having 'a lot of people at your party' is that it's more likely that something breaks. Some people don't want the list to be an accurate reflection of the worldwide average opinion, but rather an overview of their opinion of what is the best machine. And it is this (very small) percentage of 'ballot stuffers' that can sometimes have a big impact on the list. For example, rating all the machines in the top 10 with a low grade, and then rating your own personal favorite with a 10 is an example of this behavior, also known as "troll voting". Regretfully, to avoid these kind of problems, we had we had to take some measures.

Rules of Rating

The first measure we took was adding some 'Rules of Rating' to the site. This way we can delete ratings of the troll voters and point them to these rules as the reason for deletion. Deleting troll ratings really helps keeping the list clean and helps making it a fair overview of what the Pinball Community considers the most popular 100 machines ever created.

So please consider the following simple rules when voting.

  • Take some time for your rating
  • Try to have a nuanced view
  • Only rate a machine if you feel that you have played it well enough to judge it.
  • No "Troll Voting" please! Do not rate down games in an effort to make your favorite game's rating relatively higher.
  • Users found rating machines with multiple accounts will have their accounts blocked and ratings deleted.
  • Users that have only ratings at the top or bottom of the rating scale (basically these are troll votes) will not receive an approved status.
  • Rating comments should be in English - We realize that this is an international site but we want to keep the content universal! You can always use Google Translate to quickly translate your review into English.

In other words, please be fair and realistic! Rating machines with only 10's is not considered nuanced and fair rating.

Validation

Obviously, we can't monitor the submitted ratings any second of the day and so it could happen that one user would mess up the listing with "troll votes" for some hours, before we could delete these ratings and restore the list. It became clear that we needed some kind of validation if we wanted the Top 100 to keep its value.

The first ratings that a user submits will not immediately count in the Top 100. After the user's sixth rating, a ratings administrator will either approve or dissaprove the ratings by holding them against our Rules of Rating. If the ratings are serious, they will be approved and all of a user's past and future ratings will count in the Top 100 calculation! The ratings will still be monitored and abuse might downgrade the status again, pulling all of the user's ratings from the top 100 again. Please note that no Karma points will be granted before your ratings have been approved.

The Pinside Pinball Top 100

The joined results of all the user ratings is joined in a listing of machine rating totals, featuring the most popular 100 pinball machines.

The top 100 in all its beauty.

When will the list be renewed?

The list is compiled every Sunday night at 00:00 GMT. All votes that have an approved status at that moment will count in the Top 100 for that week.

What about my personal top list?

Rating machines will not only contribute to this Top 100, it will also provide you with a listing of your favorite machines. It is great fun to actually see a list of your favorite machines and the results can even be quite interesting. You can find an overview of all your ratings by visiting the your 'Profile' page in your 'My Pinside' section.

I have a question about the Pinball Top 100 not answered here?

Have a look at the Pinball Top 100 FAQ.

Hey there! Got a moment?

Great to see you're enjoying Pinside! Did you know Pinside is able to run thanks to donations from our visitors? Please donate to Pinside, support the site and get anext to your username to show for it! Donate to Pinside