Another idea would be to have a top 25 list that is measured differently. Instead of having users rate pins randomly, you have each participant actually rank 1-25 from a list of the top 100 pins. This way, every participant actually ranks all of the machines. You would then simply tabulate scores by assigning values to rank (e.g. 1 point for a first position ranking, 2 for a second position, etc.) From here, the pin with the lowest total would be ranked #1, the next lowest #2, etc.
Not saying to replace the top 100, but this would be an interesting addition to see how the two different methods would compare.