The drop targets, kickout holes and rollovers mostly fire the B/500 point relay or the C/5000 point relay. Does the Ball Count advance if you manually trip either one of those?
It seems likely that you have a short somewhere which will be a little harder to resolve. Here is the Ball Count circuit:
King Pin Ball Count (resized).jpg
Use this to narrow down where the short might be. For example, if you block the switch on the O/Ball Return relay in the red box with a folded piece of paper, does the problem go away? If so you know that the problem is in that part of the circuit.
Another way to find shorts is to turn down the lights and look for sparks between contacts when the Ball Count unit advances.
If the problem only happens when scoring 500 or 5000 points, there's likely a Score Motor switch involved in advancing the Ball Count at the wrong time. Another approach might be to disconnect the Score Motor service jack, hit a 500 point target, then manually turn the Score Motor slowly to see if you can figure out which switch stack is active when the Ball Count advances.
The whole time you're poking around, look for bare wire, bent solder tabs, solder blobs, etc. that might get unrelated things touching.
/Mark