This will be important to understand but I'll assume that the score motor is running because it takes multiple steps of the player unit (and therefore multiple pulses from the score motor) to advance to the next ball from all players except player 4. If the problem happens and the remaining players' lights flash in order (player 2, then player 3, then player 4) and you end up with player 1 on the next ball, that would indicate that the score motor is running.
But either way it will help to eliminate some of the possible paths that may be advancing the player unit.
Target Alpha player unit (resized).jpg
Try blocking switches with a piece of folded paper and playing the game for a while to see if the problem persists. Start with the switch on the AX relay highlighted in red. You may need to add the paper after the game has reset and started a new game. If the problem persists that would eliminate the paths through the S and P5B switches.
Then remove the paper from the AX relay switch and use it to block the switch on the P relay after the game has started. You may need to keep the ball from draining. If the problem persists that eliminates the paths through the coin unit.
If neither of these experiments change anything you may need to start tracing wires along the paths to the Add Player Unit solenoid looking for frayed wires, bent solder tabs, solder blobs, etc.
/Mark