It will only need 4 balls in the trough no matter what you set the balls per game to.
The optos might test ok, but could still be affected by ball vibration and causing one to activate and the game thinks it needs to put another ball in the shooter lane. The opto decoder board could be having an issue, or a switch matrix problem could be your issue. I would take all the balls out, go into switch test and make sure that when you bang your hand on the trough area that no switches activate. Then feed one ball at a time and make sure each activates as expected including the shooter lane switch and trough eject switches.