Agreed with both above - it's almost certainly just dirt on the score unit. But could be a broken wire #40 between 1st EB and the score lights. Very unlikely, but possible.
Do the scores flash when the start relay is pressed? As shown in the snippet above, that bypasses the score unit, so you could narrow it down a bit more easily.