It's working through luck more than anything. If you've got a Philips brand 5101 RAM, their spec says data retention voltage can be as low as 1 volt. Most other 5101 spec for retention is down to 2V only.
You've likely got a problem in that reset circuitry CactusJack posted above. The CMOS 4528 chip at Z2 will work down to a voltage around 3V (2 batteries). Lower voltage than this and it's off/unstable.
Looks like Z2 is used to pulse the reset line to the CPU on startup and my guess with Z2 having proper battery power is that it's not getting triggered or it's pulsing too quickly. Capacitors C31 and C32 affect the pulse period.
Diodes CR37, CR38 and transistor Q5 are used to sense valid power on startup and trigger Z2 so it holds the CPU in reset until power is stable.
This is my best guess.

With the game on, what voltages do you measure across diode CR37, zener diode CR38, all legs of transistor Q5 and pins 4 and 11 of Z2?

Those readings look ok to me.
Have you got a logic probe?
With the problem, are the solenoids momentarily thunking on power-up or are they locking on?
Sorry, I've got to head off. Maybe someone else will follow up and/or you could try replacing C31 and C32 as CactusJack suggested.

Hook up the three batteries again to the MPU board.
Connect the logic probe across the batteries so it has power with the game off.
Set the Logic probe switches to CMOS and Pulse and put the probe on pin 6 of Z2.
Power on the game, does the logic probe briefly indicate any change on pin 6 when you power on? If you don't notice anything, power off, keep the logic probe on pin 6 and change the logic probe switch to MEM (memory) and power back on. If there was a change the logic probe will remember it.

If you're not getting any change, Z2 is suspect. If you are getting change the pin 6 pulse could be happening too quickly; C31 and C32 could be suspect.
Good luck.

LATCH switch position on your logic-probe is the same as MEM position I mentioned.
Take it off latch and see if the probe indicators briefly change/flicker at pin 6 when you power on the game and also when you power off. Do the same on the three other output pins of Z2 being pins 7, 10 and 9.
From what I can gather of the circuit:
Output pins 6 should pulse high and 7 should pulse low on power-up
Output pins 10 should pulse high and 9 should pulse low on power-down

Is there battery corrosion on this MPU board around Z2?

Can you tell us what the logic probe indicates when you touch ground (pin 8 of Z2) and also what it indicates when you touch power at Z2 (pin 16)? Your logic probe is an old type with indicators I'm not quite familiar with. Just want to understand if green LED off means low and on means high or if orange LED on means low and red LED on means high.

Can you also tell us what your logic probe is indicating at those four output pins of Z2?
Pin 7 should be the inverse of pin 6, and pin 9 should be the inverse of pin 10.

Ok, so green LED off means your probing a logic "low" level and when the green LED is on you're probing a logic "high" level.

You mean the green LED is off (logic low) at both pins 9 and 10 when the game is on? If yes, Z2 is very suspect. These two pins should not indicate the same logic level at the same time. If pin 9 is low, pin 10 must be high and vice versa. There is a small chance this could be caused by Z1 since Z2 pin 9 goes to Z1 and Z1 might be pulling it low. With the game on a few minutes, do either of these chips get hot to touch?

If the Pulse LED doesn't flash and/or the Green LED doesn't change state when you power the game on and off, that Z2 chip doesn't appear to be doing anything and I would replace it.
If your local electronic store doesn't have any 4528 chips, then just run the game on one battery until you do your next delivered parts order.

Possibly since you had to disconnect the MPU board to replace Z2.

Are the display test modes working?

Are you losing data in the battery backed RAM when it's occurring?