HI Zacaj, I have the same problem but with a System 7 board. It boots fine with driver board disconnected. I tested with Leon's and Marco's Test Rom.

If I try with a known good driver board the MPU will not boot. If I leave the driver board connected, but remove all the PIAs then it boots.

I wonder if it is a problem with a flaky connection somewhere on the MPU where it is connected well enough to get the MPU booting but the added load of any PIA on the Driver board causes a problem.

That didn't take too long to find, the problem was with the interconnect on the MPU board, the pins at one end (the ground pins) did not have a good connection to ground. I must have damaged them when I desoldered the old header. I added a wire bridge to ground and now the MPU boots with the driver board attached. Hope this helps you to find your problem.