First off, amazing project!
I know it's a very different environment, but the ATMega interrupt handler wasn't fast enough to reliably latch some of the very brief signals on the Stern driver bus when I was working on the TRON ramps, we had to add high speed latches to buffer them. I have no idea if you'll run into similar issues, but it was a pain to debug, since it was intermittent.