This could have been a good opportunity to show your son a positive social interaction with a stranger.
Quarters on the glass, or a "we stopped in for lunch, would you mind if we played a game before we had to go?" would have probably gone a long way. Maybe the guy thought your son was just watching him play. Some people are just oblivious and a polite "pardon me" gets their attention.