Forget the cap - that isn't your issue.
Your pictures show the switch leaf hard against the rubber. It should be about 1/4" away from the rubber AT LEAST - probably more.
Then, adjust the switch accordingly.
When the leaf is against the rubber, as in your picture, the switch never gets a chance to rest because the rubber is "springy" and so it will continue to trigger. When the ball belts the rubber you would be surprised how far the rubber deflects.