Another simple solution is to use some of the "blue tack" putty. Material is like gum and you can break off little pieces and roll 3 small pieces to fit on the inside of the bumper cap. There's enough compression and resistance to easily hold the cap in place. And, like the other easily reversible methods, you can simply take it off and discard with no permanent compromise. I believe the last pack of this material at Home Dump cost under $2.
I also like the putty to help hold down a score card or instruction card without resorting to double sided tape. (a few pieces in each corner)
Happy Holidays, all.
J