Yep had to use a punch on mine. They are a pain to get started. It does not have to come all the way out. Instead of a vise I drilled a small hole in a 2x4 and put the pin over the hole and used a punch. Figured that way I would not mark up the plunger in a vise. Hope this helps.