I've run into this several times, I do like to preserve the original number whenever possible but sometimes the cabinet needs work in that area.
I sand and prime very lightly and can save it like#4656, but #7181, I used individual number punches to try to recreate and wasn't 100% happy with the results but at least there's a number there.