Another vote for the scotch brite flap drill bit. I put them in the vise to hold em down. Coarse one then the fine. Wipe down, then wax and you're good.
Doing it by hand requires more elbow grease than I'm willing to invest. You also bust your fingers on the sharp corners.

If you shine it up to a mirror polish, it will eventually get a line where the ball travels