Steel wool and soapy water will do wonders. If that doesn't do enough then vinegar or Evaporust soak.

Hailrazer is right on.

Also had luck with Simple Green and Scotch Brite Pads.

Coke and aluminum foil

Yup I use a balled up piece of aluminum and water / simple green . Works amazing

I used to do steel wool and vinegar. I go straight to evaporust soaking now and deal with any little details later. It's a miracle worker. I've tossed a few sets of legs I'm certain evaporust could've saved now that I've used it a bunch.

This is what I do and it works amazingly well.
Leaned it from my old neighbor who would restore vintage bicycles.
This is what they would use on lightly rusted handlebars, rims etc.
I know it sounds crazy, but give it a shot.