I have a little portable recorder (a Tascam DR-05) for events, since it can fit in my pocket and has some decent-sized mics on it for a portable unit, but I've not used any content from it yet for the podcast (I keep forgetting to bring it!).
In terms of background noise, reducing sensitivity on the mic to try and only capture sound near it is the best thing (on the Yeti this is done via a gain control knob; at home I keep it all the way down, but if people are around I have to turn it up the further from the mic they sit).
Beyond that, you can try to clean it up in post, but that works better for pulling out consistent sounds (like a fan humming in the background) versus other people talking.
We used the Yeti to record Keri Wing (first female winner of a PAPA circuit event) for our 10th episode outside in a public area. It serves as a good example of how hard it is to deal with background noise. While the conversation itself is clear, the background noise is also always there (I got a lot of it to the point where you couldn't tell what everyone in the background was saying all the time, but I never was happy with my cleaning effort on it and wish I'd done a better job in both pre- and post-production). On retrospect I should have pulled the gain down even more, but live and learn. Honestly, my recommendation is don't record in loud public places, it is just way too much work to deal with cleaning it up (or accept it and apologize to listeners before playing the clips!).