Easiest thing would be to master your audio before upload.

Each caller gets their own 'track' in a multi-track recording, then adjust volumes, cut out whatever you need to, etc. across all tracks, then normalize and compress. That'll take care of 99% of audio balance issues. The rest is a matter of getting the gain set on each track or running an amplify filter (up or down) to get them all close before the compression and normalization. Export as a single stereo track and you're set. I use Audacity for this purpose - it's free and works on Windows, Mac and Linux.

If you're using Skype (on Windows) for the calling software, I recommend Amolto call recorder, as it automatically splits each caller into their own .mp3 track automatically, as well as providing a combined track.

I don't know what capabilities the anchor interface uses, but I'm willing to bet that it's really streamlined and not tailored for making good quality audio.