Quoted from gordonshumway:
But in looking around here, 4 MM seems to be the number that makes you a man! 500K takes 20 minutes - 4MM has to be all day. I must be missing some very big payoffs in here somewhere?
The highest difficulty of the mode you complete (1-star to 5-stars), changes the multiplier you are awarded for your entire mini wizard mode score. I don't remember the exact values, but it's something like 1-star is a 0.5X multiplier up to 5-stars which is a 3X or 4X multiplier. That makes a huge difference on the points you get for the mini wizard mode.
Sure, you can complete a 1-star mode and quickly start Into the Fire, but the points you get in return are not worth it. It's much better to wait until you have a 4 or 5 star mode completed before you start the wizard mode.