Quoted from DaveH:I've always taken the opposite course. Every extra dollar I pay towards the principal on my mortgage is free money in my pocket. It is a dollar I never have to pay interest on again, so it is all mine. It is like I'm paying myself. And unlike the markets, it is a guarantee. I don't have to wonder what millions of other people do. I don't worry about the market getting spooked. I just keep paying myself.
I play it like goatdan but your reasoning is also sound. Depending on your specific financial situation, appetite for risk, etc. either approach has merits. Also you don't have to be one or the other. You can invest AND pay down the mortgage to get some upside exposure investing and guaranteed return when paying off the mortgage.