When did you decide that you had it with your mortgage and it had to go? For me personally it was when I had a spell of being unemployed.... I'm freelance and money isn't always a sure thing. Getting rid of a huge monthly expense and not owing ANYTHING to ANYBODY is going to be a huge motivator to get this done. How about you?
I've always hated the idea of being controlled by somebody else. When my parents divorced, they had to sell a big house because a single parent couldn't make the payment. So when I bought my condo, I bought something well within my means. Now that I find myself in my 30s... I'm getting older. I don't want to be an employee forever. Paying off the condo seems like one of the best ways to lower the barrier to having better control of my time on a day to day basis.
Yes I recently got into a "big" argument (lets call it a health discussion) with a friend of mine... he said it made no financial sense to pay off the mortgage with interest rates so low. For me... it's peace of mind.... being freelance my job is even less secure than everyone else in this crazy economy. Knowing that my home is paid for will let me breath a lot easier at night.
I think people love calculating interest rate vs. market return hedges, because it feels very scientific. But the history and the science suggest that (1) such hedges only work over very long time periods and (2) very few people actually invest the difference, so financially it doesn't actually work out. On top of that, calculated risk taking is a huge factor in getting higher net worth. So I think even if you're not measuring it by emotional quality of life, even if you're just looking at it coldly from a numbers-increasing money POV, there's still a very good argument that paying off a responsible home (i.e. that is, or very likely will be 25% or less of your net worth... put another way, valued at < 3X your annual income) will, completely aside from its financials, help a person earn more.