Infinity is a convenience that pays off in terseness. There's constructive mathematics, but it's wordy and has lots of cases. You can escape undecidablity if you give up infinity. Most mathematicians consider that a bad trade.