For some reason you just woke up in your body from 1996, but you still have your mind and knowledge from today. How do you exploit this kind turn of events?
If you tried to build the next facebook, would you be able to do it with the technology of the day? There is no asp.net, php doesn't come out until next year, and java is crude. Even if you did build it, the time just might be wrong. The world may not be willing to accept twitter yet.
Lets say you know enough about tech from the day, and you've found an idea who's time it is. Are you physically capable of executing? I am 23 so in 1996 I was still in elementary school. Domains cost $70, and as a kid, thats a lot.
One major difference for me in the intervening period is the availability of free development platforms. In 1996 I couldn't have coded up a basic LAMP webapp even if I knew how and options for free desktop app development were rather restricted; the idea of, as a relative novice, coding what I could with today's tools but in gcc and a text editor just doesn't bear thinking about. The barriers to entry were just so much higher.
Plus, as you mentioned, if you want to host online then that becomes vastly more expensive too. A good part of the explosion of technology that we're enjoying now is a product of it becoming far more viable for students to play and produce something of comparable quality to commercial endeavours, at least superficially.
All that said:
It took me years to learn that action today is almost invariably better than action tomorrow. Still not always great at putting that knowledge into practice. I'd have drummed that into myself, hard. Tinker. Experiment. Test concepts. Remember that what isn't viable today may be perfectly viable in a few years time as technology and adoption levels improve. Ideas multiply when you execute because you gain a better insight into the possible.
Don't just think, do.