The need to generate publishable papers means that a researcher can only participate in activity that leads to such a paper. He can't try to work on that idea for 5 years, because if no big papers follow, he's toast /he'd probably lose funding long before that).
You have to earn the right to work on your idea for 5 years and get paid. Otherwise we would be funding all kind of crackpots. First you demonstrate you're a good researcher by producing good results. Then you can work on whatever you feel like (either by getting hired at places like DeepMind, or by finding funding sources that want to pay for what you want to work on).
This is what I meant. In our society, only very few, usually already rich, can try their own ideas. Most of us have to stick with known ideas that bring profit to business owners or meaningful visibility to universities. When I was in college, I had to work on ideas approved by my professor. Now I have to work on ideas approved by my corporation. But if I had money, I'd work on something completely different. Sure, in 15 I will be rich and can start doing my own stuff, but I'll also be old and my ability will be nowhere near the peak at 25 years.
What would you work on if you could? Would you say you deserve to be paid for 5 years of uninterrupted research? Do you think you have a decent chance to make a breakthrough in some field? These are the questions I ask myself.
I have some interesting ideas about managing software complexity in general (i.e. why this complexity inevitably snowballs and how we could deal with that), or about a better way to surf the internet (which may be a really big idea, tbh). But all these are moonshot ideas that gave a slim chance of success, while I need to pay ever raising bills. On the other hand I have a couple solid money making business ideas that I'm working on and that will bring me a few tens of millions, bit will be of no use to society, and I have a fallback plan: a corporate job with outstanding pay, but that brings exactly nothing to this world (it's about reshaping certain markets to make my employer slightly richer).
Do I deserve to be paid for 5 years for something that may not work? "Deserving" something doesn't have much meaning: we, the humans, merely transform solar energy into some fluff like stadiums and cruise ships. Getting paid just means getting a portion of that stream of solar energy. There is no reason I need to "deserve it" as it's unlimited and doesn't belong to anyone. A better question to ask is how can we change our society so that all, especially young, people would get a sufficient portion of resources to not think about paying bills.
Chances to make a breakthru are small, but that doesn't matter. It's a big numbers game: if chances are 1 to million, we let 1 billion people try and see 1000 successes. The problem currently is that we have these billions of people, but they are forces by silly constraints of our society to non stop solve fictional problems like paying rent.
When you have tenure, you can work on whatever you want for as long as you want. Nobody works on an idea for five years without publishing anything, though. Progress is made step by step.
Take Albert Einstein as an example, who arguably made one of the largest leap in physics with his theory of general relativity. He never stopped publishing during that time.
When you have tenure, you can work on whatever you want for as long as you want
Not quite. When you are a professor, you essentially become a manager for a group of researchers. You don't really do research yourself. Therefore, your main obligation becomes finding money to pay these researchers. So in reality you can only support the research someone is willing to pay for (via grants, scholarships, etc).
