And for the most part, this herculean effort is wasted. Most people just want to see latest tweets from people they follow. Everything else is fluff to manipulate engagement metrics, pad resumes and attempt to turn twitter into something it's users never wanted.
Most people probably follow more people than they're capable of reading all the latest tweets of, so some sort of ranking/prioritisation makes total sense. And Twitter is ad funded, so they need to also show relevant ads where it makes sense/money.
Just guessing, but a lot of the resources are probably devoted to making money for the business, not padding resumes. Others have pointed it out, but showing tweets doesn't generate revenue without additional infrastructure.