I don't understand people like you who can look at the current numbers and say "Oh, it's not so bad" while also conveniently ignoring that the numbers are growing exponentially. People that are raising the alarm are not talking about the numbers now, they are talking about the numbers in the future if nothing is done to address it.
The numbers are growing exponentially until they don't. You can argue that they would have, had extreme measures not been put in place, and there are reasonable arguments for that. But you have to keep track of the fact that it is an argument you're making and not an observation directly from the data. Otherwise you won't update properly on new information.
For example, one thing I've seen people worry about is that there could be huge numbers of undetected infections; one guy quoted me 1.5 million for the US's 40k known cases. This would be wonderful news if true - death rates and hospitalization rates are 37 times smaller than believed, and the theoretical peak is nearer than we'd thought? But I've universally seen people present these scenarios as bad news, because they've internalized "numbers going to grow exponentially" as an observed fact rather than a contingent conclusion.
In every country where we have seen case load sigmoid out, there have been heavy mitigation and suppression efforts put in place that preceded it by several weeks. Thus far, many European countries and the US have only just started to get serious about this. Therefore I am fairly confident that we will be in the exponential growth phase for a while longer.
One can predict the future with bounded error bars though. Weeks of observation yielded some numbers including R0 around 2 and CFR around one digit percent (yes, we still don't know if it's around 1% or 2%, but it is certainly not 0.1% [1] or 10%), so 0.1---5% of the total population will die somehow due to COVID-19 if absolutely nothing is done. For the reference, less than 1% of the total population dies each year in developed countries. Enough reason to be alerted.
[1] CFR is a function of age (notably among others) and more careful analysis would involve demographics. Still, the rough order of magnitude doesn't change.
Well, why don't we just wait around and sit on our hands until hundreds of thousands are dead and this hidden variable makes its presence known? Sounds like a winning strategy to me.