The critique is that you don't naturally know right from wrong and there are plenty of cases of men truly convinced they are virtuous doing horrific things. So you can see why it's helpful for a roman emperor but probably not the best thing for modern society.
Generally - I don't think we at all know what is the most 'virtuous' thing to do either individually or in aggregate as a society. You can see this with the EA movement where people convinced they dedicated their lives to helping (or maybe one day helping) others really are just personally enriching themselves and committing fraud.
We can build or expand - is that virtuous? We can reduce or minimize our impact - is that virtuous? We can seek health, prosperity or purpose for most of us but maybe that dooms some percentage to misery - is that virtuous? I think deep humility that we really don't know and can only incrementally experiment and still mostly be wrong is the best we can do.