I was told that as a young man, and I almost followed the advice and started a career in metallurgy instead of my actual love, programming. Fortunately I became a programmer, and I've loved doing it for 30+ years, made a pile of money, and my self-image is fine.
So, there's 1 anecdote in favor of following your love.
You have no actual idea what were consequences on your self-image, only a very close person who has seen whole transition, can be critical thinker and brutally honest with you can at least potentially tell some balanced evaluation.
Ie judging by your comment your ego seems a bit inflated here to keep things polite, and that in turn distorts literally everything you perceive, about yourself and everything else.
Also, its trivial to reason that rocket rise of over-compensation of basically all IT work in past few decades compared to literally any other engineering science twists this reasoning, but it fall apart quickly in almost every other scenario. (not arguing against high salaries, there are reasons for them, but at the end anything-IT is just another engineering/science branch, stellar people are everywhere but usually a bit more humble).
So, there's 1 anecdote in favor of following your love.