You can do this with the Google Keyboard. Long-press the comma on the left side to open the settings menu, and then it's Preference > "Show emoji key switch". Set that to off and it should hide it under the same comma key long-press.

Edit: I should say this is Android 10 on Pixel 3a, not sure about other versions/manufacturers.