I tried this with a Raspberry Pi zero, and I basically couldn't get it to work. I noticed it also got very hot very fast.

Are there any challenges that you ran into while building your project

I used this https://www.waveshare.com/10.3inch-e-Paper-HAT.htm

with a Pi Zero W with no issues other than occasional hangups of the signalling but that was solved with a software watchdog.

Here are some pics of my setup. I had to de-solder the Waveshare header and solder on a 90 degree header to get this to fit in the frame in a low profile. (Ugh I wish things were sold without headers soldered)

https://imgur.com/FQlan27

https://imgur.com/rot8knd

As far as getting hot really fast, the first thing I can think of is if you accidentally plugged in the GPIO rotated 180 degrees -- even briefly -- you might have fried one or both boards.

Second thing I can think of, although it's more likely to just damage the e-Ink rather than make things become hot -- is e-Ink displays don't like to be hot-plugged/unplugged -- you should have all the connections plugged in before switching the system on.

Can you set this up as a calendar?

Yeah, but it involves a lot of development. I made this a few months back : https://github.com/ugomeda/esp32-epaper-display

I'm working on a more "user-friendly" version :)

Yeah you can do whatever you want with it, it just displays an image, so as long as you can generate the image you want you can load it up.

Just note that it takes a couple seconds to refresh so don't expect too much in the way of animations.