Same here. I have been daydreaming about starting from scratch and building up. Of course, everyone's definition of "from scratch" might be different, but for me the starting point would be an FPGA prototyping board. I'd start by implementing my own CPU, memory controller and IO peripherals. Then I'd move onto creating a development toolchain for the platform. Then writing an OS and so on.