It doesn't have to be 24/7, but, please, not everybody has a stay at home spouse to do shopping in the middle of the day.
When I lived in Europe there was a furniture store next to my house. An owner had a large poster glued to the window about how beneficial would it be for the economy of everyone used his store instead of Ikea. Being a huge fan of small local business I honestly tried to stop by. But theres no way I'll take a day off work to do it. Ikea won.
Well, you can leave the office at 4pm if you need to. It's also not uncommon to just take one of your 20+ week days off on short notice.
In many cities there are special days (thursday here) where many large and small shops are open until 9pm; one day doesn't break the bank. Big hardware stores, supermarkets and other larger chains are commonly open 9-9 every day. Smaller stores can not do this, and their owners also want normal work hours like you.
Yeah, it makes far more sense to be open 11-7 or something instead of 9-5 if your clientele will likely also be at work. A lot of smaller restaurants near me are only open for lunch and dinner (11-2, 4-9).