Tell them that you check email once an hour (or whatever interval you prefer) and respond to everything in batches. You find that allows you to better focus on your core work and that you're overall more productive that way. If (s)he asks you to change your routine just say no.
Some of my team have fought this battle and lost. If you don't respond, you get a tap on the shoulder which is even more distracting. Unfortunately, the number of people who find this annoying are in the minority, so it is what it is. If it's the worst thing I have to deal with at work, I'll live.
I'd prefer the manager getting up and tapping me on the shoulder even if that is more distracting. Then at whatever review you might have (or not) say that constant interruptions have a negative affect on the work you do. Your manager sounds like a bad one.
I don't want to heap the blame entirely on him. Other people I work with also hold the opinion that emails should be responded to as soon as possible, so he has support. It's all relative, and they certainly pay me well enough for the trouble.
Sometimes yes, sometimes no. Distraction factor also counts. My comment was probably a little arrogant though. The reason I said I'd prefer that was so that the manager would have to expend effort to get an answer immediately, to offset my own distraction. "If I have to be distracted then I'm taking you down with me" kind of thing.
Tell them that you check email once an hour (or whatever interval you prefer) and respond to everything in batches. You find that allows you to better focus on your core work and that you're overall more productive that way. If (s)he asks you to change your routine just say no.
Try it. It's very empowering.