I think that pause-by-default is more important than keyboard support. Put another way, I don't think that "play" as in "play a movie" is the right metaphor. I think that by default, the user should control progression forward and back, like scrolling through slides or flipping the pages of a book. Each person is going to want to linger on different steps for different periods of time, because we're all going to want to read different things along with what just changed.
Not sure either way, but if you keep play, the user should be able to click both play and pause without moving their mouse. currently, play appears a couple inches to the left of pause.
For me, this issue is made worse by the fact that the text boxes can pop up basically anywhere... More than once in watching the first story, I only just located a new text box just as it was disappearing, and I had no hope of reading it.